Be alarmed—avoid catastrophies

Issue 20/2010
www.funkschau.de
October 22, 2010
Issue 20/2010 • October 22, 2010 funkschau
Everything at a Glance!
Four Monitoring Solutions Put to the Test
REPRINT
-1-
funkschau Issue 20/2010 • October 22, 2010
Be alarmed—avoid catastrophes
By Dirk Jarzyna
Network monitoring software helps identify errors and
malfunctions in the network and on network devices
before they lead to a possible catastrophe. They are also
useful for capacity planning. funkschau tested four of
the best known examples in this area.
A lot has been going on in the field of
network monitoring software in the past
12 months. In particular, the majority of
manufacturers have further expanded support for virtual environments, overhauled
user interfaces and added many new functions and features. But in the final analysis,
not all products have actually improved:
some have revealed weaknesses which
render them unsuitable for many environments. The products tested in this roundup
were Ipswitch’s “WhatsUp Gold Premium
14.3,” ManageEngine’s “OpManager 8.7,”
Paessler’s “PRTG Network Monitor 8” and
SolarWind’s “Orion Network Performance
Monitor 10.0”.
Paessler PRTG Network Monitor 8
Your network at a glance with software
that’s “Made in Germany.” “Fast installation, intuitive operation,” a statement that’s
not just prominently displayed on Paessler’s
website, but one that’s right on the money
too. Setup on a Windows computer took
less than five minutes, after which the software itself could be used without incident,
even by a newcomer to network monitoring, without much need for familiarization
or even training. So far, so good. Of course,
whether the program installs five minutes
sooner or later is neither here nor there, but
no network administrator wants to spend
long searching around in the management
interface or looking things up in the manual
in order to complete run of the mill—or
even less usual—tasks.
PRTG Network Monitor reports on the current status of the network and connected
devices, but also gathers data in order to display long-term trends. Information gathered
over 30 days, for example, helps network
experts to identify bottlenecks and react by
increasing or redistributing capacities or implementing other changes. PRTG Network
Monitor is particularly suited to monitoring
device availability, bandwidths, and network
resource loads—and not just on a single
LAN. Remote probes allow centralized monitoring on several networks at once, and can
also be used for load sharing during CPUintensive processes such as packet sniffing
or NetFlow monitoring. Remote probes
open up a wide variety of applications to
the PRTG user. The software can conceivably be used to monitor customer networks,
all branches of an organization, or separate
networks within a company (e.g. LAN and
DMZ). But a complex infrastructure doesn’t
mean that PRTG Network Monitor is complicated to use, because all this monitoring
requires no more than a single PRTG Core
Server. The product scales up to 30,000 sensors per installation.
What remote probes do not do is increase
the availability of the PRTG monitoring system. For that there now is the PRTG Cluster
Failover Solution included with version 8.
This allows for up to five instances of PRTG
working together: one primary and up to
four secondary servers. This type of PRTG
cluster offers several additional advantages:
100% uptime even during software updates and server crashes, as well as automatic failover, means this solution can have
multiple points of presence monitoring.
All servers are constantly monitoring all of
the sensors. This is interesting not just from
the point of view of error tolerance, but also
because it allows the administrator, for example, to measure and compare response
times from various locations in the network
and to calculate aggregate up and downtimes.
-2-
PRTG Network Monitor monitors complex, distributed networks
with just one PRTG core server. One installation of the product
supports up to 30,000 sensors.
KEY FACTS
PRTG Network Monitor 8.0
Manufacturer: Paessler
Type: Monitoring software
Price: 100 sensors, 1 core server €357,
unlimited license (unlimited number of
sensors, 1 core server) €4,165
Web: www.paessler.com
Pros/cons:
Complete package
Simple setup and
configuration
Very good priceperformance ratio
Issue 20/2010 • October 22, 2010 funkschau
PRTG clustering is not expensive: all PRTG
licenses immediately permit a single failover
installation in which two PRTG instances
work together. Additional licenses are only
required for three or more nodes.
The product offers a total of four user interfaces: an Ajax-based web interface, a
slimmed-down HTML interface for older
browsers and mobile devices (IE 6/7, Android, Blackberry), a Windows GUI, and an
iPhone app. The most frequently used interface, Ajax, is elegant and simple to operate.
The welcome page contains a prominent
button for the most important tasks to be
carried out immediately after installation,
including auto-discovery of networks and
connected devices. Discovery in the local
network through an IP address range of 0
to 255 was done at lightning speed and
was complete and accurate when tested.
The program can also install the necessary
sensors for monitoring during the discovery
process, if desired. Discovery takes more or
less time depending on the number of sensors selected. Additional sensors are very
easy to add to selected devices or groups
afterwards.
The range of sensors included was already
impressively large in previous versions. Sensors for monitoring all the usual network
services (ping, HTTP, HTTPS, POP3, DNS …),
for QoS, Radius Server, SLAs, Exchange and
Syslog servers, as well as support for SNMP
and WMI, packet sniffing, NetFlow and
sFlow have long been standard. Virtualization continues to be popular. Therefore, the
latest version of PRTG contains a range of
new and/or extended sensors tailored especially to this which allow monitoring of the
hardware information from a VMware-ESX/
ESXi server via WBEM, monitoring of a virtual machine (VM) on Xen servers, and HyperV storage device monitoring. The WMI sensors and special Linux and Mac OS sensors
have also increased in scope. Paessler has
tested sensors with the most current distributions. But strangely, Red Hat does not
appear on the list published on Paessler’s
website, while Fedora does.
The major components of a PRTG installation are the PRTG server for data storage
as well as one or more probes. The probes
carry out the actual work. They connect to
the server automatically, load the sensor
configurations determined for them, and
carry out the monitoring functions. As the
probes initiate connections to the server, a
server or connection failure does not affect
monitoring. The first “local” probe is already created by the setup program. It runs
on the PRTG server and monitors all sensors
in the system. The hierarchical organization
of the probes, device groups, and devices
makes system administration easier. This allows many settings, such as login information, to be transmitted from probes via device groups to individual devices by means
of inheritance.
PRTG Network Monitor provides clear
acoustic and visual notification of new
alarms, warnings, and other messages.
Upon accessing a notification or selecting
an affected sensor, the program provides
very good explanations of what the problem
actually is and what the individual values
and adjustable parameters mean. Several
dashboards, diagrams and lists represent
current system statuses and historical data
in a meaningful way. The filter options are
excellent. Report generation has meanwhile
been decoupled from the web server and
user interface, so that the user experience
is no longer affected. Anyone who wants
to can produce network maps that visually
represent the monitored network or parts
thereof—unfortunately only on a manual
basis. To this end there are approximately
280 different icons available, representing
network devices which can be connected to
each other using (network) lines. The program shows alarms, warnings, etc. underneath the icons. Creating a map of this kind
is somewhat cumbersome, however. We
would like to see a function that automatically includes the selected devices or device
groups in a map.
PRTG Network Monitor 8 meets all the demands we make of a monitoring product.
The product is complete, easy to install,
flexible to use, easily scalable and provides
good levels of notification and explanation.
It also isn’t as hard on the pocket as many
competing products. During testing it impressed us with its high levels of usability,
reliability, and precision.
Ipswitch WhatsUp Gold Premium 14.3
WhatsUp Gold is another popular network,
server, and application monitoring product that has received a lot of praise. So we
were all the more disappointed by WhatsUp Gold’s performance during testing. We
can’t agree with manufacturer Ipswitch’s
claim that the product is “simply the most
intuitive, complete and cost-effective network management toolset available today”.
Nothing was particularly intuitive, the product was only complete with the addition of
extra applications or plug-ins, and in view
of the underwhelming performance that
WhatsUp Gold Premium gave in its basic
version, the software is too expensive.
-3-
The web interface in WhatsUp Gold is quite pleasant. Less
pleasant however are the many empty fields.
KEY FACTS
WhatsUp Gold 14.3
Manufacturer: Ipswitch
Type: Monitoring software
Price: from €1,591 (Standard Edition,
up to 100 devices), €6,715 (Distributed
Edition, up to 500 devices). Prices do not
include optional plug-ins. Prices for more
than 500 devices on request.
Web: www.whatsupgold.com
Pros/cons:
Highly scalable
Variety of editions and add-ons
offers flexibility, makes it difficult to
choose a product
Requires the Remote Site Edition for
remote sites
WhatsUp Gold Premium executes an automatic device discovery, carries out SNMP
and WMI monitoring, produces an automatic graphic representation of the network, runs actions automatically when the
status of a monitored device changes or a
threshold value is exceeded, generates reports and notifies administrators of alarms.
In the Premium Edition, this all functions
on a single network which can, however,
be as big as you like. In principle, WhatsUp Gold can also scale over distributed
networks, but for this other editions are
required. Customers must carefully consider what functionality they require and
which devices or networks they actually
want to monitor, as the different WhatsUp
Gold editions vary in terms of performance,
functional scope and, of course, in price.
funkschau Issue 20/2010 • October 22, 2010
This doesn’t make choosing easy. For example, anyone who needs a product that
performs the same functions as PRTG Network Monitor can immediately forget about
the entry-level product, WhatsUp Gold Premium.
The entire setup of WhatsUp Gold Premium takes quite a long time. The reason
for this is partly because a Microsoft SQL
server must first be installed, if not already
present. The setup routine installs Microsoft
SQL Server 2005 Express Edition if it doesn’t
find any other SQL server. Following installation a Windows GUI starts up. A large “Start
Here!” button in the Welcome Center
shows what follows next. With one click the
Quick Start Assistant starts up to configure
the Administrator Notification (e-mail) and
the network login information (for SNMP,
Windows, ADO, Telnet, SSH and VMware).
Once this has been completed, the discovery process starts. This is done by the assistant via IP range scan, SNMP smart scan,
VMware scan, or host file scan. The devices
found should then be added by the administrator to the WhatsUp Gold database.
The results of discovery in the test network
were not encouraging. While the IP range
scan, for example, did discover all devices in
the network, the program clearly had some
difficulty identifying details such as the type
of device (Windows station, Windows server, Linux server, Switch, etc.). Just like PRTG
Network Monitor, WhatsUp Gold can configure both active and passive monitors for
discovered devices. To do so, however, the
discovery process must identify the respective device roles—and that’s exactly what it
didn’t really do in our test. That meant a lot
of manual work later on that no administrator would want to take on, especially as
almost no administrator has the necessary
time for it. With previous versions of WhatsUp Gold, discovery and monitor configuration functioned perfectly. Why it isn’t working in the current version remains a mystery.
The administrator connects monitors (in
PRTG these are the sensors) with standard
procedures which describe a range of actions that WhatsUp Gold carries out as soon
as an error or change in status occurs on a
device. For example, when an error is identified, WhatsUp Gold sends an e-mail notification, plays a particular sound, or opens
a pop-up message on a selected computer.
WhatsUp Gold actively polls devices in the
network in order to determine changes in
status. For this the program uses the aforementioned preconfigured monitors or those
created by the administrator. Performance
monitors observe the resources of a device,
such as disks, interfaces, and memory.
Depending on the answers received,
WhatsUp Gold carries out actions, such as
reporting to the administrator or restarting
a service. The Alert Center in the web GUI
displays consolidated alert information and
simplifies the management of notification
procedures. Administrators are kept up-todate on what’s happening in their network.
But the Alert Center does not provide explanations with as much detail as the comparable feature in PRTG Network Monitor. Apart
from that, this interface does not provide as
clear an overview.
Neither the Windows GUI nor the web application is as intuitive to operate as the
interfaces in PRTG. When the message “Interface down” was received, for example,
it did immediately find detailed information
on the problem. Overall, there are simply
not enough automated tasks in WhatsUp
Gold for our liking—the program leaves too
much to be done by the administrator. For
example, device information such as MAC
addresses and operating system names and
versions must be filled in manually. However
it is possible that this has something to do
with the failed discovery/monitor configuration—we just don’t know.
New in version 14.3 are ready-made views
that support tasks such as remote site management. Existing monitoring settings can
be easily copied to new devices, which does
make the devices available somewhat more
quickly. Ipswitch has made some changes
to database storage and optimization and
information retrieval technology, which
improve the performance. In terms of databases, there is also additional support for
standard SQL clustering.
ManageEngine OpManager 8.7
Here we have a monitoring product that
really didn’t do anything for us. ManageEngine is indeed a specialist in management
applications, and OpManager, the company’s network monitoring software, is the
first choice for more than 700,000 network
administrators in 93 countries, according to
the manufacturer. But after repeated testing of the software, we still really couldn’t
work out why this should be. It may well be
that 25,000 IT administrators download the
product each month—we’ve done it too—
but whether they actually use it after initial
testing, or put it back in the virtual drawer,
is not as easy to find out… The product is
very powerful overall and thoroughly capable of monitoring networks, network de-
-4-
The web interface of OpManager uses widgets, making it highly
customizable, but the system is very sluggish.
KEY FACTS
OpManager 8.7
Manufacturer: ManageEngine
Type: Monitoring software
Price: €1,995 for the Professional Edition
(up to 100 devices), €27,995 for the
Deluxe Edition (up to 2,000 devices).
Prices do not include optional add-ons
and plug-ins. Prices for more than 2,000
devices on request.
Web: www.manageengine.com
Pros/cons:
Highly customizable user interface
A lot of manual configuration needed
Errors in device classification
vices and services, identifying performance
bottlenecks, notifying administrators and
generating reports, but it’s quite an effort to
set up the software so that it operates the
way an administrator wants. There are some
products that make it easier for the user.
For a start, prospective users should be
aware that this is a product that only actually delivers useful information when all devices to be monitored support SNMP. This
also applies to WhatsUp Gold to a certain
extent, but the effect is not as severe as in
OpManager. While it’s good when a product makes active use of standards, but being exclusively based on SNMP is surely not
the way. For our part, we simply cannot imagine a professional network environment
that has each and every device switched to
SNMP, or even supports such a thing.
Issue 20/2010 • October 22, 2010 funkschau
But this would have to be the case to enable
complete management of the network and
the devices used in it.
Like so many manufacturers, ManageEngine also offers OpManager in several editions, with some additional add-ons and
plug-ins. It’s not easy to determine which
editions contain which expansions. This
makes it difficult to calculate a final price
for the product. In any case, it starts from
free. However the free version only supports
a maximum of ten nodes and is therefore
only suitable for short product tests. For the
fully-featured Distributed Edition, suitable
for monitoring distributed networks, you’ll
need to hand over all of $9,995. This price
applies to monitoring of up to 250 nodes.
What isn’t clear is whether plug-ins such as
Cisco IPSCA or IPSLA monitor, the NetFlow
Analyzer or the NCM plug-in are already included.
The program setup took quite a while in
our test. OpManager uses a Microsoft
SQL server or MySQL server as a database.
ManageEngine includes MySQL, which was
fortunate in our case, as we simply couldn’t
get OpManager to function with our Microsoft SQL Server 2008. We also had no luck
in using OpManager or the program’s web
console on a Windows 7 computer.
Here we encountered serious compatibility problems. Installed on a Windows 2003
server, it eventually worked “so-so”.
After the first run of the web client, the program wants to carry out auto-discovery. OpManager supports automatic discovery and
smart classifications with device and interface templates, mass imports, and process
templates.
Initial discovery does require a little manual
input, for example the administrator must
select the services to be discovered, including DNS, web, SQL, HTTP, and POP, and then
specify an IP range. The subsequent discovery was very time-consuming, even on just
one subnetwork over a range of 0 to 255.
The recognition rate was okay, the classification for devices that supported SNMP
was half-decent. We say “half-decent,”
because we object to finding all our Windows 7 desktops classified as servers. These
errors can be rectified, but other test candidates have shown that there is no need
for such errors to arise in the first place. Another thing: if a program cannot manage to
classify devices that lack SNMP support on
the terminals, it still shouldn’t limit itself to
only reporting the IP addresses of the devices. Why not do a short DNS query or use
NetBIOS names, like other programs do?
At least then we would have some idea of
the device in question.
The pleasant-looking and easy to use client web application uses standard Internet
Explorer. Used with this browser, it’s not
exactly fast. OpManager uses lots of Java—
and between the frequent page changes,
the administrator can happily pop out for
a coffee without missing anything. The application makes extensive use of widgets.
In previous versions of OpManager, this
caused problems for some browsers which
have since been rectified. Widgets are okay,
because, among other things, they allow
administrators to configure the interface
whatever way they want.
Without intervention by an administrator, OpManager monitors next to nothing.
Almost all monitors must first be configured and assigned to devices. Fortunately,
templates make this process easier. Those
prepared to make the effort will finally be
rewarded with a system with good fault
management, performance, and device
monitoring. The program produces respectable real-time graphics, historical reports on
availability, utilization, response times, and
inventory. The WAN monitoring is limited to
monitoring the availability of WAN links, reports on performance analyses, and capacity planning. The administrator will only gain
complete functionality with the addition of
the optional WAN monitor add-on.
The alarm system is serviceable, if a little sluggish. But the program provides almost no explanation of alarms, adjustable
parameters, etc. What is an alarm such as
“ColdStart: zero” supposed to mean to an
administrator? Administrators must already
be very familiar with it in order to gain much
benefit from this product.
SolarWinds Orion Network
Performance Monitor 10.5
Orion Network Performance Manage, Orion
NPM for short, is (as its name suggests) focused on monitoring network performance.
If an administrator also wants to keep an
eye on the performance of network applications or manage the network configuration,
they’ll need to use a separate product and/
or module that is, of course, sold separately.
NetFlow traffic analysis, IP address management, and IP SLA management are also
only possible with optional extensions. Even
without extensions, Orion NPM is already
a very complicated product that requires a
lot of patience right from the setup. There
are about 380 MB to be unpacked and
installed—and it takes time. One can only
-5-
Orion NPM is a very expensive product, and administrators can
only access its full functionality by purchasing optional extras.
KEY FACTS
Orion Network Performance
Monitor 10.0
Manufacturer: SolarWinds
Type: Network monitoring software
Price: 100 elements €2,015, unlimited
elements €20,350, Prices do not include
options such as Enterprise Operations
Console or scalability engines.
Web: www.solarwinds.com
Pros/cons:
Flexible alert system
Tedious set-up
Prices
Additional scalability engines may be
necessary in large distributed networks
hope that the setup works first time and
doesn’t—as happened to us—simply trail off,
due to packet errors, for example. We finally got Orion NPM to work on a Windows
Server 2003 (SP2) with .NET Framework
3.5 and the Microsoft SQL 2005 Database
(Express Edition) that Orion NPM installed
itself.
A standard installation is designed to monitor approximately 1,000 nodes. While Orion
NPM scales up significantly higher, it is however advisable to install additional standby
engines, multiple polling engines and/or additional web servers. For high availability environments, purchase of the Orion Failover
Engine is also recommended. The additional
servers/engines and the Failover Engine are
purchased in the form of so-called scalability engines, with prices starting at €5,700.
funkschau Issue 20/2010 • October 22, 2010
For distributed networks with multiple instances of Orion NPM, the Orion Enterprise
Operations Console operates as the command center. Unexpectedly, the EOC is an
optional component costing an additional
€4,000.
The software setup took quite a while in
our test. SolarWinds says that the product
is “up and running” in less than an hour.
It didn’t take an hour, but compared to
other products, Orion NPM crawled along
at a snail’s pace. Like WhatsUp Gold, Orion
NPM demands a Microsoft SQL Server and if
necessary installs Microsoft SQL Server 2005
Express Edition. The latter proved fortunate
in our test, as we had no luck with a previously-installed Microsoft SQL Server 2008
Express Edition. Orion also expects to find
functioning Internet Information Services
and .NET Framework. Checking installation
requirements and the additional software
required is nothing new and generally not
a problem, but PRTG and OpManager demonstrate that it doesn’t have to be so.
Orion NPM offers two user interfaces, a
Windows GUI and a web console. Most administrators will use the web console, which
is well designed and easy to master. But
this interface is not as customizable as the
one from ManageEngine, although it’s just
as sluggish. In addition to these user interfaces, numerous applications also appear in
the Windows start menu after installation;
these serve to customize the Orion NPM
installation or maintain the database, for
example.
Conclusion
If you want comprehensive monitoring in
an enterprise network, you’d better be
very careful. It’s all too easy to fall into a
“cost trap,” where a product that seemed
cost-efficient has to be extended at great
expense. The majority of manufacturers
tempt purchasers with inexpensive “standard editions,” which can often do nothing
more than monitor a couple of nodes in an
individual network segment. If expansion is
required later, for example for distributed
monitoring, NetFlow or SLA monitoring,
the manufacturers strike and charge prices
which elsewhere would get you almost a
complete package covering all eventualities.
Take Paessler, for example. Paessler’s PRTG
Network Monitor 8.0 is a complete, highlyscalable and very easy to operate product
at a price at which experienced administrators would not expect to see a professional
monitoring solution in the enterprise class.
But as our test showed, appearances can
be deceiving. Among the four products
tested, PRTG Network Monitor 8.0 shone as
the most fully-featured and easiest to use
monitoring package at an acceptable and
transparent price.
In terms of functionality, the other products were fully on a par with PRTG Network
Monitor—but often only after the installation of separate products, add-ons, or plugins.
Ralf Ladner
© 2010 WEKA FACHMEDIEN GmbH
The initial discovery uses SNMP and ICMP. In
a local sub-network with 255 nodes, it’s relatively quick to run. The product functions
precisely and even identifies every individual
network interface and all protocols running
on them. The alert system is flexible, like the
discovery it operated perfectly in our test
and is easy to use. Like the other products,
Orion NPM generates alerts when an event
occurs or a threshold value is exceeded. The
program offers a range of options for reacting to alerts, including the usual notification
options, execution of an automatic script or
program, and an escalation sequence. Configuring network alerts is not particularly
difficult. The product allows administrators
to define device dependencies and to configure alerts for contiguous events and/or
SolarWind’s continuous statuses over a certain period. For example, the system can be
set up so that it doesn’t produce an alert
immediately if CPU utilization exceeds 90
percent, but only if this utilization lasts for
more than five minutes.
-6-
TEST PROCEDURE
Monitoring Software
The test products were installed on a
network on which several Windows
Server 2003/2008 machines; one
Exchange Server and one SQL Server
are operating. The machines within
the network were connected via Fast
Ethernet switches and WLAN routers; connection to the internet was
through an ADSL router. The client
machines worked with various operating systems, including Windows
XP, Windows 7, and Linux. The services and protocols operating on the
network included TCP/IP, DNS, POP3,
SMTP, IMAP, SNMP, HTTP, HTTPS, and
FTP.
Following initial installation and configuration of the monitoring programs, we allowed them to search
the network and gather information
about the installed services and protocols for a time. After that, threshold values were set and actions to be
carried out were defined. We examined whether the programs identified when a threshold value was exceeded, a system status changed and
performance was interrupted, and
if they reacted as intended. Among
other things, we evaluated the priceperformance ratio, user-friendliness,
and the way in which the products
supported monitoring of physically
distributed networks.