1 How to Monitor Performance Contents

1 How to Monitor Performance
Contents
1.1. Introduction .....................................................................................................................
1.1.1. Purpose of this How To ........................................................................................
1.1.2. Target Audience ...................................................................................................
1.2. Performance - some theory .............................................................................................
1.3. Performance - basic rules ................................................................................................
1.4. Recognizing some common performance problems ...........................................................
1.5. Monitoring, and optimizing, the performance .....................................................................
1.5.1. Tools and mechanisms .........................................................................................
1.5.2. Interpreting the request.log ....................................................................................
1.5.3. Caching ................................................................................................................
1.5.4. Analyzing Search ..................................................................................................
1
1
1
1
3
3
4
4
5
7
9
1.1 Introduction
CQ5 encompasses several applications, and interacts with several more.
Performance (or the lack of it) is one of the first things that your users notice, so as with any
application with a user interface, performance is of key importance. To optimize the performance of
your CQ5 WCM installation various attributes of the behavior should be monitored.
1.1.1 Purpose of this How To
To give a short introduction on what to monitor, and how, when optimizing the performance of your
CQ5 implementation.
Note
This document deals primarily with the mechanics of monitoring performance. Issues
such as setting the target metrics of the performance you want to achieve are covered
elsewhere. See the CQ5 WCM Project Manager Guide for more information.
1.1.2 Target Audience
• Power Users
• System Administrators
• Project Managers
1.2 Performance - some theory
The problems that cause performance issues are often difficult to track down, even when their
effects are easy to see.
A basic starting point is a good knowledge of your system when it is operating as normal. If you
don't know how your environment "looks" and "behaves" when it is performing properly, it can
be difficult to locate the problem when performance deteriorates. This means that you should
spend some time investigating your system when it is running smoothly and ensure that collecting
performance information is an ongoing task. This will provide you with a basis for comparison
should the performance suffer.
Page 1 of 9
CQ 5.1 WCM
Copyright 1993-2008 Day Management AG
How to Monitor Performance
The following diagram illustrates the path that a request for CQ5 content can take - and therefore
the number of different elements which can impact the performance.
Figure 1.1. CQ5 request - the web-chain
Performance is also a balance between Volume and Capacity:
Volume
the amount of output that is processed and delivered by the system.
Capacity
the system’s ability to deliver the volume.
This can be illustrated in various locations throughout the web-chain.
Page 2 of 9
CQ 5.1 WCM
Copyright 1993-2008 Day Management AG
How to Monitor Performance
Figure 1.2. Capacity vs. Volume
There are several functional areas which are often responsible for impacting the performance:
• Caching
• Application (your project) code
• Search functionality
1.3 Performance - basic rules
Certain rules should be kept in mind when optimizing performance:
1. Performance tuning must be part of every project.
2. Do not optimize early in the development cycle.
3. Performance is only as good as the weakest link.
4. Always think about capacity vs. volume.
5. Optimize important things first.
6. Never optimize without realistic goals.
Note
Bear in mind that the mechanism you use to measure performance will often affect
exactly what you are trying to measure. You should always try to account for these
discrepancies, and eliminate as much of their effect as possible; in particular browser
plug-ins should be de-activated wherever possible.
1.4 Recognizing some common performance problems
The following lists common performance issues which occur, together with proposals on how to
spot and counteract them.
Page 3 of 9
CQ 5.1 WCM
Copyright 1993-2008 Day Management AG
How to Monitor Performance
Table 1.1. Recognizing common performance problems
Area
Symptom(s)
To increase capacity...
To reduce volume...
Client
High client CPU usage.
Install a client CPU with
higher performance.
Simplify (HTML) layout.
Low server CPU usage.
Upgrade to a faster browser. Improve client-side cache.
Some clients fast, some
slow.
Server
Network
CPU usage low on both
servers and clients.
Remove any network
bottlenecks.
Improve/optimize the
configuration of the client
cache.
Browsing locally on the
server is (comparatively)
fast.
Increase network
bandwidth.
Reduce the "weight" of
your web pages (e.g. less
images, optimized HTML).
Cluster your web-servers.
Reduce the hits per page
(visit).
Web-server CPU usage on the webserver is high.
Use a hardware loadbalancer.
Application Server CPU usage is high.
Cluster your CQ5 instances. Search for, and eliminate,
CPU and memory hogs (use
code review, timing output,
etc).
High memory consumption.
Improve caching on all
levels.
Low response times.
Optimize templates and
components (e.g. structure,
logic).
Repository
Cache
1.5 Monitoring, and optimizing, the performance
Performance issues may stem from a number of causes that have nothing to do with your website,
including temporary slowdowns in connection speed, CPU load, and many more.
It may also impact either all your visitors, or only a subset of them.
All this information needs to be obtained, sorted and analyzed before you can optimize the
performance.
If you experience a performance issue:
• try to replicate it: with one (or preferably more) standard web-browsers, on a different client that
you know has good general performance and/or on the server itself (if possible)
• check whether anything (related to the system) has changed within an appropriate time-space,
and if any of these changes could have impacted the performance
• collect as much information as possible to compare with your knowledge of the system under
normal circumstances
1.5.1 Tools and mechanisms
The following gives a short overview of some of the tools available for monitoring performance.
Page 4 of 9
CQ 5.1 WCM
Copyright 1993-2008 Day Management AG
How to Monitor Performance
Note
Some of these will be dependent on your operating system.
Table 1.2. Tools and mechanisms for monitoring performance
Tool
Used to analyze...
Usage / More information...
request.log
Response times and
concurrency.
Interpreting the request.log.
truss/strace
Page Loads
Unix command. Include the misc.truss log level to
INFO.
CQ5 status
monitor
Monitor CQ5 requests;
including request-type,
long-running requests,
pending requests.
Thread dumps
Observe JVM threads. Dependent on the operating system, e.g. kill -QUIT
Identify contentions,
<pid> on Unix/Linux whereas Ctrl-Break on Windows.
locks and long-runners.
System calls
Identify timing issues.
Calls to System.currentTimeMillis() or
com.day.util.Timing are used to generate
timestamps from your code, or via HTML-comments.
Note: These should be implemented so that they can
be activated / deactivated as required; when a system is
running smoothly the overhead of collecting statistics will
not be needed.
Apache Bench Identify memory leaks,
selectively analyze
response time.
For full details: http://httpd.apache.org/docs/2.0/programs/
ab.html; basic usage is: ab -k -n <requests> -c
<concurrency> <url>
Search
Analysis
Execute search queries Analyzing Search.
offline, identify response
time of query, test and
confirm result set.
JMeter
Load and functional
tests.
http://jakarta.apache.org/jmeter/
JProfiler
In-depth CPU and
memory profiling.
http://www.ej-technologies.com/
truss/strace,
lsof
In depth kernel call and Unix/Linux commands.
process analysis (Unix).
1.5.2 Interpreting the request.log
This file registers basic information about every request made to CQ5. From this valuable
conclusions can be extracted.
1.5.2.1 Monitoring traffic on your website
The request log registers each request made, together with the response made:
09:43:41 [66] -> GET /author/y.html HTTP/1.1
09:43:41 [66] <- 200 text/html 797ms
By totaling all the GET entries within a specific periods (e.g. over various 24 hour periods) you can
make statements about the average traffic on your website.
1.5.2.2 Monitoring response times with the CQ5 request.log
Page 5 of 9
CQ 5.1 WCM
Copyright 1993-2008 Day Management AG
How to Monitor Performance
A good starting point for performance analysis is the request log.
You can find the request log at data\author\logs and data\publish\logs for the author
and publish instance respectively.
The log looks as follows (the lines are shortened for simplicity):
09:43:41
09:43:41
09:43:47
09:43:48
09:43:49
09:43:49
[66]
[66]
[67]
[67]
[68]
[68]
->
<->
<->
<-
GET
200
GET
200
GET
404
/author/y.html HTTP/1.1
text/html 797ms
/author/z.gif HTTP/1.1
image/gif 141ms
/author/x.html HTTP/1.1
text/html 0ms
This log has one line per request or response:
• The date at which each request or response was made.
• The number of the request, in square brackets. This number matches for the request and the
response.
• An arrow indicating whether this is a request (arrow pointing to the right) or a response (arrow to
the left).
• For requests, the line contains:
• the method (typically, GET, HEAD or POST)
• the requested page
• the protocol
• For responses, the line contains:
• the status code (200 means “success”, 404 means “page not found”)
• the MIME type
• the response time
Using small scripts, you can extract the required information from the log file and assemble the
statistics you want. From these, you can see which pages or types of pages are slow, and if the
overall performance is satisfactory.
1.5.2.3 Monitoring search response times with the CQ5 request.log
Search requests are also registered in the log file:
22.05.2008 12:27:40 [338] -> GET /author/playground/en/tools/search.html?
query=dilbert&size=5&dispenc=utf-8 HTTP/1.1
22.05.2008 12:27:42 [338] <- 200 text/html 1562ms
So, as above, you can use scripts to extract the relevant information and build up statistics.
1.5.2.4 Monitoring the numbers and impacts of concurrent users
Again the request.log can be used to monitor concurrency and the system's reaction to it.
Page 6 of 9
CQ 5.1 WCM
Copyright 1993-2008 Day Management AG
How to Monitor Performance
Tests must be made to determine how many concurrent users the system can handle before a
negative impact is seen. Again scripts can be used to extract results from the log file:
• monitor how many requests are made within a specific time span e.g. one minute
• test the effects of a specific number of users all making the same requests at (as close as
possible) the same time; e.g. 30 users clicking Save at the same time
22.05.2008 12:27:22 [333] -> GET /author/libs/Personalize/content/statics.close.gif
HTTP/1.1
22.05.2008 12:27:22 [334] -> GET /author/libs/Personalize/content/statics.detach.gif
HTTP/1.1
22.05.2008 12:27:22 [335] -> GET /author/libs/CFC/content/imgs/
logo.rZMNURccynWcTpCxyuBNiTCoiBMmw000.default.gif HTTP/1.1
22.05.2008 12:27:22 [335] <- 304 text/html 0ms
22.05.2008 12:27:22 [334] <- 200 image/gif 31ms
22.05.2008 12:27:22 [333] <- 200 image/gif 31ms
22.05.2008 12:27:22 [336] -> GET /author/libs/CFC/content/imgs/
logo.rZMNURccynWcTZRXunQbbQtvuuCMbRRBuWXz0000.default.gif HTTP/1.1
22.05.2008 12:27:22 [337] -> GET /author/titlebar_bg.gif HTTP/1.1
22.05.2008 12:27:22 [336] <- 304 text/html 0ms
22.05.2008 12:27:22 [337] <- 304 text/html 0ms
1.5.2.5 Timing Statistics for Page Loading
To see timing statistics for page loading you can use Ctrl-Shift-U - with ?
debugClientLibs=true set in the URL.
1.5.3 Caching
The following diagram shows the different cache locations that can be used for the various content
types.
Figure 1.3. What can be cached where?
The following can act as a rough guide for target values:
Page 7 of 9
CQ 5.1 WCM
Copyright 1993-2008 Day Management AG
How to Monitor Performance
Figure 1.4. Cache vs. Uncached - maximum hits / second
Although there are many algorithms to ensure that data is retrieved from the source system when
appropriate, circumstances can arise where the data residing in a cache is out of date. Retrieving
every page individually is the only guaranteed method of ensuring your content is up-to-date, but it
is very costly in terms of response, and can indeed cause knock-on effects.
This is particularly relevant when using personalized pages, where at least some content of a
page is dependent on the user, and the account they used to login.
Figure 1.5. Cache speed vs. Data Integrity
Page 8 of 9
CQ 5.1 WCM
Copyright 1993-2008 Day Management AG
How to Monitor Performance
1.5.3.1 Optimizing your content for cache performance
Make sure you use realistic cache settings for the browser cache. If you have disabled the browser
cache for development, this may increase traffic and decrease responsiveness.
1.5.4 Analyzing Search
First steps to analyzing the search function can be made with Monitoring search response times
with the CQ5 request.log.
However, once you have determined the response time, you may need to analyze why the request
is taking the time it does, and what can be done to improve the response.
Page 9 of 9
CQ 5.1 WCM
Copyright 1993-2008 Day Management AG