A Simple Loading Performance Test on zkdemo

From Documentation
DocumentationSmall Talks2006OctoberA Simple Loading Performance Test on zkdemo
A Simple Loading Performance Test on zkdemo

Author
Henri Chen, Principal Engineer, Potix Corporation
Date
October 05, 2006
Version
Applicable to ZK 2.1.2 and later.


The Test

The loading performance test is done by using the record and playback method. First record the http requests sent by client into a testing script and then playback with multiple client threads simultaneously to simulate multiple users. We found the zkdemo testing script can perform 400 simultaneous client threads on a Mac mini equipped with 512MB memory with an 163.5 ms average response time.


The Server

  • Mac mini (Intel) on the 100 Mb local area network.
  • 1.66GHz Intel Core Duo Processor
  • 512MB DDR2 SDRAM (minus 64MB shared display buffer)
  • Mac OS X 1.4.7
  • Tomcat 5.5.20 with maxThreads set to 1000
  • Java 1.5_06 with max heap set to 384MB (-Xmx384M)


The Client

  • Desktop PC on the 100 Mb local area network.
  • 1.4GHz Pentium 4
  • 256MB RAMBUS PC-800 DRAM
  • Window 2000 Professional
  • grinder-3.0-beta30
  • Java 1.5_06 with default heap size


The Sample Application

  • zkdemo-all.war (ZK 2.1.2)
  • Rewritten all zscript codes (BeanShell interpreter code) with compiled java classes.
  • Increase page definition caching size and some other fine tunes.


The Methodology

  • Record testing script with Grinder TCPProxy.
  • Playback and collect information with Grinder Agent and Grinder Console.
  • The testing script is playback for 30 runs for each thread.
  • The simultaneous threads number is changed from 50 ~ 450 for each test.
  • The first run (run 0) is ignored to let the server reach its stability.


The Results

  • ART: Average Response Time for a test.
  • Max ARRT: Maximum Average Request Response Time for a test.
  • Max RT: Maximum Response Time for a test.
Threads ART (ms) ART Standard Deviation (ms) Max ARRT (ms) Max RT (ms)
50 2.62 7.42 5.9 261
100 4.6 10.52 10.8 1202
150 5.43 14.26 10.9 1252
200 12.2 48.4 25 1292
250 27.85 94.46 56.4 2504
300 52.63 140.62 105 1672
350 118.04 204.72 155 2013
400 163.5 266.72 248 2113
450 Grinder (client side Java) complains "Out Of Memory"

Step By Step


The server side

deploy the zkdemo-all.war to Tomcat server.

The client side

1. Unzip the zktest.zip into a directory, say c:\zktest.
2. Edit setGrinderEnv.cmd to reflect your testing environment.
  • GRINDERPATH=c:\grinder-3.0-beta30
  • GRINDERPROPERTIES=c:\zktest
  • JAVA_HOME=c:\jdk1.5.0_06
3. Edit the file grinder.properties. Change the thread number and make sure the url is point to your Tomcat server.
  • grinder.threads=50.
  • url=http://X.X.X.X:8080
4. Open a command box:
  • Change directory to c:\zktest
  • Execute "startConsole"
  • "The Grinder Console" GUI console should popup.
  • On the grinder GUI console, set "Ignore 0 Samples" to 120 ~ 480 depends on the thread number.
5. Open another command box (Let's call it "agent" command box)
  • Change directory to c:\zktest
  • Execute "startAgent"
6. On the grinder GUI console, select menu "Action/Start Processes". The grinder start sending requests to server and collecting information.
7. The "agent" command box will show up something like:
2006/10/3 AM 09:34:28 (agent): worker xxx-0 started
2006/10/3 AM 09:34:33 (process xxx-0): starting threads
8.When the test is done, the "agent" command box will show messages as follows:
2006/10/3 AM 09:54:06 (agent): finished, waiting for console signal
9. On the grinder GUI console
  • Select "Results" tab and you will see the statistics for the test.
  • The new collected data would be logged in c:\zktest\log
10. Change the threads number, restart the "agent" and "console" and start over.

Summary

Under our testing environment, the zkdemo testing script can perform 400 simultaneous client threads on a Mac mini equipped with 512MB memory with an 163.5 ms average response time. For those interested in the collected raw data, you can find them here. The ZK team also welcome you to collect your own data, see what happens, and give us feedback. That is why open source software is better, isn't it?




Copyright © Potix Corporation. This article is licensed under GNU Free Documentation License.