Five Times Faster - The Performance Test On ZK 3.0 RC

{{Template:Smalltalk_Author|
|author=Dennis Chen, Engineer, Potix Corporation
|date=September 14, 2007
|version=
}}



=Introduction=

::After a series of stress test and reviewing the kernel code, we found out 2 bottlenecks on ZK 2.4.1 and fixed them in ZK 3.0 RC.

::# The executing time is too expensive when rendering components. ZK uses templates to render components, and the EL is generally used in templates to simplify the variable access and templete maintenance. However, when the concurrent access rises to a large number, the overhead on rendering component with EL is too heavy.
::# Threads spend too much time on waiting the synchronization when many threads access to the same cache under current cache mechanism.

::ZK 3.0 RC solves these 2 bottlenecks by using the renderer class and new cache mechanism. The test result shows ZK 3.0 RC is four ~ five times faster than ZK 2.4.1.



=Test Environments=
<br><br>

==Server Side - A Desktop==
<br>

===Hardware===

::* CPU : AMD Athlon 64 Processor 3800+ 2.41G
::* Memory : 1.93 G
::* Network Adaptor : Broadcom 44x 10/100 Integrate Controller
::* Hard Disk : HITACHI HDS728080-PLA380 80G
<br>

===Software===

::* OS : WINDOW XP Professional SP2
::* JDK : SUN JDK 1.5.0
::* Server : TOMCAT 5.5.20
::** -Xms128m
::** -Xmx512m
::** maxThreads=1000 (in server.xml)
::** acceptCount=100 (in server.xml)
<br><br>

==Client Side - A Laptop==
<br>
===Hardware===

::* Intel Core Duo CPU T2350 1.86G
::* Memory : 0.99G
::* Network Adaptor: Broadcom 440x/100 Integrated Controller
<br>

===Software===

::* OS : WINDOW XP Professional SP2
::* JDK : SUN JDK 1.5.0
::* Apache JMeter 2.3RC3
<br><br>


==Network==
::100Mbps LAN connected by a small 5 port switch


=Test Scenario=

:On server side, a clean ZK 3.0 RC web application is installed, and we create a simple zul ([http://www.zkoss.org/smalltalks/zk30rctest/testcase1.zul.txt testcase1.zul]) to test the performance, this page contains 2 windows, 4 groupboxes ( 2 use ‘3d’ mold), 2 captions, 80 labels and 80 textbox (40 use multiline), no zscript is used in this test case.

:On client side, we use JMeter to measure the test result. We use a fixed number of concurrent threads for each test from 50 to 1000. The thread number increases 50 per second from 0 to the fixed number (50 ~ 1000) in each test. In each test, every thread requests the server page with 20 loops continuously.

:We also do same testing on ZK 2.4.1 to compare the performance difference.



=Some notes before testing=

:Before we do performance testing on ZK, we must check some rules to avoid invalid result.

:* Remember to add cookie manager on threads for continuous requests (ex, Http Cookie Manager in JMeter). Since we use cookie to remember session id , we must manage the cookie for each continuous request(same session). If not, each request will create a new session, which will create a mass number of session in our testing.
:* Set max-desktops-per-session to 1 (a configuration in zk.xml). ZK will clear each desktop when the browser changes pages or is closed, but this does not happen in the testing program, because no ajax request is sent to server to notify the clean desktop process. This means, after 100 thread running 20 requests continuously, the application will keep 1000(100*10, 10 is the default max-desktops-per-session) desktops, and all its children components, not 100.

:There are also some performance tips in develop guide which you should obey if you want to get a good response time in your application.



=Test Result=

:The following chart shows the test result. The response time in ZK 3.0 RC is four ~ five times faster than ZK 2.4.1.
<center>[[Image:chart1.png]]</center>
:In ZK 2.4.1 when concurrent threads are larger than 700, the server is no response after continuous request. You can download the test result data [http://www.zkoss.org/smalltalks/zk30rctest/testresult.csv here].



=Test on Your Environments=

:Following is the steps that you can test ZK on your environments.
<br>

==Server Side==

::1. Download and install zkdemo on your server
::2. Configure your tomcat
:::* Set –Xms### and –Xmx### depending on your test case, ### is the memory size( –Xms and –Xmx are the minimal and maximal heap size of JAVA runtime).
:::* Set maxThreads and acceptCount in tomcat/conf/web.xml (smaller maxThread will cause connection rejection when the number of concurrent threads is larger than it).
::3. Configure zk.xml, set max-desktops-per-session to 1 and disable event thread.
<pre>
<session-config>
	<max-desktops-per-session>1</max-desktops-per-session>
</session-config>
<system-config>
	<disable-event-thread/>
</system-config>
</pre>
::4. Download testcase1.zul.txt to webapps/zkdemo/, and rename it to testcase1.zul
::5. Start the tomcat
<br>


==Client Side==

:#  Download and install [http://jakarta.apache.org/jmeter/ JMeter]
:# In JMeter, add a Thread Group,and set the Number of Threads, Ramp-Up Period and Loop Count depends on your test case.
:# In Thread Group, add a Config Element/Http Cookie Manager to handle cookie in each thread.
:# In Thread Group, add a Sampler/Http Request HTTPClient , and set the Path to <nowiki>http://youserver:port/zkdemo/testcase1.zul</nowiki>.
:# In Thread Group, add a Listener/Aggregate Report, and you can see the aggregated report here when running the test.
:# Run JMeter to do testing.



=Download=

:* <tracker url="http://prdownloads.sourceforge.net/zk1/zk-demo-3.0.0-RC.zip?download" > zk-demo-3.0.0-RC.zip</tracker>
:* [http://www.zkoss.org/smalltalks/zk30rctest/testcase1.zul.txt testcase1.zul.txt]
:* [http://www.zkoss.org/smalltalks/zk30rctest/testresult.csv test result data]



=Summary=

:After using the new renderer and the new cache mechanism, ZK 3.0 RC now is four ~ five times faster than ZK 2.4.1. The performance enhancement increases computer efficiency and saves time for impatient users. We will keep finding out any bottlenecks or memory leaks in ZK and hope you could share your test result with us.
<br>
<center>[[Image:ruler.gif]]</center>


<comment>http://docs.zkoss.org/wiki/Five_Times_Faster_-_The_Performance_Test_On_ZK_3.0_RC</comment>


[[Category:ZK]]
[[Category:Performance]]
{{Template:Smalltalk_Footer|
|name=Potix Corporation 
}}