GWT Benchmarks: GWT+JsVM vs JavaVM

  What is GWT?

GWT (Google Web Toolkit) is a tool for compiling Java into JavaScript code; with GWT you can write both your web application and your server side code using the Java language.

 

What about GWT performance?

I have worked with GWT in the recent years and a simple question has arisen:

"How does the same Java code perform when running inside the Java virtual machine VS running inside the browser?"

I have decided to do a bunch experiments and write this article after taking in account these facts:

• GWT developers state that the generated JavaScript code may be better than the handwritten one
• V8 JavaScript Engine and the other JavaScript engines have been highly improved in the recent years

It also makes sense to measure the perormance of GWT applications running on different browsers, Firefox, Opera, Chrome, Explorer in order to see who sucks and who rules.  

I think that these results may be useful from a distribuited High Perfomance Computing perspective (via web client).

 

 ..but what is performance?

I'm looking to numeric and data crunching performance; input/output, graphics and multimedia performace are out of topic.

I have measured execution time of the same code compiled with GWT, running inside the browser JavaScript VM(JsVM), and running "natively" inside the Java SE7 Virtual Machine (JavaVM).

Little execution time means high performace, it's quite simple. 

 

The Benchmark suite (Internals)

I have developed a little suite (EDIT: sources released) in order to run a bunch of deterministic single threaded benchmarks covering: strings crunching, numerical crunching and data manipulation; The suite performs these benchmarks after a warm up.

The warm up has been configured in order to make sure that all the benchmark code is JITted first.

To be clear, the JsVM compiles the JavaScript after a lot of execution cycles (e.g: 11000) but JavaVM  is going to compile the  bytecode to native code in few cycles.

After the warm up, the benchmark is executed lot of times and the overall execution time is recorded, it's the same approach than taking an average execution time.

The GWT and native benchmark suites share the same Java code and benchmark configurations, only the Main classes are different.

Here is the list and description of the performed benchmarks:

• CollectionSortBenchmark, Test Java collection sort and shuffle, manipulation.
• FFTBenchmaark, Computes FFT's of complex, double precision data, GSL code taken from scimark
• RegexBenchMark, apply a Regex (Javascript compatible) to a text
• SieveBenchmark, Computes prime numbers into a given numeric range
• StringCrunchBenchmark, String and StringBuilder manipulation, append, delete  

These benchmarks have been performed twice with different configurations, one time using little data sets, and a second time on a huge data sets (big arrays, big lists and so on).

This is intended to track the JsVM and JavaVM behavior when playing with memory demanding applications, yes, this looks like an Good&Old Garbage Collector stress test. ;-)
 

The Benchmark Setup

The suite has been executed on an asus K51 notebook equipped with:

• Intel Core2Duo T6600@2.2Ghz
• 4Gbyte RAM
• Windows7 Home (x64)

The web benchmark suite is loaded from a glassfish AP running on another server, the last GWT 2.5 SDK has been used to compile the benchmarks into Javascript code (Obfuscated javascript).

Let's see the Results

Here you have a bar graph (Figure.1) generated with (The Great) R; the bar heights represents the benchmark execution time, normalized towards the JavaVM 1.7.0 execution time. One little bar means little execution time and high performance, please note that the y-axis has a logaritmic scale.

The experiment name (x-axis) have a number appended on it, bigger the number, bigger is the data size used: FFTBenchmark5 is less memory demanding than FFTBenchmark6.

Take a look at the graph, the first benchmark on the left is the CollectionSortBenchMark3, the red bar represents the JavaVM (ver.1.7.0_05) execution time, the yellow bar is about the Chrome browser (V8 engine) and is about 2 times slower than native javaVM; the other browsers looks about 10 times slower than java code running "native".
The blue bar refers to Microsoft Internet Explorer version 9 (MSIE).

Figure.1
Littler execution times (tiny bars) means higher performance

Last note about these results, I have had to remove a Sieve benchmark on a greater numeric range (big numeric arrays), it's my fault, I have not tuned right the number of iterations and the javaVM has completed the task in ZERO ms!! Quite strange and annoying but I have had not the time to grasp inside this.

Another note, there are few limitations on the browser about the array size, Firefox has an higher limit but the problem applies too. So, sometimes all the JsVMs suck.

Conclusions

Results in Figure.1 are quite self-explanatory.

From a GWT application standpoint Chrome rules, aka V8 rules.
A java program running inside the Chrome browser is going to perform about 2 times slower, but it's so near to JavaVM, so near.

Another interesting point is Chrome regular expression execution time: Java Regex bechmarks performs far better in Chrome than in JavaVM; You know, the GWT Regex classes are backed directly by low level JsVM functions and they look terrific.

Kudos to Chrome regex implementation.

The Java GWT performance in Opera,Firefox and MSIE sucks a bit, running the code on those browsers is 5 or 10 times slower than running on JavaVM.

Firefox sucks a lot with numeric workloads or crunching big arrays of numeric data, running 40 (or 50) times slower than the JavaVM .

I hardly suspect that the GWT developers devote themselves to Chrome optimization, and for this reason GWT compilation for the other browsers sucks. I have no clue, do you have? :-)

From an HPC standpoint, GWT apps on Chrome performs very well compared to native Java Virtual Machine apps, so, GWT looks like a good tool for writing distributed High Performance Computing web clients.

PS:

Comments and criticisms are welcome