Web and WebWorks Development

[billbsb]

Hi there! On this thread I would like to keep my findings on the Playbook's browser performance and compliance with W3C standards. On these two aspects, the PB Browser is a winner but on performance it is a totally different story. The goal is to have an idea of what runs really fast and well on the Playbook and avoid the slow, glitchy parts.

I didn't develop any of the pages or tests listed here. I just randomly come across a webpage that makes great use of JavaScript and CSS and try them on some tablets to see if the websites are still usable.

Usually I test on Playbook(2.0.1.358), iPad 1st(5.1.1) and Motorola Xoom 10.1(4.0.3).

Unfortunately, I don't have any other 7" device right now.

Please share your comments, results for other tablets and other cool pages or performance tests.

The first entry is Green Sock's performance comparison page:

http://www.greensock.com/js/speed.html

All tests running with 50 particles, unless specified otherwise.

Playbook	iPad	Xoom	Framework
37	37	36	jQuery
16	12	11	YUI 3
29	22	17	MooTools
36	38	36	Zepto
30	74	74	TweenLite
23	21	22	TweenLite (200p)
3	2	3	TweenLite (1000p)

Remarks:

• I think for most people a difference less than 5fps when it's already running above 24fps is not noticeable.
• YUI 3 crashed Safari on iOS and also the Playbook browser. The FPS was also dropping steadily. Maybe a memory leak.
• The Xoom appear to drop frames to achieve most of its good performance. There was a clear difference on the quality of the animation even when running on same fps.

[billbsb]

I just came across this very cool video showing off some Playbook OS 2.0 and OS 1.0.8 browser comparison and I ran it on my test machines. Here are the results for the HTML 5 version and a bonus Flash version:

-	Playbook	iPad	Xoom
HTML	36	13	23
Flash	27	--	26

Remarks:

 - On the Xoom the animation is very choppy for HTML version so there must be something wrong with the FPS counter.

 - The Flash version runs very fluid on both supported platforms.

Here are the links:
Test 1: http://www.craftymind.com/factory/guimark3/bitmap/GM3_JS_Bitmap.html

Test 1 FLASH : http://www.craftymind.com/factory/guimark3/bitmap/GM3_Flash_Bitmap.html
Test 2: http://remysharp.com/demo/rubiks/

Test 3: http://hakim.se/experiments/html5/origami/

[billbsb]

Microsoft is coming strong to the Tablet wars with their new Windows 8 platform. They are making sure that the new Internet Explorer 10 will be THE best browser to surf the web and to prove that IE 10 got what it takes, they've came up with Test Drive. This website has some serious cutting edge, sometimes proprietary, technologies beeing pushed to the limits. 

Off course, having THE best browser is also RIM's goal and they are doing an amazing job. Just check out the following test results of the Chalkboard benchmark. The results are in seconds, lower is better.

-	Playbook	iPad	Xoom
seconds	8.9	194.39	17.43

Remarks:

- The Playbook was the only device capable of reproducing the intended audio while running the tests although it was going crazy with the volume controls (almost throws me out of my chair).

That was just an amazing result for the Playbook! Even running this test on my Macbook Pro was hard: only Safari manage to do it with sounds in 15.45s! Chrome 18 and Firefox 13 didn't even finish.

I can't wait to get my hands on a Windows 8 tablet and also the New iPad to see how outdated the Playbook is.

Cheers.

[twindsor]

That's some cool stuff. I've seen a lot of those tests, but hadn't heard about the last one. Will need to check it out.

In your second post, I believe some of the comments from the tester mentioned that the Xoom and even the iPad were incorrectly reporting the framerate as higher than the actual framerate, which might be more visible on the Xoom than the iPad.

Are you planning on running any of these with the latest 2.1.0 Beta software? I think some will improve even more.

There's also a "micro-benchmark" called Sunspider that tests raw JavaScript performance. PlayBook does all right, but I don't know how the others perform: http://www.webkit.org/perf/sunspider/sunspider.html

[billbsb]

Hi Tim,

If you are into performance comparison, this is quite cool indeed. For devs, it's important to know which techniques works best on which platform. For instance, what's faster on the Playbook to hide an image? Change its visibility or opacity property? And on iOS? Thanks for the tips and I'll definitely try the Sunspider test.

So, finally I've got the iPad 3- the quad-core retina display top notch tablet, Apple's best creation so far. It runs on iOS 5.1.1 - the latest and same as running on the iPad. Here are the results:

Playbook	iPad	iPad 3	Xoom	Framework
37	37	70	36	jQuery
16	12	29(dropping)	11	YUI 3
29	22	38	17	MooTools
36	38	102	36	Zepto
30	74	100	74	TweenLite
23	21	51	22	TweenLite (200p)
3	2	6	3	TweenLite (1000p)

GM3

-	Playbook	iPad	iPad 3	Xoom
HTML	36	13	59*	23
Flash	27	--	--	26

*: Running the test on landscape, it reaches only 3.8 fps!

Chalkboard

-	Playbook	iPad	iPad 3	Xoom
seconds	8.9	194.39	210.2	17.43

As you can see, the iPad 3 is rock solid with it's newer powerful hardware and iOS 5.1.1. Only GM3 on landscape and Chalkboard were weird. Whatever's going on the code there, it's definitely not optimized for iOS.

Let's see what the Playbook OS 2.1 will bring for us! Hopefully will not take long to arrive. Now I can't wait to get my hands on Google's Nexus 7 with Chrome and Microsoft's Surface IE 10.

Cheers

[billbsb]

Today I bring you the results from running the Sunspider benchmark that Tim suggested. The big difference though is that I have updated my Playbook to OS 2.0.1.688, just released. All browsers are based on WebKit and the results surprised me:

	Playbook	iPad	iPad 3	Xoom
Total	2268.7ms +/- 1.0%	2806.1ms +/- 0.4%	1797.5ms +/- 0.4%	2217.1ms +/- 1.7%
3d	338.6ms +/- 0.8%	383.6ms +/- 0.4%	211.2ms +/- 0.4%	373.8ms +/- 10.2%
access	252.7ms +/- 2.4%	353.5ms +/- 0.7%	222.5ms +/- 0.6%	190.1ms +/- 4.9%
bitops	138.2ms +/- 0.7%	158.3ms +/- 0.7%	140.7ms +/- 0.5%	152.5ms +/- 10.4%
controlflow	18.6ms +/- 2.0%	19.9ms +/- 1.1%	15.4ms +/- 2.4%	18.6ms +/- 4.9%
crypto	179.7ms +/- 2.4%	170.1ms +/- 1.6%	122.4ms +/- 0.8%	154.2ms +/- 4.2%
date	351.9ms +/- 2.9%	450.0ms +/- 2.4%	268.5ms +/- 0.9%	349.4ms +/- 1.0%
math	166.8ms +/- 0.5%	328.7ms +/- 0.5%	175.1ms +/- 0.5%	145.5ms +/- 5.6%
regexp	135.7ms +/- 0.3%	95.7ms +/- 2.2%	70.7ms +/- 0.5%	120.5ms +/- 3.9%
string	686.5ms +/- 1.4%	846.3ms +/- 0.5%	571.0ms +/- 0.9%	712.5ms +/- 1.6%
full results	link	link	link	link

Ok, now that's very revealing. The iPad 3 is the clear winner. Actually, if you compare the Playbook with the New iPad you get;

** TOTAL **:  *1.26x as slow*  1797.5ms +/- 0.4%  2268.7ms +/- 1.0%   significant

Which is ... significant! But remember of what is the New iPad in terms of hardware. RIM will release the 4G Playbook in a few weeks with a small hardware upgrade and I wonder if it will be enough to deliver the expected performance.

This was a tough test for our small warrior and it shows that the latest implementation is still missing a few optimizations. Personally, I would focus on the string performance and try to get it as fast as possible since JavaScript and CSS uses a lot of strings.

Cheers!

[billbsb]

And here's a quick one to compare the previous version of Playbook OS 2.0.1.358 with the latest version, Playbook OS 2.0.1.668:

358	668	Framework
37	43	jQuery
16	18	YUI 3
29	34	MooTools
-	41	Dojo (new)
-	34	TweenJS (new)
36	50	Zepto
30	60	TweenLite
23	41	TweenLite (200p)
3	4	TweenLite (1000p)

GM3

-	358	668
HTML	36	39
Flash	27	25

Chalkboard

-	358	668
seconds	8.9	8.0
score(new)	-	535.8

This is GREAT news! There are some serious improvements on Playbook's browser! RIM is doing their work and hard. Too bad I didn't do the Sunspider test before updating. Only the new Flash Player plugin that suffered a small decrease in performance - but I heard this new version has increased security.

Share this to the world if you like it.

Cheers!