Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Web and WebWorks Development

Reply
New Member
Posts: 1
Registered: ‎03-25-2014
My Device: 9900
My Carrier: AT&T

BB 7 - Unable to download 50,000 records from a JSON webservice to a SQLite db using Javascript

On 9900s/9930s over AT&T 4G or VZW 3G cellular w/BBOS 7.1 via BES: We have a native app which downloads about 50,000 rows from SQLserver into a SQLite db for use case where users are intermittently disconnected from wireless coverage (i.e. offline access).  On device db = about 50 MB. Data goes from internal SQLserver -> internal Webservice -> JSON response -> BES --> native client app ->locally stored SQLite db in the database folder. User can then search the data locally. Periodically, the native code checks for server-side data updates and applies them. Client app calls the webservice, gets a row count, then requests 500 rows at a time, processes them/binds them to the db, then asks for the next 500 rows until all 50,000 rows are posted to the on-device SQLite db.

 

Was asked to migrate app to HTML5 on BBOS7. Using Webworks, attempted to code same functionality but couldn't get javascript to access the sqlite table, so used javascript to call the webservice and (using the data from the webservice json response) to build a Websql datastore in the appdata folder. Still asking for 500 rows at a time. Performance chugs with numerous disconnects and the need to retry web service requests required. After processing about 10,000 rows, it fails with out "out of heap memory" error. After each 500 rows, am resetting variables for the rows to null. Didn't solve it. Tried reducing request to 50 rows, didn't solve it.

 

Is there a way to:

(a) release the memory/caches/etc. after each 500 record return from the webservice during processing

(b) improve the speed of putting the json formatted data into the websql store

(c) alternatively, I was thinking maybe we could sideload a sqlite table and then just use html5 to update it. If so, is there a way to use javascript in the BB browser to access an existing on-device sqlite db?

 

Thanks in advance.

 

Julie

Retired
Posts: 1,382
Registered: ‎07-02-2009
My Device: BlackBerry Bold 9900
My Carrier: Bell

Re: BB 7 - Unable to download 50,000 records from a JSON webservice to a SQLite db using Javascript

Hi Julie - sorry for the delayed response to your post.

 

a) There are no direct calls to garbage collection methods. The runtime cleans up after itself, however it is very possible/easy to have memory leaks in JavaScript (http://javascript.info/tutorial/memory-leaks).  I would recommend connecting to your app using remote web inspector and run a profile while you are interacting with your DB. If you see a spike in memory usage, perhaps it can help narrow down where a leak is and give you an idea how to fix it. https://developer.blackberry.com/html5/documentation/v2_1/debugging_using_web_inspector.html

 

b) Again, suggest using remote web inspector to evaluate performance and narrow down which functions / code snippets are slowing things down.  If you are formatting & concatenating lots of strings, it may be the source of a bottleneck.

 

c) no this is not possible. You must create the schema and then fill it from within the le app.

 

Hope that helps.

Follow me on Twitter: @n_adam_stanley
-------------------------------------------------------------------------------------------------------------------------
Your app doesn't work? Use BlackBerry remote web inspector to find out why.