02-24-2010 12:59 PM
I am writing a Mobile Application for BlackBerry on JDE 4.6, currently I am stuck on one issue.
Basically the logic of application is something like..
At the startup from Splash Screen a Web Service be called which download JSON Data from Server(WS will send 2000 records in compressed format, and app will unzip it and Parse JSON and after that will store data in a Vactor, this process will be continues for 2,00,000 Records) approximate size of these records is 10 MB.) I am displaying download progress, its take approximate 70 Minutes time to complete download all Hotels Records.
So, Above logic is working fine, but the time which its take is not something which mobile user will be accept, so I have to go with any alternate solution(Multithreading kind of logic), so App can complete downloading quickly. I tried to use 2 Theads one for Downloading, and second for Parse Data and insert into Vactor, but unfortunetly its not giving me desired result, maybe I have used wrong approach, so I would be happy to know what would be your view\approch if you will be writing code for this application…
Note: As per App logic first time data must be download from Web Server, so I can’t go with way to embed all data with application, it must be download from Web Server.
Any Solution\comment is Welcome…
02-24-2010 02:29 PM - edited 02-24-2010 04:35 PM
I agree with the thought of splitting the Threads up as you have done, however it does not surprise me that the time taken was still significant. It seems likely to me that the network time is the most significant contributor to the delay. To prove that, you could just throw away the downloaded data and time that.
You could be making the download process more complicated than it needs to be by encoding the encrypted data to Base64 or attempting to process the data using some sort of stream processing. So have a good look at that processing.
In addition, I'm not sure if you really are using ZIP compression, but that is probably not the most efficient form for a Blackberry device, when processing it in this way. GZIP would be much better.
I've a load more other thoughts, but am interested to hear what your results are from the download processing before suggesting anything else to look at.
Edit: Having read RexDoug's Post, I think the point he makes is a very good one. Also if you have 2,000,000 rows with 10Mb data, that is 5 bytes per row, which doesn't sound reasonable.
02-24-2010 02:32 PM
I think your approach is flawed.
Why download 2M records to the blackberry? The information will be obsolete by the time you finish downloading it. This is 1980's approach, not 2010, and it is not a "wireless" approach.
Most folks will not wait. They'll think your app is broken.
And to make matters worse, many Blackberries will have insufficient storage to perist this much data.
Retrieve the data as needed from the web service. One approach would be to return only the hotel info for the current locale of the device, or possibly the destination city, or whatever.
For example, I wrote an application that provides restaurant information, The data is requested from the server for the immediate vicinity of the device.
Otherwise, I would have to store 10M records on the device.
02-24-2010 04:17 PM
Have you also explored the option of creating a blackberry specific mobile website for this specific problem?
Its an alternative to creating custom java apps that render regular news/content/lists etc.
02-25-2010 02:15 AM
My two cents:
- Download only limited records based on your criteria. Example, city, hotel category, tags, etc.
- Download first hand very limited information for the records downloaded. The details should be queried further on user selection.
02-25-2010 04:30 AM
I agree with Pooja,
we have developed a similar app, in which we have different XML files as webservices and depending on the user selection we connect to particular WS and parse the xml content for display.
02-25-2010 04:53 AM
Thanks Peter for sharing your view, I used Gzip, and regarding record size actually its more then 10 MB , as there is one field which having HTML Content.
Thanks RexDoug for your comments, Actually I not described application correctly, the logic of Application is something like, user must be download all data into local for a Region(Assume regions like USA, UK, Asia ), and then he must be able to view all data without internet connection, in future whenever application will get Internet connection it will download latest data from server, so I must allow user to download entire database into local, and whole functionality should be working within local data. And Application don't have any location aware system, so there will be not any meaning like to download data for that particular location only.
02-25-2010 05:03 AM
i would suggest to put the file on sd card then. you can download it on your PC and copy it to your BBs sd card. downloading 10MB over a mobile is a feat on itself.
02-25-2010 06:06 AM
Thanks Simon, But it will be not possible to have manually download that file, actually data is comming from various feeds, on server there is logic for process some feeds\third party Web services, are preparing data, and that data will be reflect users on their deviec for respective regions. As App will be avaiable on BB Store also, and user will purchase Regions, which they want.
02-25-2010 06:27 AM
well, best of luck selling your 70 minutes download app