Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Java Development

Reply
Developer
Posts: 97
Registered: ‎10-06-2009
My Device: Not Specified
Accepted Solution

Storing Large number of Records on device

Is it possible to store thousands of records on device like 75000 records in persistant store.Each record will contain 5 string values.Is it a good practice to store that much of data on device ?? will it effect the performance of the app/device ??

Regular Contributor
Posts: 51
Registered: ‎01-13-2011
My Device: 9800
My Carrier: vodaphone

Re: Storing Large number of Records on device

No, because, if u want to store large number of records in ur device then ur device will hang. i will suggest u please use sql lite databse to store information.
Vishal Lakhmapurkar
Developer
Posts: 97
Registered: ‎10-06-2009
My Device: Not Specified

Re: Storing Large number of Records on device

my application should support devices starting  from 4.5 os.I can't use sql lite

Regular Contributor
Posts: 51
Registered: ‎01-13-2011
My Device: 9800
My Carrier: vodaphone

Re: Storing Large number of Records on device

for this you can use persistance
Vishal Lakhmapurkar
Regular Contributor
Posts: 51
Registered: ‎01-13-2011
My Device: 9800
My Carrier: vodaphone

Re: Storing Large number of Records on device

you can use persistance but problem is thar you can not view ur data rather then sql lite.

 But i think you can store your records in persistance.

 

 

Vishal Lakhmapurkar
Developer
Posts: 97
Registered: ‎10-06-2009
My Device: Not Specified

Re: Storing Large number of Records on device

My requirement is not viewing the whole data, I have to store the whole data in store and whenever user searches for a string i have to search that string in the store and display the details of that record.Thats why i am asking is it good approach to store 75000 or more number of records in persistant store, and will it effect the app/device performance if i search through those many number of records each time.

Developer
Posts: 19,636
Registered: ‎07-14-2008
My Device: Not Specified

Re: Storing Large number of Records on device

[ Edited ]

It is possible to store a significant amount of data on device in persistence store.  But you will have to be very careful with it.

 

When using persistent store for large amounts of data, you need to careful of two things:

a) The size of the data

b) The number of Objects that you have, i.e. the number of Object handles you use.

 

If we look at these quickly we might get some ideas about the practicality of what you are trying to do.

 

First data size.  You are talking about storing 75,000 * 5 Strings.  Each of these Strings is an Object, and in some testing I did, each Object had an overhead of about 100 bytes.  Assume each of these String also contains 200 characters.  Then this will be 400 bytes on the device.  So for each String we are talking about 500 bytes of Storage required.  So multiply this out, and we see you database might need

5 * 75,000 * ( 100 + 400) bytes of storage

 

So roughly this is 200 MB.  I think older devices do not have the memory to support this sort of size.  For example, the 8300 Curve has only 64 MB, and that has to include OS and Applications and email. 

 

Now look at Object Handles.  Each Object has a handle.  For Persistent Objects I believe you can have up to 3 handles, one for the Object in Persistent Store, one for the RAM copy you are working on and one for the non persisted copy you last updated but have not committed yet.  Or something like that.

 

There are restrictions on the number of Object Handles.  In older device (I'm talking 7290 here) it was something like 32,000 possible Object Handles.  I don't know what it is in newer devices, I've not checked.  but the point here is that you are asking for at least 375,000 Object handles.  That will also blow older devices. 

 

There are ways round all of this that we can go into if we need to  For example you can group Objects to reduce handle usage.  You could compress the Strings to save space. But this is all a lot of work. 

 

If you look at the statistics, the vast majority of apps sold today are sold to OS 5.0 or later users.  You can see this and additional figures here:

http://us.blackberry.com/developers/choosingtargetos.jsp

 

Do you really want to go to the trouble of creating a database solution for older devices?

 

If you really do, then go through the sort of figures I have given you above to demonstrate viability on the target phones. 

 

Edit:

I see you have just asked about performance.  If you can get the data in persistent store, I think its performance will be significantly better than if the data was in an SQL Database.  But that is not to say the database will be slow, just my experience is that peristent store is faster. 

 

 

 

Developer
Posts: 97
Registered: ‎10-06-2009
My Device: Not Specified

Re: Storing Large number of Records on device

[ Edited ]

Thanks peter for detailed explanation.Came to know few new things from ur post such as Object Handle, i have no idea about what it is. i have to check that.When coming to my requirement, it would be easy if i tell it clearly.My application is a suggestive search application, one can search for people using diff criteria from the database.For that we used client server communication as user enters a char client app will hit the webservice and will fetch matching names, along with other details(First name, last name,middle name, id, ..) .But hitting a web service for each character is not that effective, so we thought once user enters 3 chars we will hit the web service and will fetch all matching results and from then on client app will take care of suggestion (It will search through the retrieved list and will display first 25 matching names, this sol appeared reasonable, but when we came across few combinations(3 character) which have more than 1000 matching results we stucked up again, because it is taking more than 25 sec on simulator(We didnt tested this on device), transfering 1000 or more records will obviously take time but it doeesn't make any sense for the suggestive search app.so i came up with this persistant store, performance  question.From your post it's clear that I can opt for persistent store than sql lite (it's good for my situation as my app has to support old devices(starting from os 4.5)).But now 2 other questions arise one iss the number of object handles supported by device and the other one is memory size. I can assure that each of the 5 strings will contain only 10 chars, and when i searched in the site to know more about object handles i came across this http://docs.blackberry.com/en/developers/deliverables/17952/Performance_of_the_persistent_store_1253... they they have diff cal.I have to learn more about object handles then only i can come to a conclusion.What if i store the data in 2 diff persistent stores??   suggest me.  The feel of suggestive search should remain thats the big problem Smiley Sad    

Developer
Posts: 19,636
Registered: ‎07-14-2008
My Device: Not Specified

Re: Storing Large number of Records on device

"when we came across few combinations(3 character) which have more than 1000 matching results we stucked up again"

Don't send them all then, do a 'more' implementation, send the first 50 at most, and if the BlackBerry user wants to see more that can get it.  But most likely they will add extra data to the search - the BlackBerry user is not going to scroll through 1000 records. 

 

"(We didnt tested this on device), "

It will be significantly slower especially for a wireless (non WiFi) connection. 

 

"as my app has to support old devices(starting from os 4.5))."

I hope you are being paid well.  This code will be redundant in 2 years.  My opinion only,,,,

 

"2 other questions arise one iss the number of object handles supported by device and the other one is memory size."

You will have to research this yourself, I do not have access to anything you don't have in this regard. 

 

There is another consideration.  How are you going to get this data onto the device?  Do the maths on the size of data that you will need, remembering that a cod file is restricted to about 8 MB (from memory).  And calculate in the download time. 

 

Honestly I think you are wasting your time and your clients money.  By the time you have this developed (which will be months) the 4.5 devices will all be replace with devices that run OS 5.0 or above.  If not, then cost out this development, cost out the replacement costs and make sure you are saving money. 

Developer
Posts: 97
Registered: ‎10-06-2009
My Device: Not Specified

Re: Storing Large number of Records on device

App works like this

 

For example if user enters 'dav', client app will hit the web service and webservice will return 1000 matching results.but the app will display first 25 names.now as user enters other characters like below

 

davi

 

as the first 3 char are same client app will search through the 1000 names and will list first 25 matching names.this process goes on untill the user finds the req name or there is no such name(No results are found).

 

client app will hit the webservice only if the starting 3 letters combination changes.if suppose user enters

 

ex: tom

 

and so on.This was clients req that devices from os 4.5 should be supported i can't do any thing about that.i am searching for other ways as i want to give a good sol to my client, and i dont want to waste their money.I.It took 10 - 15 days for developing  in 2 diff ways (hitting for each char and hitting for 3 chars and carrying out search on client side).if i am sure of the persistent store thing may be it will take another 7 or max 10 days.

 

May be i will put all the data in file in xml format or normal format and i will store that data in persistant store once the app is installed.text file should not take much size.Or may be i will find some other solution with a help in this forum from people like u Smiley Happy.

 

From start we are taking care that app works on following platforms 4.5, 4.6, 4.7, 5.0 and 6.0. 

 

Thanks for the help Peter Smiley Happy