Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Native Development

Reply
Developer
Posts: 206
Registered: ‎05-15-2012
My Device: None
My Carrier: Telus

What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess

Has anybody done any testing on this?

 

I need to read ALL the data that is being stored on disk into memory.

 

I load a fairly large data set, from disk, when the app I'm working on starts, and it can take 5-10 seconds for the data to load. Right now the data is stored as XML, but I could convert it to whatever is the quickest to access. I suspect another DataAccess type might be faster.

 

Any info is appreciated.

----------------------
Check out my app, Alien Flow for reddit

And of course, like my post if you found it helpful or informative!
Developer
Posts: 6,473
Registered: ‎12-08-2010
My Device: PlayBook, Z10
My Carrier: none

Re: What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess

The answer probably depends on the nature of the data involved but my guess is that for most types of "interesting" data you'd find SQLite to give the best results. (Not based on testing, since I don't have your data...)

Peter Hansen -- (BB10 and dev-related blog posts at http://peterhansen.ca.)
Author of White Noise and Battery Guru for BB10 and for PlayBook | Get more from your battery!
Developer
Posts: 206
Registered: ‎05-15-2012
My Device: None
My Carrier: Telus

Re: What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess

The data is pretty typical. Technically the data is relational, but that's mostly irrelevant since all the "joins" would be done in C++ code, not using an SQL query (whether that is a good idea or not is questionable, but that is the case and I don't want to rewrite someone else's code).

 

Basically I'm loading thousands of records that each contain about 5-10 fields. The types of the fields are exclusively int and string.

----------------------
Check out my app, Alien Flow for reddit

And of course, like my post if you found it helpful or informative!
Developer
Posts: 6,473
Registered: ‎12-08-2010
My Device: PlayBook, Z10
My Carrier: none

Re: What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess

If you'll be doing lots of custom processing in C++ then a custom data format may make sense, for ultimate speed.

Aside from that, note that of the three formats mentioned above only SQLite would not require parsing the integers from textual form, as it stores integers natively (and in compact fashion as well, generally).

I'll stick with my educated guess that you'd get the best results from SQLite here.

Peter Hansen -- (BB10 and dev-related blog posts at http://peterhansen.ca.)
Author of White Noise and Battery Guru for BB10 and for PlayBook | Get more from your battery!
Developer
Posts: 1,739
Registered: ‎04-08-2010
My Device: Passport Silver, PRIV
My Carrier: Telekom.de, O2, Vodafone

Re: What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess

using JSonDataAccess with some thousands of records I also got very fast results.

 

but it's still on my ToDo to write an alternate impl using SQLite to compare speed and memory-consumation

-------------------------------------------------------------------------------
ekke (independent software architect, rosenheim, germany)

BlackBerry Elite Developer
BlackBerry Platinum Enterprise Partner
International Development native Mobile Business Apps
BlackBerry 10 | Qt Mobile (Android, iOS)
Workshops / Trainings / Bootcamps

blog: http://ekkes-corner.org
mobile-development: http://appbus.org Twitter: @ekkescorner
Developer
Posts: 105
Registered: ‎09-06-2013
My Device: Z10

Re: What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess

Depends on your data , for me , as most of my data have attributes that may not exist for different objects of the same kind , i use JSON for storing data to have the nosql effect. Conversely, if u have a fixed object data format , sql would most probably be the fatest

Developer
Posts: 16,932
Registered: ‎07-29-2008
My Device: Z10 LE, Z30, Passport
My Carrier: O2 Germany

Re: What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess

i would suggest to not use sqldataaccess but qsqlquery, it allows to set forwardonly=true on select statements which improves the speed quite a lot. you can also wrap multiple statements into one transaction.
sqldataaccess also has a weakness in not returning the id after an insert (i reported that as a bug in jira)

i got a speed increase up to factor 10, so i rewrote my whole persistence level (20k lines of code).
----------------------------------------------------------
feel free to press the like button on the right side to thank the user that helped you.
please mark posts as solved if you found a solution.
@SimonHain on twitter
Developer
Posts: 1,739
Registered: ‎04-08-2010
My Device: Passport Silver, PRIV
My Carrier: Telekom.de, O2, Vodafone

Re: What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess


simon_hain wrote:
i would suggest to not use sqldataaccess but qsqlquery, it allows to set forwardonly=true on select statements which improves the speed quite a lot. you can also wrap multiple statements into one transaction.
sqldataaccess also has a weakness in not returning the id after an insert (i reported that as a bug in jira)

i got a speed increase up to factor 10, so i rewrote my whole persistence level (20k lines of code).

wow - factor 10 - are there sample apps using qsqlquery ?

 

in most cases I'm in luck to deal with GroupDataModels and JSON - getting great performance and easy development because of easy mapping from JSON Object to QVariantMap from C++ to JS Object in QML and JSON Array to QVariantList in C++ to JS Array in QML.

-------------------------------------------------------------------------------
ekke (independent software architect, rosenheim, germany)

BlackBerry Elite Developer
BlackBerry Platinum Enterprise Partner
International Development native Mobile Business Apps
BlackBerry 10 | Qt Mobile (Android, iOS)
Workshops / Trainings / Bootcamps

blog: http://ekkes-corner.org
mobile-development: http://appbus.org Twitter: @ekkescorner
Developer
Posts: 16,932
Registered: ‎07-29-2008
My Device: Z10 LE, Z30, Passport
My Carrier: O2 Germany

Re: What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess

as it is basic Qt you can get samples from old nokia sources etc, but the documentation should be enough, it's quite simple.

I would be interested in your test results comparing json and sqlite.
----------------------------------------------------------
feel free to press the like button on the right side to thank the user that helped you.
please mark posts as solved if you found a solution.
@SimonHain on twitter
Developer
Posts: 1,739
Registered: ‎04-08-2010
My Device: Passport Silver, PRIV
My Carrier: Telekom.de, O2, Vodafone

Re: What is the fastest data access type JsonDataAccess, SqlDataAccess, or XmlDataAccess


simon_hain wrote:
....

I would be interested in your test results comparing json and sqlite.

I'm working on the native BlackBerry10 Library for Kinvey.com baas - there I will implement caching with JSON vs SQLite ... so it would be a good candidate to compare both

 

... but will take some weeks - just have to finish an important project ;-)

... then finish my slides for BBJam Asia

... then spend a week in Hong Kong

 

so to be realistic: can do comparison in october

-------------------------------------------------------------------------------
ekke (independent software architect, rosenheim, germany)

BlackBerry Elite Developer
BlackBerry Platinum Enterprise Partner
International Development native Mobile Business Apps
BlackBerry 10 | Qt Mobile (Android, iOS)
Workshops / Trainings / Bootcamps

blog: http://ekkes-corner.org
mobile-development: http://appbus.org Twitter: @ekkescorner