Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Native Development

Reply
Contributor
paulgee31
Posts: 24
Registered: ‎06-14-2011
My Device: Bold 9870
My Carrier: O2
Accepted Solution

QNetworkAccessManager limiting memory usage when headless

I have a long running headless app which uploads files of 6Mb upwards to a web server. Now using the QNetworkAccessManager to post the file, I can see memory usage shoot up and soon exceed the 3Mb limit.

 

The QNetworkAccessManager then suffers bad allocations and crashes.

 

I was kind of hoping the QNetworkAccessManager would chunk the data from the file off the disk and keep the memory footprint low, however it appears it tries to suck the whole file in.

 

Any means of limiting or controlling the behaviour of QNetworkAccessManager?

 

Cheers

 

Paul.

Please use plain text.
BlackBerry Development Advisor
RSperanza
Posts: 141
Registered: ‎03-08-2012
My Device: Z10
My Carrier: Bell

Re: QNetworkAccessManager limiting memory usage when headless

Which variant of the post function are you currently using?

Please use plain text.
Contributor
paulgee31
Posts: 24
Registered: ‎06-14-2011
My Device: Bold 9870
My Carrier: O2

Re: QNetworkAccessManager limiting memory usage when headless

I'm using the post ( const QNetworkRequest & request, QIODevice * data ) and a QFile.

Please use plain text.
BlackBerry Development Advisor
RSperanza
Posts: 141
Registered: ‎03-08-2012
My Device: Z10
My Carrier: Bell

Re: QNetworkAccessManager limiting memory usage when headless

Thanks.  You are already using the call I was going to suggest.  This poses an interesting problem.  I will have to raise with my colleagues who came up with the headless app memory limits.  I will have to get someone to check the implementation of the QNetworkManager class to see if it always buffers the whole request in memory before sending it.  It shouldn't because then very large file uploads via HTTP POST would fail.  The minimum buffer may need to be reduced but we still need to find a way to make this work in the interim until that gets fixed.

 

Please use plain text.
BlackBerry Development Advisor
RSperanza
Posts: 141
Registered: ‎03-08-2012
My Device: Z10
My Carrier: Bell

Re: QNetworkAccessManager limiting memory usage when headless

Hi.

 

I just thought of something.  How are you encoding the file that you are sending?  Are you assuming that the post is encoding it correctly for you?  I am wondering if switching to the

 

QNetworkReply * QNetworkAccessManager::post ( const QNetworkRequest & request, QHttpMultiPart * multiPart )

 

call will get around this problem.  See the reference for QHttpMultipart for how to setup a multipart with a file attachment.  I am guessing that it does not bring the whole file into memory when building the request. 

 

Try that and let me know if it helps.

Please use plain text.
Contributor
paulgee31
Posts: 24
Registered: ‎06-14-2011
My Device: Bold 9870
My Carrier: O2

Re: QNetworkAccessManager limiting memory usage when headless

[ Edited ]

Hi

 

I tried the QHttpMultipart and attached the file as per the example. I ran it up and watched the memory usage blast past 3600k before bad allocations were thrown from the QNetworkAccessManager.

 

I figure they share the same routines for reading the file.

 

Nice idea but still can't move forward at the moment.

 

Thanks

 

Paul.

Please use plain text.
BlackBerry Development Advisor
RSperanza
Posts: 141
Registered: ‎03-08-2012
My Device: Z10
My Carrier: Bell

Re: QNetworkAccessManager limiting memory usage when headless

Hi.

 

We started an internal investigation to check the QNetworkAccessManager implementation.  One suggestion made to try next was to try using libcurl to perform the file upload.

Please use plain text.
Contributor
paulgee31
Posts: 24
Registered: ‎06-14-2011
My Device: Bold 9870
My Carrier: O2

Re: QNetworkAccessManager limiting memory usage when headless

I can confirm that using libcurl doesn't signicantly increase memeory usage when transferring multi-megabyte files and I was able test this in a headless app successfully.

 

However compared with QNetworkAccessManager you will need to manage the BES proxy settings and set them in the libcurl library.

 

Thanks for the suggestion.

 

 

Please use plain text.