01-24-2014 10:45 AM
I have a long running headless app which uploads files of 6Mb upwards to a web server. Now using the QNetworkAccessManager to post the file, I can see memory usage shoot up and soon exceed the 3Mb limit.
The QNetworkAccessManager then suffers bad allocations and crashes.
I was kind of hoping the QNetworkAccessManager would chunk the data from the file off the disk and keep the memory footprint low, however it appears it tries to suck the whole file in.
Any means of limiting or controlling the behaviour of QNetworkAccessManager?
Solved! Go to Solution.
01-25-2014 09:57 AM
Thanks. You are already using the call I was going to suggest. This poses an interesting problem. I will have to raise with my colleagues who came up with the headless app memory limits. I will have to get someone to check the implementation of the QNetworkManager class to see if it always buffers the whole request in memory before sending it. It shouldn't because then very large file uploads via HTTP POST would fail. The minimum buffer may need to be reduced but we still need to find a way to make this work in the interim until that gets fixed.
01-25-2014 10:07 AM
I just thought of something. How are you encoding the file that you are sending? Are you assuming that the post is encoding it correctly for you? I am wondering if switching to the
call will get around this problem. See the reference for QHttpMultipart for how to setup a multipart with a file attachment. I am guessing that it does not bring the whole file into memory when building the request.
Try that and let me know if it helps.
01-27-2014 06:17 AM - edited 01-27-2014 06:18 AM
I tried the QHttpMultipart and attached the file as per the example. I ran it up and watched the memory usage blast past 3600k before bad allocations were thrown from the QNetworkAccessManager.
I figure they share the same routines for reading the file.
Nice idea but still can't move forward at the moment.
01-28-2014 01:01 PM
We started an internal investigation to check the QNetworkAccessManager implementation. One suggestion made to try next was to try using libcurl to perform the file upload.
01-30-2014 10:10 AM
I can confirm that using libcurl doesn't signicantly increase memeory usage when transferring multi-megabyte files and I was able test this in a headless app successfully.
However compared with QNetworkAccessManager you will need to manage the BES proxy settings and set them in the libcurl library.
Thanks for the suggestion.