12-30-2013 07:09 PM - edited 12-30-2013 07:19 PM
This happens because Qt 4 uses select() internally which is limited to FD_SETSIZE sockets. Most likely the default value is 256, so no more than 256 simultaneous connections can be established.
This can also happen if you're creating and closing many connections in rapid succession. They aren't closed instantly unless SO_LINGER socket option is set to 0.
Be careful when using this option, as the data which was sent before the connection is closed won't be transmitted. I haven't used this option with Qt classes, only when working directly with sockets. If there's a way to get a socket handle from Qt class, this option can be probably set.
To increase the limit I suspect Qt has to be recompiled. This can be done, but binary size will be very large (~30-50 mb) and I'm not sure if recompiled version will be compatible with Cascades libs.
Optionally, use sockets directly. Before including any headers, define FD_SETSIZE to a higher value:
#define FD_SETSIZE 1024
1024 is the maximum value per process when using select() API.
upd. it seems QNX supports pool(), so switching to it could be another option if more than 1024 simultaneous sockets are required. But Qt network classes will be useless for this.
12-30-2013 07:59 PM
OK, I think the easier (and probably better way to do this) is to restructure the code.
The way my code works at the moment is as follows:
It loads profiles of people from a server. Each profile is turned into an object with various properties (eg. name, age etc).
The way it works at the moment is basically as per the example for ImageLoader : http://blackberry.github.io/Cascades-Samples/image
Each image is an ImageLoader object added to a model. However, I now see the problem, which is that there are sometimes 60 photos, which are all loaded at the same time.
I had one idea of using a singleton, but supposedly they are "frowned upon". What would you suggest? I really need to load the images all at once.
Also, does QNetworkAccessManager occupy a connection until destroyed? Because at the moment it is destroyed when the parent object is destroyed. But I can probably destroy it when the image is loaded.
12-30-2013 08:20 PM - edited 12-30-2013 08:48 PM
ImageLoader should be used with care. I see that it creates a new QNetworkAccessManager for every request, but never frees it. This is a leak, but it shouldn't hold the socket. I think socket is released on reply->deleteLater() call.
One QNetworkAccessManager per one connection is a possible scenario if you need to start all these requests simultaneously.
But 60 simultaneous requests will probably lag each other and may timeout. It's better to execute a few requests in parallel.
In a typical use case, there's only one QNetworkAccessManager instance and multiple connections. QNAM throttles them to a maximum of 6 simultaneous connections ("an internet friendly limit"). Alas, this value can't be configured, but the default seems to be a good choice. I would move QNAM to a singleton class and configure ImageLoaders to use this single QNAM instance.
Singletons are fine for situations like this.
p.s. I don't see where ImageProcessor is destroyed, this might be another leak.
upd: caching downloaded images in memory to avoid constantly redownloading them on scrolling can also be a good idea. I also suggest adding a size limit, droping older images when it's reached.
The image cache can be implemented as a separate class (possibly singleton, or just create it somewhere once and configure ImageDownloader instances with it).
The implementation can be pretty complex if done properly. Algorithm would be as follows:
When image url is requested:
- check the cache for url. If the image is in cache, return it.
- check if the url is being downloaded. If it is, subscribe to completion notification.
If it isn't, start downloading the url and add it to list of urls being downloaded.
- when the URL finishes downloading and the image is decoded:
remove the url from the list of urls being downloaded;
update cache with the new image;
send notification to all subscribers who were waiting for this url.
upd 2: consider switching to C++ ListView APIs if implementing this, it will be hard to do the subscribe/unsubscribe part with ListItemComponents.
This is just a generic algorithm. It can be simplified while still being efficient:
For example, ImageLoader concept can be dropped at all. There can be one ImageDownloader class which will emit urlReady(string url) signal when url finishes downloading and the image is added to the cache. All cells will subscribe to it and compare the url with the url they want to display. If it matches, they can query the image cache for corresponding image by url.
This still requires some thinking about, it's not an exact algorithm.
I hope this helps.
p.s. to everyone: if I'm overcomplicating things, please feel free to suggest other solutions. ))
12-30-2013 08:52 PM
12-30-2013 09:01 PM