Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Native Development

Reply
Contributor
cheechm
Posts: 26
Registered: ‎07-24-2009
My Device: Not Specified
Accepted Solution

Socket Problem

Getting this error:

 

[code]QEventDispatcherBlackberry: cannot register QSocketNotifier (fd too high)[/code]

 

What exactly causes this?

Contributor
cheechm
Posts: 26
Registered: ‎07-24-2009
My Device: Not Specified

Re: Socket Problem

Anyone?

Developer
Zmey
Posts: 1,512
Registered: ‎12-18-2012
My Device: PlayBook, Z10, DAC

Re: Socket Problem

[ Edited ]

This happens because Qt 4 uses select() internally which is limited to FD_SETSIZE sockets. Most likely the default value is 256, so no more than 256 simultaneous connections can be established.

This can also happen if you're creating and closing many connections in rapid succession. They aren't closed instantly unless SO_LINGER socket option is set to 0.

Be careful when using this option, as the data which was sent before the connection is closed won't be transmitted. I haven't used this option with Qt classes, only when working directly with sockets. If there's a way to get a socket handle from Qt class, this option can be probably set.

To increase the limit I suspect Qt has to be recompiled. This can be done, but binary size will be very large (~30-50 mb) and I'm not sure if recompiled version will be compatible with Cascades libs. :smileysad:

Optionally, use sockets directly. Before including any headers, define FD_SETSIZE to a higher value:
#define FD_SETSIZE 1024

1024 is the maximum value per process when using select() API.

 

upd. it seems QNX supports pool(), so switching to it could be another option if more than 1024 simultaneous sockets are required. But Qt network classes will be useless for this. :smileysad:

 


Andrey Fidrya, @zmeyc on twitter
Contributor
cheechm
Posts: 26
Registered: ‎07-24-2009
My Device: Not Specified

Re: Socket Problem

OK, I think the easier (and probably better way to do this) is to restructure the code.

 

The way my code works at the moment is as follows:

 

It loads profiles of people from a server. Each profile is turned into an object with various properties (eg. name, age etc).

 

The way it works at the moment is basically as per the example for ImageLoader : http://blackberry.github.io/Cascades-Samples/imageloader-src-app-cpp.html

 

Each image is an ImageLoader object added to a model. However, I now see the problem, which is that there are sometimes 60 photos, which are all loaded at the same time. 

 

I had one idea of using a singleton, but supposedly they are "frowned upon". What would you suggest? I really need to load the images all at once. 

 

Also, does QNetworkAccessManager occupy a connection until destroyed? Because at the moment it is destroyed when the parent object is destroyed. But I can probably destroy it when the image is loaded.

 

Thanks

Developer
Zmey
Posts: 1,512
Registered: ‎12-18-2012
My Device: PlayBook, Z10, DAC

Re: Socket Problem

[ Edited ]

ImageLoader should be used with care. I see that it creates a new QNetworkAccessManager for every request, but never frees it. This is a leak, but it shouldn't hold the socket. I think socket is released on reply->deleteLater() call.

 

One QNetworkAccessManager per one connection is a possible scenario if you need to start all these requests simultaneously.

 

But 60 simultaneous requests will probably lag each other and may timeout. It's better to execute a few requests in parallel.

 

In a typical use case, there's only one QNetworkAccessManager instance and multiple connections. QNAM throttles them to a maximum of 6 simultaneous connections ("an internet friendly limit"). Alas, this value can't be configured, but the default seems to be a good choice. I would move QNAM to a singleton class and configure ImageLoaders to use this single QNAM instance.

 

Singletons are fine for situations like this.

 

p.s. I don't see where ImageProcessor is destroyed, this might be another leak.

 

upd: caching downloaded images in memory to avoid constantly redownloading them on scrolling can also be a good idea. I also suggest adding a size limit, droping older images when it's reached.

 

The image cache can be implemented as a separate class (possibly singleton, or just create it somewhere once and configure ImageDownloader instances with it).

 

The implementation can be pretty complex if done properly. Algorithm would be as follows:

 

When image url is requested:

- check the cache for url. If the image is in cache, return it.

- check if the url is being downloaded. If it is, subscribe to completion notification.

If it isn't, start downloading the url and add it to list of urls being downloaded.

- when the URL finishes downloading and the image is decoded:

remove the url from the list of urls being downloaded;

update cache with the new image;

send notification to all subscribers who were waiting for this url.

 

upd 2: consider switching to C++ ListView APIs if implementing this, it will be hard to do the subscribe/unsubscribe part with ListItemComponents. 

 


 

This is just a generic algorithm. It can be simplified while still being efficient:

 

For example, ImageLoader concept can be dropped at all. There can be one ImageDownloader class which will emit urlReady(string url) signal when url finishes downloading and the image is added to the cache. All cells will subscribe to it and compare the url with the url they want to display. If it matches, they can query the image cache for corresponding image by url.

 

This still requires some thinking about, it's not an exact algorithm.

 

I hope this helps. :smileyhappy:

 

 

p.s. to everyone: if I'm overcomplicating things, please feel free to suggest other solutions. ))

 


Andrey Fidrya, @zmeyc on twitter
Contributor
cheechm
Posts: 26
Registered: ‎07-24-2009
My Device: Not Specified

Re: Socket Problem

Very helpful thanks. I worked out what would be suitable for me. I've already actually half implemented it.

At the moment:

- Profile object requests creates new ImageLoader object
- The ImageLoader object checks to see if it is saved (as the images frequently recur between profiles)
- If it is cached, it emits the image ready signal and the image is made availabe
- If it isn't cached then a QNAM is created and the image is downloaded

What I am going to do instead is change it so that all instantiations of ImageLoader share the same QNAM. It won't actually make too much of a difference to how the code works because the imageChanged() signal will still only be emitted once the request is completed.
Developer
Zmey
Posts: 1,512
Registered: ‎12-18-2012
My Device: PlayBook, Z10, DAC

Re: Socket Problem

The image can also be in "downloading" state. If another request arrives at this stage, it will be downloaded twice. But this is a minor issue, the image will be overwritten in cache with the image from newer request. So, this seems to be a simple and good solution.

Andrey Fidrya, @zmeyc on twitter
Contributor
cheechm
Posts: 26
Registered: ‎07-24-2009
My Device: Not Specified

Re: Socket Problem

The image can only be requested once, luckily!