01-30-2012 06:40 PM - edited 01-30-2012 06:41 PM
I agree with Peter on this: the OP has to understand the solution. "Code samples" aren't likely to be usable within the framework of the application being developed.
If it is any consolation, we also had to resolve this issue. Taken at a most simplistic level, our solution is an implementaiton of the producer/consumer pattern. It involves a "job" queue containing HTTP request objects (each one corresponds to an image to be downloaded, and implements the Runnable interface). There is a thread pool, generally 2-3 threads. Jobs are placed in the queue by producers. Whenever there is a worker thread idle, it is assigned one of the jobs. When all of the jobs are completed, all threads are idle.
This type of pattern may (or may not) be applicable to your issue. "ymmv"
In my experience, having more than 2-3 threads running concurrent HTTP requests gives you drastically diminishing results. The pipe on the BB is only so wide, and there is a very finite limit to the total number of threads (as you discovered the hard way).
One thing that we did to improve performance was to implement image caching. We often found that images are requested multiple times - for example, a "4-star" and "5-star" rating image might be in 20 different list rows (where the list contents included restaurants, for example) . We consult the cach first, and use the cached image wherever possible. Again, this may (or may not) be applicable to your situation.
This is a non-trivial implementation. If done correctly, it will serve you for many projects in the future. It deserves careful investigation and design work.
01-30-2012 06:58 PM
Agree with RexDoug's additions.
And to demonstrate that there are other implementation, my favorite implementation is also a producer/consumer but with a queue of Runnables. Means I can throw anything at it, image downloads, data parsing, whatever. The Runnables have to implement something like the Observer pattern to make sure that the initiating process knows it has finished (or failed). While this is very flexible, it is also confusing and uncontrolled - three long running downloads will stop all other processing (assuming the pool is restricted to 3).
01-31-2012 02:42 AM
I am agree with the solution suggested by searingmedia, good solution if you can modify the server content.
rest you can download multiple images by a single thread as well by following steps.