Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Java Development

Reply
New Contributor
Posts: 4
Registered: ‎06-10-2010
My Device: 8900
My Carrier: SSE

Download Combiner Class Problem

I have implemented Download Combiner
class in my code to download  large size file but it gives me a error message "Exceed the limit of  number of bytes per connection"

 

I have a pdf file  of size 1.3 mb 

and i am passing the chunck size 10240.

 

i am testing it on simmulator 9700

 

please suggest me where i am wrong  here is code

----------------------------------------------------------------------------------------------------------

public class DownloadCombiner extends Thread {
    private String remoteName;
    private String localName;
    private int chunksize;


    public DownloadCombiner(String remoteName, String localName, int chunksize) {
        this.remoteName = remoteName;
        this.localName = localName;
        this.chunksize = chunksize;

    }

    public void run() {
        try {
            int chunkIndex = 0;
            int totalSize = 0;
            /*
             * File connection
             */
            FileConnection file = (FileConnection) Connector.open(localName);
        
            if (file.exists())
            {
                file.delete();
            }
            else
            {
                file.create();
                file.setWritable(true);
                OutputStream out = file.openOutputStream();
                /*
                 * HTTP Connections
                 */
                String currentFile = remoteName;
                HttpConnection conn;
                InputStream in;
                int rangeStart = 0;
                int rangeEnd = 0;
            
                while (true) {
                    System.out.println("Opening Chunk: " + chunkIndex);
                     conn = (HttpConnection) Connector.open(currentFile,
                            Connector.READ_WRITE, true);
                    rangeStart = chunkIndex * chunksize;
                    rangeEnd = rangeStart + chunksize - 1;
                    System.out.println("Requesting Range: " + rangeStart + "-"
                            + rangeEnd);
                    
                        conn.setRequestProperty("Range", "bytes=" + rangeStart
                                + "-" + rangeEnd);
                
                    int responseCode = conn.getResponseCode();
                    if (responseCode != 200 && responseCode != 206) {
                        System.out.println("Response Code = "
                                + conn.getResponseCode());
                        break;
                    }
                    System.out.println("Retreived Range: "
                            + conn.getHeaderField("Content-Range"));
                    in = conn.openInputStream();
                    int length = -1;
                    byte[] readBlock = new byte[256];
                    int fileSize = 0;
                    while ((length = in.read(readBlock)) != -1) {
                        out.write(readBlock, 0, length);
                        fileSize += length;
                        Thread.yield(); // Try not to get cut off
                    }
                    totalSize += fileSize;
                    System.out.println("Chunk Downloaded: " + fileSize
                            + " Bytes");
                    chunkIndex++; // index (range) increase
                    in.close();
                    conn.close();
                
                    in = null;
                    conn = null;
                    /*
                     * Pause to allow connections to close and other Threads to
                     * run.
                     */
                    Thread.sleep(3000);
                }
                Dialog.alert("Full file downloaded: " + totalSize
                        + " Bytes");
                out.close();
                file.close();
            }
            //else {
            //    Dialog.alert("File already exists...");
            //    System.out.println("File already exists...");
            //}
        } catch (InterruptedException ex) {
            ex.printStackTrace();
            Dialog.alert(ex.getMessage());
        } catch (IOException ex) {
            ex.printStackTrace();
            Dialog.alert(ex.getMessage());
        }

    }
}

 

Thanks,

Rizwan Ullah

Developer
Posts: 1,474
Registered: ‎04-14-2009
My Device: Not Specified

Re: Download Combiner Class Problem

There is a limit on the number of bytes per connection over the MDS or BIS transport. In the MDS case the BES admin can change the limit. Search this forum for more information on this issue and the default limits. You need to be prepared for this exception and to handle it gracefully (and transparently to the user), probably by establishing a new connection and resuming whatever it is that you're doing.
Highlighted
Developer
Posts: 19,636
Registered: ‎07-14-2008
My Device: Not Specified

Re: Download Combiner Class Problem

[ Edited ]

Understand the post from klyubin, however I'm not sure it is appropriate in this case.  The download combiner processing attempts to break up the download into chunks and so should only get 10K per chunk.  Each chunk is obtained on a new connection.

 

So it would seem that in this case, the Server is not chunking the data for you, but instead giving you the whole file in one go. 

 

There are in my experience, two possible causes for this:

a) The Server does not support chunking - in other words the Server does not support HTTP 1.1. 

b) The Server does not chunk the file type that you are requesting.  For example in my experience servers don't chunk html, but will chunk zip files.

 

You can look at the Headers to find out this sort of stuff, but as a temporary measure, put a break point in your read/write loop and see how often you go round that.  Obviously if it exceeds 10K, then your data is not being chunked.

 

Can you tell what the Server is and also what is the file type of the file you are downloading.  Have you tried this code with any other file types and/or any other Servers?

 

Edit: Just noted you are downloading a pdf.  I have sample code that will download a pdf from RIM's site chunked.  So I would have thought that a .pdf file would be chunkable.

 

Also I don't think chunk is the official name for this process, it is just what I call it.  There is probably a more accurate name that someone will tell us....  So don't use chunk and expect everyone to understand.

Developer
Posts: 1,474
Registered: ‎04-14-2009
My Device: Not Specified

Re: Download Combiner Class Problem

Well-spotted, Peter! I missed the fact that the downloader already downloads in small chunks. One more hypothesis to investigate is that the OS is sending all these separate requests over one single HTTP/1.1 connection behind the scenes and doesn't transparently recover from the "too many bytes over one connection" error. One way to test this is to see if the server receives all these requests via the same connection or not, although that might not be fully conclusive as this connection is not the same as device -> BIS/MDS connection.