Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Java Development

Reply
Contributor
gottsc
Posts: 12
Registered: ‎10-31-2012
My Device: 9320, developer
My Carrier: Telus
Accepted Solution

Consistent HTTP Timeout setting?

Hi all,
I've used HttpConnection for a few years and now I have a need to set the timeout for HTTP connections to less than 2 minutes. I've searched around in this forum and elsewhere and learned that I should use either ConnectionFactory, which has the ability to set a timeout, or I can use HttpConnection, and set a timer and interrupt the HTTP thread.

BUT-- as far as I can tell, both of these methods have variable results. Both the ConnectionFactory and the HttpConnection/Timer methods seem to work differently in OS 5, 6, and 7, and seem to work better in the simulators than physical devices (I tested on OS5 and OS7.1 BlackBerrys).

My best results occur when I use ConnectionFactory and set setTimeLimit() and setConnectionTimeout() to my timeout value, and then I add a Timer for the same timeout value. The code checks for sufficient connectivity by using getCoverageStatus() just before trying to open the HTTP connection. This seems to work well on OS5, 6, and 7 sims, and on live devices for OS5 and OS7.1.

I'm still experiencing one problem. There are times when nothing will stop the HTTP connection before the two minute mark. This occurs when I simulate bad reception just as the connection starts. I simulate reception cut-off by starting the connection and then placing the BlackBerry into a tin box to cut off reception. Most times, the timeout works. My timeout is set for 16 seconds. On a live 5.0 device, the timeout takes over 30 seconds (not what I want, but it indicates my code is having an effect).

But if I open the tin box at 30 seconds, and check it every 15 seconds with a quick look, I've had the timeout last 2 minutes and 40 seconds-- much longer than the default 2 minute timeout. I log where this is happening, and the 2:40 delay occurs inside this section of code:

    ConnectionFactory myConFactory = new ConnectionFactory();
    myConFactory.setTimeLimit(myFactoryTimeout);
    myConFactory.setConnectionTimeout(myFactoryTimeout);
    ConnectionDescriptor myConDesc = myConFactory.getConnection(pURL);

Ignoring whether I'm stressing the connectivity incorrectly by doing this test at all, my questions to the forum experts are:
Am I doing anything incorrectly, and is there anything else I can do to get an accurate timeout in every case?
Is there an explanation for this timeout failure?

The code follows. I've simplified it slightly. Thanks!


The request code:

    public static String HttpGetResponseString(String pURL, int pConnectionTimeoutSeconds)
    throws IOException, UnanticipatedHttpResponseException {
    String myReturn = "";
    HttpConnection myConn = null;        
    InputStream myInput = null;
    Timer connectionTimerConnect = null;
    long myConnectionTimeoutSeconds = pConnectionTimeoutSeconds * 1000;
    long myFactoryTimeout = myConnectionTimeoutSeconds;
    
    try {
        // Stop if the internet is not available.
        if (!getIsCoverageSufficient())
            throw new UnanticipatedHttpResponseException("No coverage available");

        HttpTimerTask timerTaskConnect = new HttpTimerTask(null);
        connectionTimerConnect = new Timer();
        connectionTimerConnect.schedule(timerTaskConnect, myConnectionTimeoutSeconds); // Kill the request after x seconds
        
        if (Constants.DEBUG_MODE)
            Prefs.getInstance().writeErrorLog("Timer Start: " + myConnectionTimeoutSeconds + "ms");

        // Connection request
        ConnectionFactory myConFactory = new ConnectionFactory();
        myConFactory.setTimeLimit(myFactoryTimeout);
        myConFactory.setConnectionTimeout(myFactoryTimeout);
        ConnectionDescriptor myConDesc = myConFactory.getConnection(pURL);
        if (myConDesc == null)
            throw new Exception("Connection factory did not return a descriptor");

        if (timerTaskConnect.hasInterrupted())
            throw new UnanticipatedHttpResponseException("Interrupted-Open");
        
        myConn = (HttpConnection)myConDesc.getConnection();
        
        final int responseCode = myConn.getResponseCode();
        
        if (timerTaskConnect.hasInterrupted())
            throw new UnanticipatedHttpResponseException("Interrupted-Response code");

        if (responseCode != HttpConnection.HTTP_OK) {
            throw new UnanticipatedHttpResponseException("Invalid ResponseCode: " + Integer.toString(responseCode));
        } else {
            myInput = myConn.openInputStream();
            byte[] responseData = new byte[10000];
            int length = 0;
            StringBuffer rawResponse = new StringBuffer();
            
            while (-1 != (length = myInput.read(responseData))) {
                rawResponse.append(new String(responseData, 0, length));
                if (timerTaskConnect.hasInterrupted())
                    throw new UnanticipatedHttpResponseException("Interrupted-Read");
            }
            myReturn = rawResponse.toString();
        }

    } catch(Exception e){
        Prefs.getInstance().writeErrorLog("Exception: " + e.getMessage());
        throw new UnanticipatedHttpResponseException(e.getMessage());
        
    } finally {
        if (connectionTimerConnect != null)
        {
            connectionTimerConnect.cancel();
            connectionTimerConnect = null;
        }
        
         try {
             if (myInput != null) {
                 myInput.close();
                 myInput = null;
             }

         } catch(Exception e){}
         
         try {
             if (myConn != null) {
                 myConn.close();
                 myConn = null;
             }                
         } catch(Exception e){
             // Connection Closed Exception
         }  
    }
    return myReturn;
}


The Timertask:

public class HttpTimerTask extends TimerTask {
    HttpConnection _myConn = null;
    Thread _myThread = null;
    boolean _myHasInterrupted = false;
    
    public HttpTimerTask(HttpConnection myConn)
    {
        _myConn = myConn;
        _myThread = Thread.currentThread();
    }
    public boolean hasInterrupted()
    {
        return _myHasInterrupted;
    }
    public void run() {
        String myErrorName = "HttpTimerTask: ";
        try {
            if (_myConn != null)
            {
                _myConn.close();
                _myConn = null;
                if (Constants.DEBUG_MODE)
                    Prefs.getInstance().writeErrorLog(myErrorName + "Closed connection");
            }
            if (_myThread != null)
            {
                _myThread.interrupt();
                _myHasInterrupted = true;
                if (Constants.DEBUG_MODE)
                    Prefs.getInstance().writeErrorLog(myErrorName + "Interrupted thread");
            }
            
        } catch (IOException e) {}
    }
}

Please use plain text.
Contributor
gottsc
Posts: 12
Registered: ‎10-31-2012
My Device: 9320, developer
My Carrier: Telus

Re: Consistent HTTP Timeout setting?

Bump... does no one else have this problem?

 

FYI the code seems to work except in the situation I've described and hopefully will help others to set up timeouts. Please post fixes to the code if you find problems.

Please use plain text.
Developer
peter_strange
Posts: 19,601
Registered: ‎07-14-2008
My Device: Not Specified

Re: Consistent HTTP Timeout setting?

I don't use ConnectionFactory myself, so a few comments on what I do, then some suggestions for you.

 

There are various methods that will honor a timeout, for example, for BES/MDS connections you can set a ConnectionTimeout on the URL in the connection string.  This works fine.  For other connection methods, there is no way to set a timeout. 

 

So what I do is always set the timeout if I can on the connection, but also set a cancel timer for the timeout value plus 5 seconds.  This will be cancelled if I get a response.  In this timeout processing I force close the http connection, killing the processing Thread will not work, and is not in fact what i want to do anyway, I want the processing Thread t continue but in an 'error' situation.  The timer is started just before the 'getResponseCode and cancelled immediately after.

 

This seems to work fine for me. It will not work correctly if I used https processing because, for https processing, the connection is actually started by the open.  For standard http processing, the getResponseCode() usually initiates it.  So if I was checking https processing, I should really start the time on the open. 

 

If we look at your code, the difference is that you use ConnectionFactory to get your connection.  Now as I understand it:

ConnectionDescriptor myConDesc = myConFactory.getConnection(pURL);

need not return immediately, I think ConnectionFactory has the right to confirm that the connection it is giving you actually will work and this might take some time, perhaps not for the connection you do get, but for the connection methods that it attempted and failed to use. 

 

Note that as I understand it too, ConnectionFactory will only use the Timeout value if it can.  It is advisory, not mandatory. 

 

The second big difference is that I will close the connection.  You start the timing before you have an connection, so you can't do that.  Instead you interrupt the Thread.  As far as I know, this will have no impact on an http connection.  If your processing gets to the

myConn.getResponseCode();

line, before you interrupt it, then it will not be interrupted.

 

I would change your timeout processing to perform a call back in your code so that the call back can set a 'timed out' flag and close the http connection (if it has one).  Then you should test the 'timed out' flag before getting the response, and in the Exception handling, to see if the exception was just a result of the time-out.  I suspect this might work better. 

 

Hope this helps. 

Please use plain text.
Contributor
gottsc
Posts: 12
Registered: ‎10-31-2012
My Device: 9320, developer
My Carrier: Telus

Re: Consistent HTTP Timeout setting?

Thank you for this insight, Peter.

I forgot to mention that I'm using BIS and HTTPS, and that I've tried using a HttpConnection with similar results to ConnectionFactory. The HTTPTimerTask class in my example can accept an optional HTTPConnection to close and can be used like so:

    myConn = (HttpConnection) Connector.open(pURL + getHTTPConnectionString());
    HttpTimerTask timerTask = new HttpTimerTask(myConn);

Your description of how HTTPS works is the key. I originally put the timer after I got the connection, as per descriptions given here in the forums, and passed in the connection for it to be closed. But there was no interrupt occurring. In my testing (using HTTPS) I could not prove that any blocking at all was occurring after the Connector.open() or the ConnectionFactory.getConnection() and I was puzzled by that.

With HTTPS, what I'm seeing is that all the blocking occurs before I get a connection to use, so I won't be able to use your recommended method-- but at least it now makes sense why I can't use it.

As to your last recommendation-- I think I'm doing this. In the HTTP thread, I use a check to the timer object,
    if (timerTaskConnect.hasInterrupted())
which checks if the timer task attempted to interrupt the HTTP thread. If the answer is yes, I throw an exception in the HTTP thread to stop in my tracks. Is this what you mean, or am I missing something?

All in all, it seems that the only control I have when attempting an HTTPS connection is to use a TimerTask and interrupt() the http thread. It generally works except when the internet connection is flakey (as in my tin-box test).

Please use plain text.
Developer
peter_strange
Posts: 19,601
Registered: ‎07-14-2008
My Device: Not Specified

Re: Consistent HTTP Timeout setting?

"With HTTPS, what I'm seeing is that all the blocking occurs before I get a connection to use, so I won't be able to use your recommended method-- but at least it now makes sense why I can't use it."

 

Well partially.  Yes there is an exchange that goes on before the actual request starts, but that is a higher level exchange and does not involve your server code.  So I suspect there will be a significant period of time spent in the

myConn.getResponseCode();

and I think the connection breaking will stop that.  So I think there is still value in my suggestion.

 

I missed this:

    if (timerTaskConnect.hasInterrupted())

but I think in your code you have potentially race condition.  I would, in

HttpTimerTask

set this first, before closing the connectin or anything:

_myHasInterrupted = true;

There is the possibility that your interrupt will be processed before you set the variable and so your interrupt will not be correctly detected. 

 

And I would put this check

        if (timerTaskConnect.hasInterrupted())
            throw new UnanticipatedHttpResponseException("Interrupted-Response code");

before

myConn.getResponseCode();

as well as after it. 

 

To a certain extent, I disagree with using the interrupt.  You need to terminate the connection and detect timeout before you do the

myConn.getResponseCode();

But once you are past that, do you want the timer task interrupting the read from the i/O stream?  You could be interrupting useful processing.  Just a thought.

 

Finally your stream read code needs work.  Look at

IOUtilities streamToByte()

You should be able to reduce the read loop code significantly - and get the correct result from a http connection, which is a byte array.  Then you can convert it to a String if you want.  But be aware, you should not translate the incoming bytes to a String unless you are 100% sure of the encoding.  The default conversion will screw up UTF-8 byte streams. 

Please use plain text.
Contributor
gottsc
Posts: 12
Registered: ‎10-31-2012
My Device: 9320, developer
My Carrier: Telus

Re: Consistent HTTP Timeout setting?

Good point- I'll test using two timer tasks, the first just handling the code prior to getting the HttpConnection and using an interrupt (I take your point about interrupts, but I find that without the interrupt, the code blocks for a long time here). And add a second timer to just close the connection from that point on.

I'll implement your other suggestions as well, and look at streamToByte. The page encoding is single byte ASCII at present, but I need to plan for MBCS.

Thanks for these suggestions.

Please use plain text.
Developer
peter_strange
Posts: 19,601
Registered: ‎07-14-2008
My Device: Not Specified

Re: Consistent HTTP Timeout setting?

OK.

 

"... add a second timer"

 

It is possibly more efficient if you can get your first Timer to perform both functions, say by giving it the connection when you have it, and paving the code check to see if it has a connection?  I am always loath to start extra Threads on a device like the BB.

 

Please let us know how you get on, as your logic might provide a good template for other people. 

Please use plain text.
Contributor
gottsc
Posts: 12
Registered: ‎10-31-2012
My Device: 9320, developer
My Carrier: Telus

Re: Consistent HTTP Timeout setting?

[ Edited ]

I've made the changes Peter suggested and have been testing the new code, and also changed back to using httpconnection rather than connectionfactory. It seems to work fairly well but there can be occasional hangups if there is a connectivity failure during the open().

I added a method to set the HTTP Connection in the HttpTimerTask class, so that I can start the HttpTimerTask with a null HTTPConnection, and then right after the Connector.open() call, I set the HTTP Connection inside the HttpTimerTask. This means I'm only starting one thread for timertask purposes. The code to stop the HTTP connection now looks like this:
    try {
        synchronized (_myLock){
            if (_myConn != null)
            {
                _myConn.close();
                _myConn = null;
                _myHasInterrupted = true;
            }
            else if (_myThread != null)
            {
                _myThread.interrupt();
                _myHasInterrupted = true;
            }
        }            
    } catch (Exception e) {}
    
The thread.interrupt() is executed only if no connection is available to close. Note Peter's point about HTTPS means it's likely that if the open is normal HTTP, there will be no blocking in open() and therefore thread.interrupt() will never be executed.

Hope this helps others!

Please use plain text.
Developer
peter_strange
Posts: 19,601
Registered: ‎07-14-2008
My Device: Not Specified

Re: Consistent HTTP Timeout setting?

Thanks for the update.

 

In fact I have been testing this area in detail and now think that that the following is not always true:

 

"Note Peter's point about HTTPS means it's likely that if the open is normal HTTP, there will be no blocking in open()"

 

My testing suggests that a normal non http open may block.  This seems to happen when the device thinks the connection should be present, and then tries to use the connection at open time to confirm that the URL is valid, i.e. does some DNS look-up.  At least it was in there when I debugged the hold up. 

 

But the circumstances in which I have seen this are difficult to repeat and involve deliberately turning off WiFi, which is hardly a normal thing to do.  But I think there is value in starting your timeout before the open. 

Please use plain text.