Java Development

[itthipon]

Hi

I got OutofMemoryException when I uploading data with size more than 2 mb. I use http post by specifying

                    "Content-Type", "multipart/form-data; boundary=boundaryvalue"

in the request header.

The exception stacktrace from Bold 9000 is as below.

No detail message

net_rim_cldc(4AAABCA5)

 DataBuffer

 ensureBuffer

 0x3690

net_rim_cldc(4AAABCA5)

 DataBuffer

 write

 0x3C52

net_rim_crypto_1-3(4AAAC974)

 TLSOutputStream

 write

 0x45C7

net_rim_cldc-1(4AAABCA5)

 DataOutputStream

 write

 0x221D

net_rim_os-2(4AAAC894)

 ClientProtocol

 <private>

 0x1A1D

net_rim_os-2(4AAAC894)

 ClientProtocol

 writeRequest

 0x1422

net_rim_os-3(4AAAC894)

 HttpProtocolBase

 transitionToState

 0x33D7

net_rim_os-2(4AAAC894)

 ClientProtocol

 transitionToState

 0x23D4

net_rim_os-3(4AAAC894)

 HttpOutputStream

 flush

 0x334C

....

When I test on simulator 8300, i got EOFException.

No detail message
net_rim_os-2
 ClientProtocol
 <private>
 0x1917
net_rim_os-2
 ClientProtocol
 readResponse
 0x148A
net_rim_os-3
 HttpProtocolBase
 transitionToState
 0x1613
net_rim_os-2
 ClientProtocol
 transitionToState
 0x2335
net_rim_os-3
 HttpOutputStream
 flush
 0x157E

...

This code seems to work for the file with size less than 2 mb.

Any suggestions would be very appeciated.

Thanks,

Itthipon

[itthipon]

I found some information from this forum that Socket connection can be used to upload big file. So I gave this a try. My rough implementation is like blow.

 

p_url parameter in method getConnection has been appended with BB connection parameters for the corresponding connection type. 

private SocketConnection getConnection(String p_url)
{
	StringBuffer fullUrl = new StringBuffer( );
	fullUrl.append( m_url );
	if( transportExtras1 != null ) {
		fullUrl.append( transportExtras1 );
	}
	if( transportExtras2 != null ) {
		fullUrl.append( transportExtras2 );
	}
	if( extraParameters != null ) {
		fullUrl.append( extraParameters );
	}


	SocketConnection sc = null;
	SecureConnection ssc = null;
	if(this.m_url.startsWith("ssl"))
	{
		ssc = (SecureConnection) Connector.open(fullUrl.toString());
		//ssc.setSocketOption(SocketConnection.LINGER, 10);
		//ssc.setSocketOption(SocketConnection.KEEPALIVE, 1);
		//SecurityInfo info = ssc.getSecurityInfo();
		//boolean isTLS = (info.getProtocolName().equals("TLS"));
		
		sc = ssc;
	}		
	else
	{
		sc = (SocketConnection) Connector.open(fullUrl.toString());
		//sc.setSocketOption(SocketConnection.LINGER, 10);	
		//sc.setSocketOption(SocketConnection.KEEPALIVE, 1);
	}
	return sc;
}

String m_uploadImageSocketURL = "https://targetsite.com:443/upload.aspx";
byte[] m_fileBytes = byte array of files to be uploaded
String m_boundaryMessage =  beginning boundary string
String m_endBoundary = ending boundary string
 
private String  upload()
{
	try
	{
		//get base 64 encoded user's credential
		byte[] userCredential = getUserCredential();
		
		//Add HTTP Header
		int doubleSlash = m_uploadImageSocketURL.indexOf("//");
		int slash = m_uploadImageSocketURL.indexOf("/", doubleSlash + 2);
		int dot = m_uploadImageSocketURL.lastIndexOf(':');
		String uploadPaht = m_uploadImageSocketURL.substring(slash);
		String hostName = m_uploadImageSocketURL.substring(doubleSlash + 2, slash);	
		reqProp.append("POST " +  uploadPaht + " HTTP/1.1 \r\n");
		reqProp.append("Host: " + hostName + "\r\n");
		reqProp.append("User-Agent:" + HttpConnectionFactory.getUserAgent() + "\r\n");
		reqProp.append("Accept: text/html \r\n");
		reqProp.append("Authorization: Basic " + new String(userCredential) + "\r\n");
		reqProp.append("Content-Type: multipart/form-data; boundary=" + m_boundary + "\r\n");
		
		int rc;
		String url = m_uploadImageSocketURL;

		sc = getSocketConnection(url);	

		ByteArrayOutputStream bos = new ByteArrayOutputStream();
		bos.write(m_boundaryMessage.getBytes());
		bos.write(m_fileBytes);
		bos.write(m_endBoundary.getBytes());
		
		byte[] encodedPostContent = bos.toByteArray();

		reqProp.append("Content-Length: " + encodedPostContent.length + "\r\n");
		//empty line indicating the end of header/the beginning of contents
		reqProp.append("\r\n");
		
		is = sc.openInputStream();
		StringBuffer buffer = new StringBuffer();
		os = sc.openOutputStream();
		//Write Header
		os.write(reqProp.toString().getBytes());					
		
		//Write Contents
		for (int i = 0; i < encodedPostContent.length; i++)
		{			
			os.write(encodedPostContent[i]);	
		}
		os.flush();
		
		//Get response string
		int ch;					
		while ((ch = is.read()) != -1) {
			buffer.append((char) ch);
		}
			
		//Remove response header from response body message
		m_content = "";
		if(buffer.length() > 0)
		{
			int headerEndIndex = buffer.toString().indexOf("\r\n\r\n");
			if(headerEndIndex > -1)
			{
				m_content = buffer.toString().substring(headerEndIndex + 4);
			}
		}					
			
		bos.close();		
	}
	catch(Exception ex)
	{
		m_ex = (Exception)ex;
	}
	finally 
	{	
		try
		{
			if (is != null)
				is.close();
			if (os != null)
				os.close();
			if (sc != null)
				sc.close();
		}
		catch (Exception ex)
		{
			
		}
	}
}

fullUrl.toString()); //sc.setSocketOption(SocketConnection.LINGER, 10); //sc.setSocketOption(SocketConnection.KEEPALIVE, 1); } return sc; }

 The implementation above can upload file up to about 3 MB from simulator but when I tried on device, uploading 3 MB file doesn't work. Anyway there is no error happen but the file isnot actually uploaded to server. Uploading small file (52 KB) works on device.

Anyone has any idea or suggestion?

Thanks in advance,

Itthipon

[MSohm]

What BlackBerry device software version are you testing on?  You can find this under Options, About on the BlackBerry Smartphone.  You could be hitting a byte array limit in 4.5 and 4.6.  Do you see the same results in 5.0?

What line of code is triggering the exception?

[itthipon]

My Blackberry device software version that I am testing is 4.6.0.303.

I tried on simulator for 5.0 but I got an error occuring when debugger is starting up

org.eclipse.jdi.TimeoutException
at org.eclipse.jdi.internal.connect.SocketTransportService.attach(SocketTransportService.java:151)
at org.eclipse.jdi.internal.connect.SocketTransportImpl.attach(SocketTransportImpl.java:43)
at org.eclipse.jdi.internal.connect.SocketAttachingConnectorImpl.attach(SocketAttachingConnectorImpl.java:118)
at org.eclipse.jdt.internal.launching.SocketAttachConnector.connect(SocketAttachConnector.java:139)
at net.rim.ejde.A.L.E.doConnect(RIMEIDE:1273)
at net.rim.ejde.A.L.E.debug(RIMEIDE:769)
at net.rim.ejde.A.L.K.launch(RIMEIDE:608)
at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:853)
at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:703)
at org.eclipse.debug.internal.ui.DebugUIPlugin.buildAndLaunch(DebugUIPlugin.java:866)
at org.eclipse.debug.internal.ui.DebugUIPlugin$8.run(DebugUIPlugin.java:1069)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

For 4.6.1 on device, I got the following error:

Name: SocketBaseIOException

GUID: 9c3cd62e3320b498

Time: Sep 29, 2010 18:09:13

Connection reset by peer

net_rim_os-3(4AAAC894)

 ConnectionBase

 receive

 0xF9C

net_rim_os-3(4AAAC894)

 StreamProtocol

 getMoreInput

 0x9D18

net_rim_os-3(4AAAC894)

 SocketInputStream

 <private>

 0x89CB

net_rim_os-3(4AAAC894)

 SocketInputStream

 read

 0x8846

net_rim_cldc-1(4AAABCA5)

 DataInputStream

 read

 0x1DE7

net_rim_crypto_2-2(4AAAC9DD)

 SSLRecordProtocol

 read

 0x424C

net_rim_crypto_1-3(4AAAC974)

 TLSInputStream

 read

 0x43F6

This error happens after flush the output stream and try reading http response from input stream.

int ch; 
while ((ch = is.read()) != -1) {
buffer.append((char) ch);
}

For 4.5 on simulator, the similar exception as above occurs.

If you need more information, please let me know.

Thank you again,

Itthipon

[itthipon]

I also notice that the socket version of code doesn't throw exception on os.flush()

but [SocketBaseIOException : Connection resset by peer ] occurs when reading response from input stream. This seems to be the same as the result from app that compiled with 4.6.1.

The event log is here.

Name: SocketBaseIOException

GUID: 9c3cd62e3320b498

Time: Oct 01, 2010 10:14:24

Connection reset by peer

net_rim_os-3(4AAAC894)

 ConnectionBase

 receive

 0xF9C

net_rim_os-3(4AAAC894)

 StreamProtocol

 getMoreInput

 0x9D18

net_rim_os-3(4AAAC894)

 SocketInputStream

 <private>

 0x89CB

net_rim_os-3(4AAAC894)

 SocketInputStream

 read

 0x8846

net_rim_cldc-1(4AAABCA5)

 DataInputStream

 read

 0x1DE7

net_rim_crypto_2-2(4AAAC9DD)

 SSLRecordProtocol

 read

 0x424C

net_rim_crypto_1-3(4AAAC974)

 TLSInputStream

 read

 0x43F6

[MSohm]

You've posted a few different problems here (exception when uploading, when reading input and when connecting to the debugger).  Are you still experiencing all 3 issues?  Let's work through these one at a time.

Can you provide a sample and description for each issue and outline the steps to reproduce the problem you are seeing?

[itthipon]

The problems are in 2 main cases.
 1. Uploading via HttpConnection
 2. Uploading via SockectConnection

For the first case: Uploading via HttpConnection, I have the code below.

try 
{
	byte[] userCredential = getUserCredential();
	
	Hashtable requestProperties = new Hashtable();

	//Add the authorized header.
	requestProperties.put("Authorization", "Basic " + new String(userCredential));
	requestProperties.put("Content-Type", "multipart/form-data; boundary=" + m_boundary);

	int rc;
	String url = m_uploadImageURL;

	c = getHttpConnection(url, HttpProtocolConstants.HTTP_METHOD_POST);

	if (requestProperties != null && requestProperties.size() > 0)
	{
		Enumeration enumeration = requestProperties.keys();
		String key;
		String value;
		
		while (enumeration.hasMoreElements())
		{
			key = (String)enumeration.nextElement();
			value = (String)requestProperties.get(key);
			
			c.setRequestProperty(key, value);
		}
	}

	ByteArrayOutputStream bos = new ByteArrayOutputStream();
	bos.write(m_boundaryMessage.getBytes());
	bos.write(m_fileBytes);
	bos.write(m_endBoundary.getBytes());
	
	byte[] encodedPostContent = bos.toByteArray();

	c.setRequestProperty("Content-Length", "" + encodedPostContent.length);

	// Getting the output stream may flush the headers
	os = c.openOutputStream();
	
	for (int i = 0; i < encodedPostContent.length; i++)
	{
		os.write(encodedPostContent[i]);
	}
	
	os.flush(); 
	
	bos.close();

	rc = c.getResponseCode();
	if (rc != HttpConnection.HTTP_OK) {
		String msg = new String();
		msg = "Error " + rc + "\n" + c.getResponseMessage();

		throw new IOException("Service request failed with message: \n " + msg);
	}
	
	is = c.openInputStream();
	StringBuffer buffer = new StringBuffer();
	int ch;
	while ((ch = is.read()) != -1) {
		if (isCancel() == true)
		{
			return;
		}
		
		buffer.append((char) ch);
	}
	m_content = buffer.toString();
}
catch (Throwable ex) 
{
	m_ex = (Exception)ex;
}

 tested this on Bold 9000 device, there is OutOfMemoryException thrown during writing output stream.

Name: OutOfMemoryError
GUID: 9c3cd62e3320b498
Time: Oct 06, 2010 19:12:48
No detail message
net_rim_cldc(4AAABCA5)
 DataBuffer
 ensureBuffer
 0x3690
net_rim_cldc(4AAABCA5)
 DataBuffer
 write
 0x3C52
net_rim_crypto_1-3(4AAAC974)
 TLSOutputStream
 write
 0x45C7
net_rim_cldc-1(4AAABCA5)
 DataOutputStream
 write
 0x221D
net_rim_os-2(4AAAC894)
 ClientProtocol
 <private>
 0x1A1D
net_rim_os-2(4AAAC894)
 ClientProtocol
 writeRequest
 0x1422
net_rim_os-3(4AAAC894)
 HttpProtocolBase
 transitionToState
 0x33D7
net_rim_os-2(4AAAC894)
 ClientProtocol
 transitionToState
 0x23D4
net_rim_os-3(4AAAC894)
 HttpOutputStream
 flush
 0x334C

Let us start with the first case.

Thank you,

Itthipon

[MSohm · Accepted solution]

I was able to reproduce this when using an OutputStream.  Switching from OutputStream to DataOutputStream allowed me to post 3 MB of data.  Please give that a try and let me know if you run into any further issues.

[itthipon]

After changing to use connection.OpenDataOutputStream, I got this exception.

java.io.InterruptedIOException: Local connection timed out after ~ 120000

Do I need to set any property in connection param to fix this?

Thank you again for your help,

Itthipon

[MSohm]

The default timeout for a network connection on a BlackBerry Smartphone is 2 minutes.

Are you seeing the exception when attempting to make the connection or while it is in progress? Are you testing in the simulator or on a real device? There is no parameter to set this for an HTTP connection being made over direct TCP (there is a parameter when using the BES as the transport route).