Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Java Development

Reply
Contributor
EricTheRed03
Posts: 32
Registered: ‎08-13-2012
My Device: 9800
Accepted Solution

Out of Memory Error

[ Edited ]

I'm currently trying to parse a really big JSON file but I am getting an out of memory error. This happens when I convert my "ByteArrayOutputStream" (baos) to string via .toString(). I see (via mouseover in debugger) that the size of this baos is 156815 (bytes I assume). 

 

I've read about a few potential problems / solutions:

- I could either have to free memory elsewhere in my program.

- There could be a memory leak. 

- JSON file may be too large.. so I should parse it "in chunks"

 

 

1. Could someone please explain how I could be creating a memory leak? Embarrassingly I didn't think this was possible in java

2. Also.. how can I free memory if I store and pop my screens one over the other?

3. Could you point me to a tutorial for parsing JSON in chunks.. I'm not sure how to even attempt this..

Developer
peter_strange
Posts: 19,609
Registered: ‎07-14-2008
My Device: Not Specified

Re: Out of Memory Error

I am not aware of a way to parse json in chunks sorry.

 

I don't think 15,000 bytes is too large, so I think we need to look for a memory hog in your current program.

 

There are tools in Eclipse to help look for things like this.  I'm only a recent Eclipse user so I have not yet played with all of these, specifically I have not had to investigate the Object usage in any application I've written with Eclipse yet.  So I'm not the right person to help here.  But if you search the forum you will find some Threads that talk about this, like this one:

http://supportforums.blackberry.com/t5/Java-Development/Profiling-with-Eclipse-plugin/m-p/557819#M11...

 

The first place I would look is the processing that creates this baos.  Do you read this from a Server?  Can we see that code?

 

Regarding leaks in Java, it is true that Java will not let you create leaks.  But I don't think this is a leak.  A leak means that you, over time, loose memory.  So your application works first time, maybe second too and third, but after sometime it gets this problem.  I think your processing fails every time.  So not a memory leak, just a memory hog.  [BTW It is possible to create leaks in BlackBerry Java if you use some of the APIs incorrectly]. 

 

In my experience, the most common problem that triggers this is a recursive method call.  You will see this in the stack trace in the Eclipse debugger. 

Contributor
EricTheRed03
Posts: 32
Registered: ‎08-13-2012
My Device: 9800

Re: Out of Memory Error

Thank you for the reply.

 

I have managed to take my networking code and paste it in an isolated test project. Furthermore, I see it still crashes (see below)..

 

so unless it is my networking code below then it must be that the baos is too big to be converted to a string. Your thoughts?

 

At this point I am learning towards writting a custom method to parse my byte array into string chunks..

 

try{	
			conn = (HttpConnection)Connector.open(url);
			conn.setRequestMethod(method);
			
			int responseCode = conn.getResponseCode();
			
			if(responseCode != HttpConnection.HTTP_OK && responseCode != HttpConnection.HTTP_CREATED)
			{
				//_sessionData.SetRequestStatus(false, "Failed");
				conn.close();
				return;
			}
			
			String contentType = conn.getHeaderField("Content-type");
			ByteArrayOutputStream baos = new ByteArrayOutputStream();
			InputStream responseData = conn.openInputStream();
			byte[] buffer = new byte[10000];
			int bytesRead = responseData.read(buffer);
			while (bytesRead > 0)
			{
				baos.write(buffer, 0,bytesRead);
				bytesRead = responseData.read(buffer);
			}
			baos.close();
			conn.close();
			conn = null;


			try
			{
				str = baos.toString(); //application crashes here.
				str = str;
				
				_jsonResponse = new JSONObject(str);
				str = str;
			}
			catch (Exception e)
			{
				msg = e.toString();
				msg = msg;
			}
		}

 

Contributor
EricTheRed03
Posts: 32
Registered: ‎08-13-2012
My Device: 9800

Re: Out of Memory Error

I have found something very strange..

 

Converting the same byteArrayoutputstream to a char[] works fine.. but then char[] to string still fails.

 

Questions..

1/ When converting from bytearrayoutputstream.toString() I notice that the exception doesn't fall under "catch (exception e)" .. instead the application crashes.. would be nice to catch such a scenario..

 

2. Internally I see a "new String()" declaration in the method that work!.. why does this work?

 

char[] method1 = (new String(baos.toByteArray())).toCharArray();

 

and this method fail..

 

String method2 = new String(baos.toByteArray());

 

 

3. Here is the profiler data and the size of the object created. Does anything look suspicious here?

 

Untitled.png

 

Developer
peter_strange
Posts: 19,609
Registered: ‎07-14-2008
My Device: Not Specified

Re: Out of Memory Error

Sorry I can't answer your questions.

 

I'm concerned that there is something else going on here because I don't think the code you are showing us should fail.  The two things I would be most concerened about are:

a) Are you running this on a SImualtor or device that has an OS that is later than the development environment

b) Are you running this code on the Event Thread.

 

Anyway, can you replace this code, which I don't think is every efficient:

 

   ByteArrayOutputStream baos = new ByteArrayOutputStream();
   InputStream responseData = conn.openInputStream();
   byte[] buffer = new byte[10000];
   int bytesRead = responseData.read(buffer);
   while (bytesRead > 0)
   {
    baos.write(buffer, 0,bytesRead);
    bytesRead = responseData.read(buffer);
   }
   baos.close();
   conn.close();
   conn = null;


   try
   {
    str = baos.toString(); //application crashes here.
    str = str;
    
    _jsonResponse = new JSONObject(str);
    str = str;
   }
   catch (Exception e)
   {
    msg = e.toString();
    msg = msg;
   }

 

with the following:

 

InputStream responseData = conn.openInputStream();
byte [] responseBytes = IOUtilities.streamToBytes(responseData);
try {
str = new String(responseBytes, "UTF-8");
} catch (Thorwable t) {
t.printStacktrace();
msg = t.toString();
System.put.println("String conversino error: " + msg);
str = null;
}
if ( str != null ) {
try {
_jsonResponse = new JSONObject(str);
} catch (Thorwable t) {
t.printStacktrace();
msg = t.toString();
System.put.println("Json conversino error: " + msg);
}
}

Contributor
EricTheRed03
Posts: 32
Registered: ‎08-13-2012
My Device: 9800

Re: Out of Memory Error

[ Edited ]

Hi I wanted to give you guys an update. It took me a long time to realize that if you disable line breaks (the debugger) then the conversion to string works ! I'm 100% sure it is this. Thanks for the reply!!

Developer
peter_strange
Posts: 19,609
Registered: ‎07-14-2008
My Device: Not Specified

Re: Out of Memory Error

So a problem with the debugger rather than anything else!  How bizarre!

Developer
peter_strange
Posts: 19,609
Registered: ‎07-14-2008
My Device: Not Specified

Re: Out of Memory Error

Just had an "out of memory" problem and wonder if it was the same problem as you had.  It was NOT in the BlackBerry, this was an Eclipse problem.  It came up with a wee popup in the middle of the screen telling me there was an out of memory problem.  So this was a popup on my Windows machine, not an error that I saw on the BlackBerry Simulator. 

 

Is this what you saw too?

Contributor
EricTheRed03
Posts: 32
Registered: ‎08-13-2012
My Device: 9800

Re: Out of Memory Error

I don't believe so..

 

The error I saw was when I would mouseover the String variable (which had been converted from my ByteArrayOutputStream). 

 

After the mouseover the simulator would crash and the eclipse debugger would point to my string variable showing an out of memory error

Developer
peter_strange
Posts: 19,609
Registered: ‎07-14-2008
My Device: Not Specified

Re: Out of Memory Error

".. the simulator would crash ...."

 

When you get a real out of memory error on the Blackberry, it kills the Thread that was running, you may see a popup on the SImulator screen, and you will see an entry in the debug log.  So this looks like an Eclipse problem rather than a Blackberry problem.  Sorry I have been telling you the wrong things to do. 

 

That said I don't know the right things....