Welcome!

Welcome to the Official BlackBerry Support Community Forums. This is your resource to discuss support topics with your peers, and learn from each other. New to the forum? Please visit the ‘Getting Started’ link below.
inside custom component

Java Development

Reply
New Developer
deltagreen
Posts: 13
Registered: ‎03-01-2009
Accepted Solution

UDPDatagramConnection unexpected behavior while receiving datagram packets

I am experiencing some troubles with the class UDPDatagramConnection, and would hope the community could direct me to a solution. I almost got it to work, and it does work for some time, but then everything blocks, and I am not sure I understand why (although I have one theory that I would like to communicate and see if that makes sense). here are some details on the implementation.

 

- The objective:

 

Create a Midlet BlackBerry client that communicates with my .NET server through UDP sockets. I already have a functional .NET client that communicates with the same server, so server troubles are not probable.

 

public class UDPRepeaterCommunicationLayerClass extends CBaseMessageWrappingClass implements CommunicationInterface
{
...

private UDPDatagramConnection m_receiveSocket = null;
private UDPDatagramConnection m_sendSocket = null;
private Thread m_UDPreceiveThread = null;
private String m_connectAddress;

...

 

    UDPRepeaterCommunicationLayerClass(String dAddress, UnsolicitatedDisconnectNoticeable dDelegate, JitteringThresholdReachedDelegatable

dJitterDelegate, boolean isServer) throws IOException
    {   
        super(7);
        m_messagesToProcess = new Vector();
        m_pendingACK = new Hashtable();
        m_connectAddress = dAddress;
        m_isServer = isServer;
        m_unsolicitedDisconnectToCall = dDelegate;
        m_jitteringFallbackToCall = dJitterDelegate;
        ConstructSocket(m_connectAddress);
    }

 

    // The connector is created here successfully...

   // I create one for receiving (the one I am experiencing problems with), and one for sending so that I do not end-up having
    // shared resources troubles (like using the same buffer and having to synchronize the methods for sending and receiving
    // while the receiving one is blocking - that would have resulted in a dead-lock...

 

private void ConstructSocket(String dAddress)
    {
        try
        {
            if (PhoneInputMidlet.m_debug) System.out.println("Verifying if coverage is sufficient.");
            if (!CoverageInfo.isOutOfCoverage())
            {
                if (PhoneInputMidlet.m_debug) System.out.println("Coverage is sufficient.");
                if (dAddress == null)
                {
                    // dAddress = UDPRepeaterCommunicationLayerClass.m_protocol + "://localhost:7180;7181/";
                    // For Simulator: 19780/rim.net.gprs
                    // For T-movile: 19780/internet3.voicestream.com
                    // For ATT: 19780/proxy
                    dAddress = UDPRepeaterCommunicationLayerClass.m_protocol + "://" + UDPRepeaterCommunicationLayerClass.m_ipAddress + ":7180";
                }
                if (PhoneInputMidlet.m_debug) System.out.println("Opening connector " + dAddress);
                m_connectorReceiveAddress = dAddress + ";3000";
                // m_connectorReceiveAddress result with String "datagram://192.168.1.102:7180;3000"
                m_connectorSendAddress = dAddress;
                m_receiveSocket = (UDPDatagramConnection)Connector.open(m_connectorReceiveAddress, Connector.READ_WRITE, true);
                m_sendSocket = (UDPDatagramConnection)Connector.open(m_connectorSendAddress, Connector.READ_WRITE, true);
                m_address = m_receiveSocket.getLocalAddress() + ":" + String.valueOf(m_receiveSocket.getLocalPort());
                if (PhoneInputMidlet.m_debug) System.out.println("Connector address obtained: " + m_address);
                SetMaxPacketSize(kDefaultMaxPacketSize);
            }
            else
            {
                if (PhoneInputMidlet.m_debug) System.out.println("Coverage is not sufficient.");
            }
        }
        catch (IOException e) {}
    }

 

    // Later on in the code, I call StartReceiveThread to start my receiving thread... That works fine...

 

public void StartReceiveThread(CommunicationInterface dCommIntf, MessageProcessable dDelegate)
    {
        CUDPReceiverThread dThreadObj = new CUDPReceiverThread(this);
        m_UDPreceiveThread = new Thread(dThreadObj);
        m_UDPreceiveThread.setPriority(Thread.MAX_PRIORITY);   // I have tried to not change the priority here also...
        m_UDPreceiveThread.start();
        super.StartReceiveThread(dCommIntf, dDelegate);
        SetMaxPacketSize(GetMaxPacketSize());
    }

 

    // This is the real functional entry-point of my receive thread. Note that I am yielding as I should
    // so that other threads have CPU...

 

private void UDPReceiveThreadEntryPoint() throws IOException
    {
        SetMaxPacketSize(GetMaxPacketSize());
        while (m_UDPreceiveThread != null)
        {
            DoOneReceiveCycle();
              Thread.yield();
        }
    }

 

    // This is where I expect the problem to be. I get in here without troubles, I even get some packets that are received successfully
    // and that pass my quality-control (check-sum, verification of the size compared to a size value passed in a header). But then,
    // everything comes to a halt even though my server is still bombarding the client with packets. That is not what I would have expect.
    // I would have expect the receive method not to block (even though it is a blocking call) provided there are always UDP packets coming.

    // The server will never send a packet that is greater than GetMaxPacketSize() which respects the transporter's limitations (gets to
    // 1428 bytes). See code after.

 

private boolean DoOneReceiveCycle() throws IOException
    {
        if (PhoneInputMidlet.m_debug) System.out.println("Entering DoOneReceiveCycle");
        boolean gotOne = false;
        try
        {
            if (m_receiveSocket != null)
            {
                Datagram dDatagram = m_receiveSocket.newDatagram(GetMaxPacketSize());
                dDatagram.reset();
                byte[] dBuffer = new byte[GetMaxPacketSize()];
                dDatagram.setData(dBuffer, 0, GetMaxPacketSize());
                if (PhoneInputMidlet.m_debug) System.out.println("Calling receive...");
                m_receiveSocket.receive(dDatagram);
                if (PhoneInputMidlet.m_debug) System.out.println("Receive done (got back with " + dDatagram.getData().length + " bytes)");
                System.arraycopy(dDatagram.getData(), 0, dBuffer, 0, dDatagram.getData().length);
                int nBytesRec = dDatagram.getData().length;
                m_bytesUsed += nBytesRec;
                CTransportHeader dHeader = new CTransportHeader(dBuffer);
                CommSucceeded();
                if (dHeader != null)
                {
                    int dLength = dHeader.m_packetSize;
                    gotOne = true;
                    try
                    {
                        if (PhoneInputMidlet.m_debug) System.out.println("Performing quality control on packet");
                        PerformQualityControl(nBytesRec, dHeader, dBuffer);
                        if (PhoneInputMidlet.m_debug) System.out.println("Processing packet (quality control succeeded)");
                        ProcessReceivedDatagram(dDatagram, dBuffer, dLength);
                        CommSucceeded();
                    }
                    catch (CQualityAssuranceException e)
                    {
                        System.out.println(e.toString());
                        System.out.println(e.getMessage());
                    }
                    catch (NegativeArraySizeException e)
                    {
                        System.out.println(e.toString());
                        System.out.println(e.getMessage());
                        gotOne = false;
                    }
                }
            }
            else
            {
                if (PhoneInputMidlet.m_debug) System.out.println("Can't call receive on a closed socket.");
            }
        }
        catch (InterruptedIOException e)
        {
            System.out.println(e.toString());
            System.out.println(e.getMessage());
        }
        catch (Exception e)
        {
            System.out.println(e.toString());
            System.out.println(e.getMessage());
            if (PermanentCommProblems())
            {
                if (m_receiveSocket != null)
                {
                    // We had some trouble with that datagram and socket... Let's reinitialize everything from scratch then...
                    m_receiveSocket.close();
                    m_receiveSocket = (UDPDatagramConnection)Connector.open(m_connectorReceiveAddress, Connector.READ_WRITE, true);
                }
            }
        }
        if (PhoneInputMidlet.m_debug) System.out.println("Exiting DoOneReceiveCycle");
        return gotOne;
    }

 

- The prayer:

 

Would anyone have a clue what I am doing wrong? The only hypothesis that I have is that threading, in java, is cooperative and not preemptive. And, as a result, there may not be a yield inside the blocking call receive. That would result into a dead-lock that would suck all the cpu away from my main thread that is required to analyze the messages that are received and send back ACKs. If that is the case, I am in a bad spot. If that is not the case, please let me know and hopefully you can point out to an element of solution to get that puppy running.

 

Thank you!

 

Please use plain text.
New Developer
deltagreen
Posts: 13
Registered: ‎03-01-2009

Re: UDPDatagramConnection unexpected behavior while receiving datagram packets

I just want to add an element of information in regards to this problem. The more I look at that problem, the more I believe it to be a threading issue.

 

The error that I get is the following:

 

Uncaught exception: Application MidletApp(155) is not responding; process terminated.

 

I believe the BlackBerry has some type of a WatchDog to ensure an application stays responsive (a little like Vista does), and such WatchDog would not be content of the less CPU the main UI thread gets provided that my communication thread is extremely busy receiving tons of data. Does that make sense?

Please use plain text.
New Developer
deltagreen
Posts: 13
Registered: ‎03-01-2009

Re: UDPDatagramConnection unexpected behavior while receiving datagram packets

FYI, here are my findings in regards to this problem.

 

1) receive, by definition a blocking call, DOES NOT internally call Thread.yield(). That, on its own, is sufficient to starve other threads from CPU and provided the WatchDog available on the BlackBerry, it is a serious flaw in architecture not to do so.

2) yield, yield and yield more... The java threading model that is available for the BlackBerry is cooperative (and not preemptive). As a consequence, in my initial code, I only had one yield() call and that was not enough. When I started putting yield() calls a bit everywhere, the problem went away. As a derivative of that finding, I extrapolate that the java VM does not yield internally in most (maybe all) of its calls (even the blocking receive call). That is disastrous, I believe, in regards to the acrobaties that we need to go through in order to get around that aspect.

3) In order to get around the starving of my other threads during the times that nothing is received, I had to send an empty datagram periodically (since we cannot even define a timeout). The ATT of the world will be happy about it since I use bandwidth in order to get around a design flaw (providing them with $$$).

 

If someone is in position to confirm my findings, I would appreciate it. Thank you.

Please use plain text.
New Developer
bbst_chris
Posts: 8
Registered: ‎03-19-2009

Re: UDPDatagramConnection unexpected behavior while receiving datagram packets

Did you receive any response from blackberry on this? We have the exact same error in our UDP code.
Please use plain text.
New Developer
deltagreen
Posts: 13
Registered: ‎03-01-2009

Re: UDPDatagramConnection unexpected behavior while receiving datagram packets

The posting that I have put myself are still accurate in order for my problem to have gotten away, but, there is another element that I did not post about that also needed to be implemented.

 

Since the receive call sometimes stalls (starving from CPU all other threads), I have put a timer that I supervise from another thread in order to terminate the thread executing the receive in the event that it stays on it for too long. I know it is pretty ugly piece of hack in order to get the things to work, but RIM threw a pretty serious curve ball at us with this flawed receive call that does not yield or have a functional timeout mechanism.

 

Do not forget also that if thread A and thread B have the same priority, they negociate their CPU time through a preemptive model. If thread A and thread B have different priority, the negociation of CPU time is on a cooperative model. That means that for cases such as this one (where the receive call does not yield), you need to ensure the thread has the same priority as the other thread that is verifying if it is stalled in order for at least both threads to have access to some CPU time. I am not clear if that finding is true only for the simulator environment and have not validated my findings on real BB yet. I would appreciate some validation on it if at all possible from another poster.

 

Good luck.

Please use plain text.
Administrator
MSohm
Posts: 12,957
Registered: ‎07-09-2008
My Carrier: Bell

Re: UDPDatagramConnection unexpected behavior while receiving datagram packets

BlackBerry applications have the concept of a main event thread.  The main event thread handles changes to the application's Ui, input from the user and input from the system.  Applications should not perform any blocking operations or perform a large amount of processing on the main event thread.  Doing so can trigger the error you are receiving.  This is due to the fact that the BlackBerry handheld believes the application has hung because it has stopped responding to events (sent to its main event thread).

 

All IO operations, blocking operations and CPU intensive processing should be performed in its own thread to prevent this.  You can read about this here:

 

Support - Process [ApplicationName] killed due to message queue overflow
Article Number: DB-00401

http://www.blackberry.com/knowledgecenterpublic/livelink.exe/fetch/2000/348583/800451/800783/Support...

 

What Is - Blocking operation not permitted on event dispatch thread
Article Number: DB-00521

http://www.blackberry.com/knowledgecenterpublic/livelink.exe/fetch/2000/348583/800451/800783/What_Is...

 

How To - Process incoming data
Article Number: DB-00466

http://www.blackberry.com/knowledgecenterpublic/livelink.exe/fetch/2000/348583/800451/800563/How_To_...

Mark Sohm
BlackBerry Development Advisor

Please refrain from posting new questions in solved threads.
Found a bug? Report it using the Issue Tracker
Please use plain text.
New Developer
jparmar
Posts: 21
Registered: ‎09-18-2009

Re: UDPDatagramConnection unexpected behavior while receiving datagram packets

Hi All,

 

I am facing the same issue... As i observed this is due to UDP Packets recieved from remote , that makes application queue to full and exception is thrown.

 

Does anyone have any workaround for this problem ?

 

Please reply it's very urgent.

Please use plain text.
New Developer
deltagreen
Posts: 13
Registered: ‎03-01-2009

Re: UDPDatagramConnection unexpected behavior while receiving datagram packets

public void run()
    {
        while ((m_isReceiving) && (m_receiveStallThread == Thread.currentThread()))
        {
            try
            {
                if (((m_lastReceiveCall != null) && (m_UDPreceiveThread != null)) || ((m_UDPreceiveThread == null) || (!m_UDPreceiveThread.isAlive())))
                {
                    // If we stalled on a receive call for more than X seconds, we restart the thread...
                    if (((m_lastReceiveCall != null) && ((new Date().getTime() - m_lastReceiveCall.getTime()) > kStallVerificationPeriod)) || ((m_UDPreceiveThread == null) || (!m_UDPreceiveThread.isAlive())))
                    {
                        synchronized(UDPRepeaterCommunicationLayerClass.m_threadMutex)
                        {
                            m_lastReceiveCall = null;
                            Thread cleaupThread = m_UDPreceiveThread;
                            int oldPort = -1;
                            UDPDatagramConnection oldSocket = m_socket;
                            if (oldSocket != null)
                            {
                                oldPort = oldSocket.getLocalPort();
                            }
                            m_UDPreceiveThread = null;
                            if (m_socket != null)
                            {
                                m_socket.close();
                                m_socket = null;
                            }
                            m_socket = (UDPDatagramConnection)Connector.open(m_connectorReceiveAddress, Connector.READ_WRITE, true);
                            if (m_socket != null)
                            {
                                m_cleanUpStack.addElement(m_socket);
                            }
                            CUDPReceiverThread dThreadObj = new CUDPReceiverThread(this);
                            m_UDPreceiveThread = new Thread(dThreadObj, "UDP Receive Thread");
                            m_UDPreceiveThread.setPriority(kUDPThreadPriorityNotReceive);
                            if ((oldPort != -1) && (oldPort != m_socket.getLocalPort()) && (IsConnected()))
                            {
                                SendData(m_sendDataAddress, CommunicationInterface.eMsgTypeReconnect, "", -1, UDPRepeaterCommunicationLayerClass.kDefaultInterval);
                            }
                            if (cleaupThread != null)
                            {
                                try
                                {
                                    cleaupThread.interrupt();
                                }
                                catch (Exception e)
                                {
                                    if (PhoneInputMidlet.m_debug)
                                    {
                                        System.out.println("**** exception in UDPRepeaterCommunicationLayerClass.run.1: " + e.toString());
                                        System.out.println(e.getMessage());
                                        e.printStackTrace();
                                    }
                                }
                            }
                            m_UDPreceiveThread.start();
                        }
                    }
                    else
                    {
                        PerformPeriodicalTasks();
                        if (m_lastReceiveCall != null)
                        {
                            Thread.sleep(kStallVerificationPeriod - (new Date().getTime() - m_lastReceiveCall.getTime()));
                        }
                        else
                        {
                            Thread.sleep(kStallVerificationPeriod);
                        }
                    }
                }
                else
                {
                    PerformPeriodicalTasks();
                    Thread.sleep(kStallVerificationPeriod);
                }
            }
            catch (Exception e)
            {
                if (PhoneInputMidlet.m_debug)
                {
                    System.out.println("**** exception in UDPRepeaterCommunicationLayerClass.run.2: " + e.toString());
                    System.out.println(e.getMessage());
                    e.printStackTrace();
                }
            }
            PerformPeriodicalTasks();
        }
    }
        
    public void Dispose()
    {
        if (IsConnected())
        {
            ClearAllCommRetries();
            if (m_sendDataAddress.length() > 0)
            {
                SendData(m_sendDataAddress, CommunicationInterface.eMsgTypeClientDisconnect, "", 5, UDPRepeaterCommunicationLayerClass.kDefaultInterval);
            }
            Date dStartTime = new Date();
            while ((IsConnected()) && (IsAwaitingACK()) && ((new Date().getTime()- dStartTime.getTime()) < 5000))
            {
                PerformPeriodicalTasks();
                try
                {
                    Thread.sleep(50);
                }
                catch (InterruptedException e) {}
            }
        }
        StopReceiveThread();
        if (m_socket != null)
        {
            try
            {
                m_socket.close();
                System.out.println("m_socket closed successfully.");
            }
            catch (IOException e) 
            {
                if (PhoneInputMidlet.m_debug)
                {
                    System.out.println("**** exception in UDPRepeaterCommunicationLayerClass.Dispose.1: " + e.toString());
                    System.out.println(e.getMessage());
                    e.printStackTrace();
                }
            }
            m_socket = null;
        }
        ClearAllCommRetries();
        super.Dispose();
    }
Please use plain text.
New Developer
deltagreen
Posts: 13
Registered: ‎03-01-2009

Re: UDPDatagramConnection unexpected behavior while receiving datagram packets

Explanations:

 

1) Create a thread (the stall thread) that monitors the receive thread (so two threads total). When the stall thread detects the receive thread to have hold for too much time, kill it and recreate it again.

 

2) Change the thread priority around the receive call...

 

if ((m_UDPreceiveThread == Thread.currentThread()) && (m_receiveStallThread != null))

                {

                    keepPriority = m_UDPreceiveThread.getPriority();

                    m_UDPreceiveThread.setPriority(kUDPThreadPriorityReceive);

                    m_receiveStallThread.setPriority(kUDPThreadPriorityReceive);

                }

                m_lastReceiveCall = new Date();

                m_socket.receive(dDatagram);

                m_lastReceiveCall = null;

                if ((m_UDPreceiveThread == Thread.currentThread()) && (m_receiveStallThread != null))

                {

                    m_UDPreceiveThread.setPriority(keepPriority);

                    m_receiveStallThread.setPriority(kUDPStallThreadPriorityNotReceive);

                    keepPriority = Integer.MIN_VALUE;

                } 

 

There is one reason for that. The threading model in java (or on the BlackBerry) is cooperative if the priorities are different, and preemptive if they are the same. Provided there is no yield into the receive call (that is blocking), you need to make it preemptive (so at the same priority level as the stall thread).

 

That should fix your issues as it did to me (I spent about two weeks on that). 

Please use plain text.
New Developer
jparmar
Posts: 21
Registered: ‎09-18-2009

Re: UDPDatagramConnection unexpected behavior while receiving datagram packets

Hi,

 

 

Thanks a lot for your such a nice descriptive reply.

I really appreciate your help.

 

 

But can you please say what should be the value of kUDPStallThreadPriorityNotReceive,  kUDPThreadPriorityReceive and

kStallVerificationPeriod

???

Please use plain text.