10-24-2009 12:08 AM
I did some more test, it seems that what peter_strange said is right. The data is buffered somewhere.
I gave each pocket sent by the device a serial number and check when server can get it. The result is that there is a big, about 1 minute delay. For example, on the device side packet 1 was reported sent(after flush()) at 16:00:05:348, on the server side, it was received 16:01:23:434, more than one minute of delay(clock difference is calculated into that, so that 1 minute is really delay).
That says the pocket is not actually sent even after flush() call. Now my question is, is there a way to change that? 1 minute delay is too much. I do have another thread, constantly reading(blocking) data from the server side. Peter suggested reading will flush the buffer, that seems not working for me. Is there a setting or something to make flush() send right away?
I know how to log http traffic in MDS server, but I don't know how to diagnose a socket connection in MDS log, can someone give me some suggestions of what to look for in MDS log?
10-24-2009 06:22 AM - edited 10-24-2009 06:53 AM
If I recall correctly, you can change the log level of MDS logs, to make more information appear.
I also suggest you set the Event Log on the device to Debug level so see if there's anything interesting there correlating with congestion.
You should also consider using the WiFi or USB bypass for the MDS connection to see if the issue happens only when the connection is routed via the cellular interface.
Another experiment you could try is to build two instances of your application (different module names) then start the test in both instances at (1) the same time, (2) different times, and then see if the congestion occurs at the same time in both instances. You could also run the test on two BlackBerrys registered with the same BES. Again, it will be interesting if congestion occurs at the same time.
My hypothesis at the moment is that you are trying to transmit at much higher bandwidth than is available from your BlackBerry to the MDS (though not necessarily limited by the cellular network). I suspect that if you use the WiFi or USB bypass, the issue will most likely go away, unless RIM's network infrastructure or the BES/MDS is the bottleneck.
If the issue is a manifestation of insufficient bandwidth between the BlackBerry and the MDS, then it might be a good idea to log the throughput since the begininng of the test (total bytes written and flushed divided by the time elapsed since the start of the test). After a couple of super-slow write/flush operations this number may stabilize. Also, this number might change in interesting ways if you run two tests of on the same BlackBerry or on two different BlackBerrys.
10-24-2009 08:51 AM
I have some comments about the last two posts.
"Now my question is, is there a way to change that?...Is there a setting or something to make flush() send right away? "
What we do is a write and then a read on the some connection. In your code, this would be a read on inputstream. My impression is that until you do something that makes it clear to the BB that your application logic requires the data to be sent, it doesn't necessarily send it, and it will buffer it anyway.
In our environment, messages are typically in the 100's of bytes and we will get a response back within a few seconds - basically the latency of the network.
"how to diagnose a socket connection in MDS log,"
I actually don't think there will be much to learn from the MDS. I think that it will work very closely with your Server code, so I would add tracing in your Server code in preference to looking at MDS logs. We have never looked at MDS for socket debugging (but have for http). I think this delay/buffering to be on the wireless side of the processing.
"Another experiment you could try is to build two instances of your application (different module names)"
I think this is a very interesting idea. I don't think copying the module will work as the package & class names will be the same and the BB will only run one of them. Instead just create a new Alternate Entry Project to your existing test. You can create and 'enterEventDispactcher', the same class - i.e. create a new UiApplication instance. You can even have a different icon if you want.
If you do run this test, then I think the blocking will occur at the same time, but with half as much data sent - in other words, the device is attempting to send the packets as fast as it can, and eventually you flood its buffers.
"though not necessarily limited by the cellular network" - actually I think that is where the limitation is.
"I suspect that if you use the WiFi or USB bypass, the issue will most likely go away, unless RIM's network infrastructure or the BES/MDS is the bottleneck"
I'm a little confused by this comment. My understanding and experience suggests that using WiFi and USB Bypass, there is no link to the RIM infrastructure. the traffic is routed directly to the BES. So I do not think that "RIM's network infrastructure " is involved in either of these two scenarios (assuming direct use of WiFi, specifying ";interface=wifi"). Also I doubt that BES/MDS is the bottleneck, unless this is on a heavily used Server. Again from our experience, the load put onto BES/MDS for socket connections is pretty small.
Having said that, my experience with WiFi suggests that the device can actually generate data faster than it can send over WiFi. So perhaps you would still see this, just not so often or for so long. An interesting test....
One final general comment, I would actually think there may be advantage to you designing a handshaking protocol for your socket transmissions, so that you don't get into the file download problem when, if it fails part way through, you have to send it all again. Over wireless, that will be slow and could be costly for your users. And with handshaking you won't flood the BB buffers (being nicer to the other processing). My belief (I have no evidence) is that the networks work on a 2K packet size, so sending 2K blocks may be optimal. klyubin, who knows this sort of stuff, can probably give us a better figure.
10-24-2009 05:24 PM
peter_strange, you've made an excellent point about using an alternate entry point to create two processes using the same module/code. I completely forgot about this option!
I wasn't talking about Direct WiFi (interface=wifi), I was talking about transparent routing of the BES/MDS tunnel via the WiFi, USB, and Bluetooth interfaces -- no code changes required at all.
I'm fairly certain (haven't done this for several years) that least cost routing (i.e., routing BES/MDS traffic via cellular, WiFi, USB, Bluetooth interfaces depending on what's available) is transparent to the BES in the sense that all the traffic still enters the BES via the SRP tunnel (was that the name?) between the BES and RIM network infrastructure. That's how traffic of one and the same connection (e.g., socket connection) from the application's and the BES/MDS's perspective can be transparently routed via different lower-level transports and network interfaces. The BES/MDS IPPP tunnel can be switched between all these transports, but the tunnel at all times goes via RIM's network infrastrucure, which enables such switching to be transparent to the device-side applications and the BES/MDS. I'm only 95% sure about this though. I don't have a BES/MDS at hand at the moment, but it will be fairly easy to run Wireshark/tcpdump to see where the Device Manager connects to tunnel BES/MDS traffic from a BlackBerry connected via USB or Bluetooth. Similarly, you one can see where the BlackBerry connect its IPPP VPN tunnel once it attaches to WiFI network.
Regarding the socket transport, throttling isn't mandatory because the TCP protocol takes care of congestion control. It is perfectly OK to create an application that only transmits data and does not receive anything. It's perfectly fine for the application to push data into the network stack as fast as the stack accepts it (this applies to a proper TCP stack -- the BlackBerry fine is full of various surprises). The stack will adapt to the network conditions and will automatically throttle the connection. Obviously, the above is only fine if you don't care about real-time delivery, latency, and backlog issues (e.g., FTP upload use case). However, if you are using the TCP stream for encapsulating higher-level semi-realtime packets, then you have to take extra care and invent your own congestion control that will have to work fine with TCP's own congestion control.
One caveat with the above argument is that the socket://... transport in the MDS case doesn't have to guarantee TCP semantics between the device and the MDS, because only the connection from the MDS to the destination is guaranteed to be TCP. So, in theory, RIM could have implemented this transport in a way where congestion occurs before the traffic enters the TCP leg of the journey, and some mysterious mechanisms (i.e., non-TCP) could be used to alleviate this congestion.
10-25-2009 06:49 AM
P.S. peter_strange, I thought about it a bit, and now I'm no longer sure that the serial bypass (USB, Bluetooth) goes via RIM's network infrastructure. I think you might be right -- the traffic flows between the PC and the BES/MDS directly. However, it would be nice if somebody could run a quick test and confirm this.
If the serial bypass doesn't touch RIM's network infrastructure, then it's even better for testing purposes. This provides a means for controlling for RIM's network infrastructure: i.e., tests which check whether RIM's network infrastucture is affecting the results significantly or not.
I'm still fairly certain that the WiFi bypass for BES/MDS and BIS (not to be confused with Direct WiFi connections) always goes via RIM's network infrastructure. Would be nice if somebody confirmed this too...
10-25-2009 08:35 PM
"would be nice if somebody could run a quick test and confirm this" - my thoughts exactly, and I had quick go at it, but failed. Need more time.
"I'm still fairly certain that the WiFi bypass for BES/MDS and BIS (not to be confused with Direct WiFi connections) always goes via RIM's network infrastructure." - Agree that this could go either way. This I'm going to struggle to test. Hope someone else can.
Perhaps some one from RIM will step in and save the test? Fingers crossed.
10-26-2009 11:20 AM
You are both actually correct.
Let's assume our application makes a socket connection through a BlackBerry Enterprise Server (deviceside=false).
When a BlackBerry smartphone is connected using serial bypass over a Bluetooth or USB connection, the connection does not touch the BlackBerry Infrastructure. The connection goes directly from the PC the BlackBerry smartphone is connected with, to the destination BlackBerry Enterprise Server.
Things can get more complicated when using Wi-Fi. If possible, the BlackBerry smartphone will attempt to use serial bypass over Wi-Fi. This requires the device be able to reach the BlackBerry Enterprise Server. This could work if the device is connected on a corporate Wi-Fi network and can talk to the BES directly. Enterprises can also configure a VPN connection that can VPN from a non-corporate Wi-Fi network, into the corporate network and then to the BES.
If neither of those options are possible, the BlackBerry smartphone attempts to contact the BlackBerry Infrastructure over Wi-Fi. If this is successful, the connection goes from the device, to the BlackBerry Infrastructure to the corporate BES. This also works for BIS connections (browsing, email, etc...).
10-26-2009 12:19 PM - edited 10-26-2009 01:24 PM
Thanks Mark, that is very interesting information.
Apologies for the digression, OP, have you had any luck forcing the transmission with a read?
10-26-2009 12:22 PM
Thank you very much, Mark! I suppose that's the famous "relay" vs "router" dichotomy (relay is inside the RIM network infrastructure, and router is part of the BES), correct?