11-11-2010 11:48 AM - edited 11-11-2010 11:50 AM
After several long days on the phone with RIM and no clue as to what is occurring I think I may have cracked the code and if my theory is correct, it's not a RIM issue.
I installed over half a dozen additional test boxes, using various iterations of 5.0.x and MAPI/CDOs. None of them would work. I tried reusing the original IP and I tried using the IP of servers known to work with no resolution to the issue at all. Finally, I tried installing on a server 2003 (32 Bit) machine and WHALLA!! communication was normal. This narrowed it down to the fact that there is an issue of communication between a 2008 64 bit box and this mail server which is 2003 SP2 32 bit.
I did some searching and came upon my old nemesis: Receive Side Scaling (RSS)
http://support.microsoft.com/kb/951037
We did battle with RSS a while back when we were having some backup problems and when I checked the status of RSS on the NIC in the problematic mail server it was still turned on whereas it is disabled on all the other mail servers.
I think I've got my suspect cornered, but I need to wait for an outage window tonight to go all Columbo on it....more to come...

I am on AT&T. Please edit your Personal Profile with your DEVICE TYPE, DEVICE OS and Carrier
11-11-2010 12:02 PM
if that is the cause, kudos to you.
I wonder how many others this snake has bitten
Click here to Backup the data on your BlackBerry Device! It's important, and FREE!
BESAdmin's, please make a signature with your BES environment info. SIM Free BlackBerry Unlocking FAQ
Follow me on Twitter @knottyrope
11-12-2010 12:19 PM
SUCCESS!!!!
Turned off RSS in Device Manager under the Properties for the Network Card>Advanced. The mail server dropped 3 pings and after bouncing the BES servers everything responds normally. Talk about your needle in a haystack.
This issue shows it's head a lot on Broadcom Onboard NICs on Dell servers but isn't isolated to either.
So, in conclusion: If you have a 64 Bit BES server talking to a W2K3 SP2 32 Bit Mail Server and traffic is so slow you can't get activations to progress, verify that the mail server has RSS (Receive Side Scaling) disabled.
Thanks for walking through the various spots to look Knotty, and extra set of eyes always helps keep the wheels moving.
I am on AT&T. Please edit your Personal Profile with your DEVICE TYPE, DEVICE OS and Carrier
11-12-2010 12:32 PM
glad you stuck with it, looked at other options, and posted the solution here for others to see.
Click here to Backup the data on your BlackBerry Device! It's important, and FREE!
BESAdmin's, please make a signature with your BES environment info. SIM Free BlackBerry Unlocking FAQ
Follow me on Twitter @knottyrope