Interesting Traces - odd length packet failure

This problem happened to me while I was working on another problem. I set up my test environment which consisted of one system with an Intel PRO/1000 T card (copper gigabit interface) connected to a Canary GFT-1055 media converter connected to a Agilent Technologies analyzer with fiber ports connected to an Allied Telesyn AT-MC1004 media converter connected to another system with an Intel PRO/10000 T. Got that? Two systems connected with copper and a fiber based analyzer in the middle. I was just trying to do some timing tests using FTP (I realize that FTP is not the best way to test performance but this was supposed to be a quick and dirty test). Here is the summary traces from the analyzer (I removed some columns which I don't think are relevant to make it fit neatly on the page):

      Length   Source           Destination        Protocol
 1      64   XX-XX-XX-01-28-2E  FF-FF-FF-FF-FF-FF  ARP  Request from 172.16.2.120 for 172.16.2.34 
 2      68   XX-XX-XX-42-3B-6E  XX-XX-XX-01-28-2E  ARP  Reply 172.16.2.34 is at XX-XX-XX-42-3B-6E   
 3      66   172.16.2.120       172.16.2.34        TCP  t4696  -> ftp    Flags=....S. Seq=3496445664 
 4      64   172.16.2.34        172.16.2.120       TCP  ftp    -> t4696  Flags=.A..S. Seq=2004086213 Ack=3496445665
 5      64   172.16.2.120       172.16.2.34        TCP  t4696  -> ftp    Flags=.A.... Ack=2004086214     
 6     145   172.16.2.34        172.16.2.120       FTP  220 hostname FTP ser..                           
 7      64   172.16.2.120       172.16.2.34        TCP  t4696  -> ftp    Flags=.A.... Ack=2004086301     
 8      67   172.16.2.120       172.16.2.34        FTP  USER tt<0D><0A>                                  
 9      67   172.16.2.120       172.16.2.34        FTP  USER tt<0D><0A>                                  
10      67   172.16.2.120       172.16.2.34        FTP  USER tt<0D><0A>                                  
11      67   172.16.2.120       172.16.2.34        FTP  USER tt<0D><0A>                                  
12      67   172.16.2.120       172.16.2.34        FTP  USER tt<0D><0A>
13      64   172.16.2.120       172.16.2.34        TCP  t4696  -> ftp    Flags=.A...F Seq=3496445674 Ack=2004086301
14      64   172.16.2.34        172.16.2.120       TCP  ftp    -> t4696  Flags=.A.... Ack=3496445665

You will notice that it starts off OK but I never get an ACK for my USER command. I could ping 172.16.2.34 and get responses even while I never got an ACK for USER:

      Length   Source           Destination        Protocol
18      78   172.16.2.120       172.16.2.34        ICMP     Echo request    
19      78   172.16.2.34        172.16.2.120       ICMP     Echo reply      
20      67   172.16.2.120       172.16.2.34        FTP      USER tt<0D><0A>
21      78   172.16.2.120       172.16.2.34        ICMP     Echo request    
22      78   172.16.2.34        172.16.2.120       ICMP     Echo reply      
23      78   172.16.2.120       172.16.2.34        ICMP     Echo request    
24      78   172.16.2.34        172.16.2.120       ICMP     Echo reply      
25      78   172.16.2.120       172.16.2.34        ICMP     Echo request    
26      78   172.16.2.34        172.16.2.120       ICMP     Echo reply      
27      78   172.16.2.120       172.16.2.34        ICMP     Echo request    
28      78   172.16.2.34        172.16.2.120       ICMP     Echo reply      
29      67   172.16.2.120       172.16.2.34        FTP      USER tt<0D><0A>
30      78   172.16.2.120       172.16.2.34        ICMP     Echo request     
31      78   172.16.2.34        172.16.2.120       ICMP     Echo reply       
32      78   172.16.2.120       172.16.2.34        ICMP     Echo request     
33      78   172.16.2.34        172.16.2.120       ICMP     Echo reply       
34      78   172.16.2.120       172.16.2.34        ICMP     Echo request     
35      78   172.16.2.34        172.16.2.120       ICMP     Echo reply       
36      78   172.16.2.120       172.16.2.34        ICMP     Echo request     
37      78   172.16.2.34        172.16.2.120       ICMP     Echo reply       
38      78   172.16.2.120       172.16.2.34        ICMP     Echo request     
39      78   172.16.2.34        172.16.2.120       ICMP     Echo reply       
40      78   172.16.2.120       172.16.2.34        ICMP     Echo request     
41      78   172.16.2.34        172.16.2.120       ICMP     Echo reply       
42      67   172.16.2.120       172.16.2.34        FTP      USER tt<0D><0A>
43      78   172.16.2.120       172.16.2.34        ICMP     Echo request    
44      78   172.16.2.34        172.16.2.120       ICMP     Echo reply      
45      78   172.16.2.120       172.16.2.34        ICMP     Echo request

The really observant will notice something odd about the frames containing the USER command - and I mean that literally. The length of those frames is odd while all the other frames have an even number of bytes. A quick check using ping shows that odd length frames do not get a response while even length frames do.

      Length   Source           Destination        Protocol
 1      78   172.16.2.120       172.16.2.34        ICMP     Echo request 
 2      78   172.16.2.34        172.16.2.120       ICMP     Echo reply   
 3      79   172.16.2.120       172.16.2.34        ICMP     Echo request 
 4      80   172.16.2.120       172.16.2.34        ICMP     Echo request 
 5      80   172.16.2.34        172.16.2.120       ICMP     Echo reply   
 6      81   172.16.2.120       172.16.2.34        ICMP     Echo request 
 7      82   172.16.2.120       172.16.2.34        ICMP     Echo request 
 8      82   172.16.2.34        172.16.2.120       ICMP     Echo reply   
 9      83   172.16.2.120       172.16.2.34        ICMP     Echo request 
10      84   172.16.2.120       172.16.2.34        ICMP     Echo request 
11      84   172.16.2.34        172.16.2.120       ICMP     Echo reply   
12      85   172.16.2.120       172.16.2.34        ICMP     Echo request 
13      86   172.16.2.120       172.16.2.34        ICMP     Echo request 
14      86   172.16.2.34        172.16.2.120       ICMP     Echo reply   
15      87   172.16.2.120       172.16.2.34        ICMP     Echo request

What could cause this? Intel reports a compatibility problem between the PRO/1000 T and some Cisco devices that can result in CRC errors (I noticed the CRC errors whenever I sent the odd length pings). See http://www.intel.com/support/network/adapter/pro100/sb/cs-007655.htm and http://downloadfinder.intel.com/scripts-df/Detail_Desc.asp?&Inst=Yes&DwnldID=6880.The pages don't mention anything about odd length packets but when I downloaded the Solaris OS driver and looked at the files in it I found a note about CRC errors and odd length packets. While the pages talk only about Cisco devices it is obvious that the problem can occur when using other devices as well. In my case the Allied Telesyn media converter showed the problem but the Canary does not. When I reversed the direction of the ping (34 to 120) I get what appears to be a perfectly normal trace but the ping command still times out on the odd length pings because host 34 gets a CRC error on the responses.

      Length   Source           Destination        Protocol
 1      78   172.16.2.34        172.16.2.120       ICMP     Echo request 
 2      78   172.16.2.120       172.16.2.34        ICMP     Echo reply   
 3      79   172.16.2.34        172.16.2.120       ICMP     Echo request 
 4      79   172.16.2.120       172.16.2.34        ICMP     Echo reply   
 5      80   172.16.2.34        172.16.2.120       ICMP     Echo request 
 6      80   172.16.2.120       172.16.2.34        ICMP     Echo reply   
 7      81   172.16.2.34        172.16.2.120       ICMP     Echo request 
 8      81   172.16.2.120       172.16.2.34        ICMP     Echo reply   
 9      82   172.16.2.34        172.16.2.120       ICMP     Echo request 
10      82   172.16.2.120       172.16.2.34        ICMP     Echo reply   
11      83   172.16.2.34        172.16.2.120       ICMP     Echo request 
12      83   172.16.2.120       172.16.2.34        ICMP     Echo reply

The good news is that there are drivers at Intel that fix the problem for Windows and Solaris. It also appears that the Linux driver (http://www.kernel.org) is fixed.

This page was last modified on 05-03-30

Send comments and suggestions
to ndav1@cox.net