lwip checksum incorrect when outputting a pbuf chain.

[lpcware]

Content originally posted in LPCWare by Johnny D on Wed Apr 16 05:23:49 MST 2014
Hi, I have lwip running on the M0 core of an LPC4357.

Every time the function lpc_low_level_output() is called with a pointer to a pbuf chain, the checksum it sends out is not correct (verified with wireshark, and also the trace from the receiving device. The trace that comes out looks like this:

tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 46
tcp_write: queueing 8760:8773
tcp_write: 47 (after enqueued)
tcp_output_segment: 8760:8773
lpc_low_level_output: pbuf packet 2000256c sent, chain 0, size 67, index 41, free 38   <--- checksum appears fine
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 47
tcp_write: queueing 8773:8786
tcp_write: 48 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 48
tcp_write: 49 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 49
tcp_write: 50 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 50
tcp_write: 51 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 51
tcp_write: 52 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 52
tcp_write: 53 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 53
tcp_write: 54 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 54
tcp_write: 55 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 55
tcp_write: 56 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 56
tcp_write: 57 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 57
tcp_write: 58 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 58
tcp_write: 59 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 59
tcp_write: 60 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 60
tcp_write: 61 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 61
tcp_write: 62 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 62
tcp_write: 63 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 63
tcp_write: 64 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 64
tcp_write: 65 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 65
tcp_write: 66 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 66
tcp_write: 67 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 67
tcp_write: 68 (after enqueued)
lpc_rxqueue_pbuf: Queueing packet 2000933c at index 13, free 0
lpc_low_level_input: Packet received, 64 bytes, status 0x00400321
tcp_receive: queuelen 68 ... 
67 (after freeing unacked)
tcp_receive: queuelen 67 ... 
66 (after freeing unacked)
lpc_tx_reclaim: Reclaiming sent packet 200020ec, index 35
lpc_tx_reclaim: Reclaiming sent packet 2000214c, index 36
lpc_tx_reclaim: Reclaiming sent packet 200023ec, index 37
lpc_tx_reclaim: Reclaiming sent packet 2000244c, index 38
lpc_tx_reclaim: Reclaiming sent packet 200024ac, index 39
lpc_tx_reclaim: Reclaiming sent packet 2000250c, index 40
lpc_tx_reclaim: Reclaiming sent packet 2000256c, index 41
lpc_rxqueue_pbuf: Queueing packet 20002654 at index 14, free 0
lpc_low_level_input: Packet received, 64 bytes, status 0x00400321
tcp_receive: queuelen 66 ... 
65 (after freeing unacked)
tcp_receive: queuelen 65 ... 
64 (after freeing unacked)
lpc_rxqueue_pbuf: Queueing packet 200028d8 at index 15, free 0
lpc_low_level_input: Packet received, 64 bytes, status 0x00400321
tcp_receive: queuelen 64 ... 
63 (after freeing unacked)
tcp_receive: queuelen 63 ... 
62 (after freeing unacked)
lpc_rxqueue_pbuf: Queueing packet 20002b5c at index 16, free 0
lpc_low_level_input: Packet received, 64 bytes, status 0x00400321
tcp_receive: queuelen 62 ... 
61 (after freeing unacked)
tcp_receive: queuelen 61 ... 
60 (after freeing unacked)
lpc_rxqueue_pbuf: Queueing packet 20007f20 at index 17, free 0
lpc_low_level_input: Packet received, 64 bytes, status 0x00400321
tcp_receive: queuelen 60 ... 
59 (after freeing unacked)
tcp_receive: queuelen 59 ... 
58 (after freeing unacked)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 58
tcp_write: 59 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 59
tcp_write: 60 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 60
tcp_write: 61 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 61
tcp_write: 62 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 62
tcp_write: 63 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 63
tcp_write: 64 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 64
tcp_write: 65 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 65
tcp_write: 66 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 66
tcp_write: 67 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 67
tcp_write: 68 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 68
tcp_write: 69 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 69
tcp_write: 70 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 70
tcp_write: 71 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 71
tcp_write: 72 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 72
tcp_write: 73 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 73
tcp_write: 74 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 74
tcp_write: 75 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 75
tcp_write: 76 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 76
tcp_write: 77 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 77
tcp_write: 78 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 78
tcp_write: 79 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 79
tcp_write: 80 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 80
tcp_write: queueing 9333:9345
tcp_write: 82 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 82
tcp_write: 83 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 83
tcp_write: 84 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 84
tcp_write: 85 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 85
tcp_write: 86 (after enqueued)
tcp_write(pcb=2804b37c, data=2000f118, len=13, apiflags=1)
tcp_write: queuelen: 86
tcp_write: 87 (after enqueued)
lpc_rxqueue_pbuf: Queueing packet 200095c0 at index 18, free 0
lpc_low_level_input: Packet received, 64 bytes, status 0x00400321
tcp_receive: queuelen 87 ... 
86 (after freeing unacked)
tcp_receive: queuelen 86 ... 
85 (after freeing unacked)
lpc_rxqueue_pbuf: Queueing packet 200030d8 at index 19, free 0
lpc_low_level_input: Packet received, 64 bytes, status 0x00400321
tcp_receive: queuelen 85 ... 
84 (after freeing unacked)
tcp_receive: queuelen 84 ... 
83 (after freeing unacked)
lpc_rxqueue_pbuf: Queueing packet 2000335c at index 20, free 0
lpc_low_level_input: Packet received, 64 bytes, status 0x00400321
tcp_receive: queuelen 83 ... 
82 (after freeing unacked)
tcp_output_segment: 8773:9333
lpc_low_level_output: pbuf packet 200025cc sent, chain 43, size 67, index 42, free 44
lpc_low_level_output: pbuf packet 2000262c sent, chain 42, size 13, index 43, free 43
lpc_low_level_output: pbuf packet 20004e0c sent, chain 41, size 13, index 44, free 42
lpc_low_level_output: pbuf packet 20007ef8 sent, chain 40, size 13, index 0, free 41
lpc_low_level_output: pbuf packet 200087e8 sent, chain 39, size 13, index 1, free 40
lpc_low_level_output: pbuf packet 200090bc sent, chain 38, size 13, index 2, free 39
lpc_low_level_output: pbuf packet 200090e4 sent, chain 37, size 13, index 3, free 38
lpc_low_level_output: pbuf packet 2000910c sent, chain 36, size 13, index 4, free 37
lpc_low_level_output: pbuf packet 20009134 sent, chain 35, size 13, index 5, free 36
lpc_low_level_output: pbuf packet 2000915c sent, chain 34, size 13, index 6, free 35
lpc_low_level_output: pbuf packet 20009184 sent, chain 33, size 13, index 7, free 34
lpc_low_level_output: pbuf packet 200091ac sent, chain 32, size 13, index 8, free 33
lpc_low_level_output: pbuf packet 200091d4 sent, chain 31, size 13, index 9, free 32
lpc_low_level_output: pbuf packet 200091fc sent, chain 30, size 13, index 10, free 31
lpc_low_level_output: pbuf packet 20009224 sent, chain 29, size 13, index 11, free 30
lpc_low_level_output: pbuf packet 2000924c sent, chain 28, size 13, index 12, free 29
lpc_low_level_output: pbuf packet 20009274 sent, chain 27, size 13, index 13, free 28
lpc_low_level_output: pbuf packet 2000929c sent, chain 26, size 13, index 14, free 27
lpc_low_level_output: pbuf packet 200092c4 sent, chain 25, size 13, index 15, free 26
lpc_low_level_output: pbuf packet 200092ec sent, chain 24, size 13, index 16, free 25
lpc_low_level_output: pbuf packet 20009314 sent, chain 23, size 13, index 17, free 24
lpc_low_level_output: pbuf packet 200021ac sent, chain 22, size 13, index 18, free 23
lpc_low_level_output: pbuf packet 200021d4 sent, chain 21, size 13, index 19, free 22
lpc_low_level_output: pbuf packet 200021fc sent, chain 20, size 13, index 20, free 21
lpc_low_level_output: pbuf packet 20002224 sent, chain 19, size 13, index 21, free 20
lpc_low_level_output: pbuf packet 2000224c sent, chain 18, size 13, index 22, free 19
lpc_low_level_output: pbuf packet 20002274 sent, chain 17, size 13, index 23, free 18
lpc_low_level_output: pbuf packet 2000229c sent, chain 16, size 13, index 24, free 17
lpc_low_level_output: pbuf packet 200022c4 sent, chain 15, size 13, index 25, free 16
lpc_low_level_output: pbuf packet 200022ec sent, chain 14, size 13, index 26, free 15
lpc_low_level_output: pbuf packet 20002314 sent, chain 13, size 13, index 27, free 14
lpc_low_level_output: pbuf packet 2000233c sent, chain 12, size 13, index 28, free 13
lpc_low_level_output: pbuf packet 20002364 sent, chain 11, size 13, index 29, free 12
lpc_low_level_output: pbuf packet 2000238c sent, chain 10, size 13, index 30, free 11
lpc_low_level_output: pbuf packet 200023b4 sent, chain 9, size 13, index 31, free 10
lpc_low_level_output: pbuf packet 20002de0 sent, chain 8, size 13, index 32, free 9
lpc_low_level_output: pbuf packet 20002e08 sent, chain 7, size 13, index 33, free 8
lpc_low_level_output: pbuf packet 20002e30 sent, chain 6, size 13, index 34, free 7
lpc_low_level_output: pbuf packet 20002e58 sent, chain 5, size 13, index 35, free 6
lpc_low_level_output: pbuf packet 20002e80 sent, chain 4, size 13, index 36, free 5
lpc_low_level_output: pbuf packet 20002ea8 sent, chain 3, size 13, index 37, free 4
lpc_low_level_output: pbuf packet 20002ed0 sent, chain 2, size 13, index 38, free 3
lpc_low_level_output: pbuf packet 20002ef8 sent, chain 1, size 13, index 39, free 2
lpc_low_level_output: pbuf packet 20002f20 sent, chain 0, size 1, index 40, free 1          <--- checksum incorrect
lpc_tx_reclaim: Reclaiming sent packet 200025cc, index 42
lpc_tx_reclaim: Reclaiming sent packet 2000262c, index 43
lpc_tx_reclaim: Reclaiming sent packet 20004e0c, index 44
lpc_tx_reclaim: Reclaiming sent packet 20007ef8, index 0
lpc_tx_reclaim: Reclaiming sent packet 200087e8, index 1
lpc_tx_reclaim: Reclaiming sent packet 200090bc, index 2
lpc_tx_reclaim: Reclaiming sent packet 200090e4, index 3
lpc_tx_reclaim: Reclaiming sent packet 2000910c, index 4
lpc_tx_reclaim: Reclaiming sent packet 20009134, index 5
lpc_tx_reclaim: Reclaiming sent packet 2000915c, index 6
lpc_tx_reclaim: Reclaiming sent packet 20009184, index 7
lpc_tx_reclaim: Reclaiming sent packet 200091ac, index 8
lpc_tx_reclaim: Reclaiming sent packet 200091d4, index 9
lpc_tx_reclaim: Reclaiming sent packet 200091fc, index 10
lpc_tx_reclaim: Reclaiming sent packet 20009224, index 11
lpc_tx_reclaim: Reclaiming sent packet 2000924c, index 12
lpc_tx_reclaim: Reclaiming sent packet 20009274, index 13
lpc_tx_reclaim: Reclaiming sent packet 2000929c, index 14
lpc_tx_reclaim: Reclaiming sent packet 200092c4, index 15
lpc_tx_reclaim: Reclaiming sent packet 200092ec, index 16
lpc_tx_reclaim: Reclaiming sent packet 20009314, index 17
lpc_tx_reclaim: Reclaiming sent packet 200021ac, index 18
lpc_tx_reclaim: Reclaiming sent packet 200021d4, index 19
lpc_tx_reclaim: Reclaiming sent packet 200021fc, index 20
lpc_tx_reclaim: Reclaiming sent packet 20002224, index 21
lpc_tx_reclaim: Reclaiming sent packet 2000224c, index 22
lpc_tx_reclaim: Reclaiming sent packet 20002274, index 23
lpc_tx_reclaim: Reclaiming sent packet 2000229c, index 24
lpc_tx_reclaim: Reclaiming sent packet 200022c4, index 25
lpc_tx_reclaim: Reclaiming sent packet 200022ec, index 26
lpc_tx_reclaim: Reclaiming sent packet 20002314, index 27
lpc_tx_reclaim: Reclaiming sent packet 2000233c, index 28
lpc_tx_reclaim: Reclaiming sent packet 20002364, index 29
lpc_tx_reclaim: Reclaiming sent packet 2000238c, index 30
lpc_tx_reclaim: Reclaiming sent packet 200023b4, index 31
lpc_tx_reclaim: Reclaiming sent packet 20002de0, index 32
lpc_tx_reclaim: Reclaiming sent packet 20002e08, index 33
lpc_tx_reclaim: Reclaiming sent packet 20002e30, index 34
lpc_tx_reclaim: Reclaiming sent packet 20002e58, index 35
lpc_tx_reclaim: Reclaiming sent packet 20002e80, index 36
lpc_tx_reclaim: Reclaiming sent packet 20002ea8, index 37
lpc_tx_reclaim: Reclaiming sent packet 20002ed0, index 38
lpc_tx_reclaim: Reclaiming sent packet 20002ef8, index 39
lpc_tx_reclaim: Reclaiming sent packet 20002f20, index 40
lpc_rxqueue_pbuf: Queueing packet 200035e0 at index 21, free 0
lpc_low_level_input: Packet received, 64 bytes, status 0x00400321
tcp_receive: queuelen 82 ... 
81 (after freeing unacked)
tcp_receive: queuelen 81 ... 
80 (after freeing unacked)

Has anyone else run into this problem?

I found this thread https://groups.google.com/forum/#!topic/osdeve_mirror_tcpip_lwip/bL6Z5J3uRWA on the net but it looks like (from looking at the source code) the LPC driver should cope with this OK. I wonder what else it could be, or even how to go about testing this any further?

[lpcware]

Content originally posted in LPCWare by Johnny D on Wed Apr 16 08:40:12 MST 2014
I also just found this:
http://comments.gmane.org/gmane.network.lwip.devel/8476

which I think is the same issue, so it may genuinely be an lwip bug. I notice that the current version of LPCOpen doesn't contain this fix.

[lpcware]

Content originally posted in LPCWare by Johnny D on Wed Apr 16 07:56:50 MST 2014
A further discovery:

I enabled TCP_CHECKSUM_ON_COPY_SANITY_CHECK and everything suddenly started working. Looking more closely at the code, with this enabled lwip will re-calculate the checksum for a pbuf chain using the old method and compare it to the checksum generated "on the fly" (new method when using LWIP_CHECKSUM_ON_COPY). If the result is different, it will discard the checksum generated by the new method and use the one generated by the old method.

So, for now I have disabled LWIP_CHECKSUM_ON_COPY and everything seems fine and dandy.

Does anyone have any ideas why this new method of generating checksums for pbuf chains doesn't work? I have tried using all three standard checksum algorithms plus my own one written in assembler and it makes no difference, it fails in every case.