K60 ethernet checksums

[davidsherman]

I've managed to get the K60 working under FreeRTOS TCP stack using the K60 tower board and TWR-SER module, but I'm running into problems with checksums.  I have the hardware configured with the hardware accelerator functions turned on, and FreeRTOS TCP is configured to have the hardware provide the checksums.  I've modified the FreeRTOS routines to ensure the checksum fields for the IP header and protocol headers are zeroed out.  So far, ping (ICMP) works, but TCP is getting erroneous IP header checksums according to Wireshark.

I see in the K60 documentation (K60P144M100SF2V2RM, page 1216) that it says:

"for ICMP, the checksum is calculated  over the complete ICMP datagram, in other words, without the IP header"

"for TCP and UDP the checksums contain the header and data sections and values from the IP header, which can be seen as a pseudo-header that is not actually present in the data stream"

What exactly does that mean?  I make sure the IP header checksum is zero, and that the TCP protocol header checksum is zero, but somehow the IP header checksum comes out incorrect when it is transmitted.  I suspect the reason ICMP is transmitted successfully but TCP shows a checksum error is related to this difference between the two, but I don't understand what this distinction means or what needs to be done to correct it.

[s_busman]

Hi All, K66 hardware acceleration does not work when the interface is set to 100MB. In that case a checksum is only added when total frame length does not exceed 270 Bytes. It's an easy to reproduce and very consistent error. @10MB there are no problems. Maybe the K60 has the same flaw. Not every ethernet device refuses frames with false checksums. So this flaw may go unnoticed up to a point something doesn't work and the whole thing looks unreliable.  

[mjbcswitzerland]

Hi

Can you demonstrate the issue? Note that all IP stacks will reject received data with incorrect IP or TCP checksums.

Also check some binaries here which show FTP, web server, MQTT, Telnet which have no known issues at either 10M or 100M using full checksum offloading.
http://www.utasker.com/kinetis/FRDM-K66F.html
http://www.utasker.com/kinetis/TWR-K64F120M.html
http://www.utasker.com/kinetis/TEENSY_3.5.html

Regards

Mark

[s_busman]

Hi Mark,

Thanks for your email. Enclosed a picture from wireshark where frames with

missing checksums appear black. You see i send frames with increasing

length here. Like i said, this problem may go unnoticed. We did notice

however after we recently had a new router from out internet provider. We

found the DHCP request frame length was a bit longer than the 'critical

size' and checksums were missing. This new router DID TEST THE CHECKSUMS

and DID NOT ACK FRAMES WITH MISSING CHECKSUMS! So there is a real problem

here.

*Make sure the PC running wireshark has RX hardware accelleration turned

off as this also may set a checksum field as shown in wireshark to zero.*

We build our own TCP stack for K60 and K66 and it is stable and super

efficient. I don't know about the utasker solution, but our stack is much

faster than any open source we could find. I also compared my stack with

the Keil stack, no further comments on that....

And yes, I did spend days checking every bit i could find in the ENET

config and DMA descriptors. But you can never be sure. So it would be

really nice if you could confirm the flaw. Can you run the same test with

the utasker stack and see what you get?

--

Simon Busman

DTPA/SMS/ENTEG

Faculty of Science and Engineering

Nijenborgh 4

9747 AG Groningen

The Netherlands

Room: 5117.0201

Tel.: + 31 (0)50 363 3932

E-mail: s.busman@rug.nl

On Mon, Jul 22, 2019 at 11:53 PM mjbcswitzerland <

[mjbcswitzerland]

Hi Simon

Super efficient sounds good but is not very quantitative - here are some UDP echo benchmarks of the uTasker stack on Coldifre on Kinetis (with and without offloading):

http://www.utasker.com/docs/uTasker/uTaskerBenchmarks.PDF

The uTasker stack is running on many hundreds of thousands or Kinetis devices (eg. of some high quantity uses are for oil/gas platform radio based IP networking and  power station load monitoring equipment - it also runs on the K60s used for stage control for the U2 360° world tour and last year's Ed Sheeran tour (TCP/IP/Modbus).

Since I know of no issues with check sum offloading, which are used throughout in these products I would be surprised to find that things just worked by luck.

If I can find a few minutes later I'll set up a UDP test like yours and publish the results.

Regards

Mark

[mjbcswitzerland]

Simon

Attached is  FRDM-K66F binary that will send a UDP broadcast, as your application is doing, each 1ms.
It increase the payload form 1 byte to 1023 bytes and then back to 1, etc.

It uses UDP checksum and IP checksum offloading, plus IP payload checksum offloading.

I don't detect any issues - the frame with 270 byte payload looks like

You can load the attached binary to a FRDM-K66F to check at your end.

Are you using both IP and payload offloading? Which of the check sums is failing? Be sure that you always set the checksum values in the frame to be sent to 0 - leaving a random value or non-zero will cause it to generate/insert a bad value.

Regards

Mark

Complete Kinetis solutions for professional needs, training and support: http://www.utasker.com/kinetis.html
Kinetis K66:
- http://www.utasker.com/kinetis/TWR-K65F180M.html
- http://www.utasker.com/kinetis/FRDM-K66F.html
- http://www.utasker.com/kinetis/TEENSY_3.6.html
uTasker: supporting >1'000 registered Kinetis users get products faster and cheaper to market
Request Free emergency remote desk-top consulting at http://www.utasker.com/services.html

Open Source version at https://github.com/uTasker/uTasker-Kinetis

[s_busman]

Hi Mark,

Thank you for your great help. Of course i initially falsely interpreted

your good answer. I am pretty convinced now there is something in my code

that makes the offloading fail. Maybe it was unfortunate i initially tested

with a router that did not look at the checksum, and did not see the

problem earlier. Anyway i have to find out what that is so i will do more

experiments.

I use both IP checksum and UDP/ICMP offloading. I do set the initial values

to 0x0000. If the IP checksum fails, UDP checksum also fails. They both

fail or both are both OK. I will test ICMP also by putting more data in the

ping. I run RMII on a K66, so the FRDM board seems a good reference. CPU

speed 180 MHz. Running at 10MB/s everything works fine. Only running

100MB/s i see the problem.

Maybe it has something to do with my 'efficient' handling of the frame

data. I dynamically assign frame buffer space to the TX descriptor just

before i set the R and BDU bits to avoid i have to copy the frame from ram

to ram. I tried a small wait but that did not help.

I assume the set of the R bit triggers the start of the calculation. (if it

was the BDU bit, i should see a change when i introduce wait between the

set of first R and the BDU, or the other way around.).

I like to assume the transmit starts when the offloading calculation is

ready. But that seems not to be the case.

Assuming the ENET needs more time to calculate the checksums, it may make

sense to think about what actually triggers the ENET to transmit the frame.

Do you have a clue?

I have boards with K60-MII and K60-RMII as well. If you appreciate, i will

share my findings with you.

I do not have an FRDM board unfortunately.

Your portfolio and credentials are impressive. I guess that is a good plan

B.

Thanks again. Kind Regards,

Simon Busman.

--

Simon Busman

DTPA/SMS/ENTEG

Faculty of Science and Engineering

Nijenborgh 4

9747 AG Groningen

The Netherlands

Room: 5117.0201

Tel.: + 31 (0)50 363 3932

E-mail: s.busman@rug.nl

On Wed, Jul 24, 2019 at 1:15 AM mjbcswitzerland <

[mjbcswitzerland]

Simon

I don't know exactly how the offloading works but suspect that it is performed serially, when the data is sent and so don't take any noticeable time.

Of course all data that has been prepared in the Ethernet buffer descriptors needs to remain stable (not overwritten/reused) until sent out (the OWNER bit in the buffer descriptor status should be checked before reusing.

Don't forget also that in a semi-duplex environment collisions/repetitions are possible and so the time that the Ethernet controller needs to send out (and release buffer descriptor ownership) depends on collisions/repetitions too.

Regards

Mark

[s_busman]

Mark,

Enabling the TFWR bit solved the issue. Appreciate your help.

Simon

[s_busman]

Dear Mark,

Thank you for this info. Currently i'm performing test and carefully manage

the contents of buffer descriptors and various timing of ready bits,

ENET->TDAR etcetera. So far with no results but there are still some things

left to try. It is really nice to know the problem is not in the chip. I

will let you know how this goes.

Kind Regards,

Simon Busman.

--

Simon Busman

DTPA/SMS/ENTEG

Faculty of Science and Engineering

Nijenborgh 4

9747 AG Groningen

The Netherlands

Room: 5117.0201

Tel.: + 31 (0)50 363 3932

E-mail: s.busman@rug.nl

On Thu, Jul 25, 2019 at 11:48 AM mjbcswitzerland <

[mjbcswitzerland]

David

The IP CS field simply needs to be set to 0x0000 in the general case.

Check whether you have tx checksum offloading enabled and also tx payload checksum offloading enabled, plus that you enable store and forward so that the checksum calculation is inserted only after the complete transmission has been checked.

Note that you may need to make each frame controllable since there are some protocols that don't work with the HW checksum (IGMP for example) or if you will be using the TCP/IP stack on multiple interfaces (or PPP, SLIP, RNDIS) where not all of the checksum offloading rules apply in each routing case.

I also use the CS offloading together with tail-taggiing switches where I need to set different values to 0x0000 on transmission to compensate for the fact that there is a tail tagging byte transmitted on the RMII that otherwise confuses the offloading HW (by adjusting the initial value, one can compensate it out for the particular frame).

Regards

Mark

[davidsherman]

Thank you for the tips, Mark.  Those protocols where hardware checksum might be problematic are good to know about.  Apparently FreeRTOS was doing some additional modification to the IP header.  I found the places where it would insert the calculated checksum and forced them to zero, now the TCP and ICMP packets seem to be behaving themselves.

[cgraulle]

Hi David,

I currently use FNET coupled to FreeRTOS on K60 as network layer and I envisage to use FreeRTOS-TCP instead. I saw that you manage to write the interface to the K60 FEC. Would you share your work ? That will facilitate my start on this new stack.

Regards

Christophe

[davidsherman]

Hi Cristophe,  I take it you have a driver for the K60 ethernet in FreeRTOS and you just want to use it with FreeRTOS-TCP, correct?  I'll try to clean things up and show you what I did.  The biggest headaches for me were making a working driver since the ethernet module needed the endian swaps for the buffer descriptors to get it to transmit and receive properly, and then I had to find the places in FreeRTOS-TCP where the checksums needed to be zeroed out.  The stack could be configured for hardware or software checksums, but it assumed the hardware didn't have the requirement to have the checksum zeroed out.  There were a few places that needed to be modified to ensure the checksums were zeroed out to use the hardware checksum feature.