FNET - TCP/IP Stack

[butok]

FNET TCP/IP stack: Version 0.5.0 released. 
 

FNET Home page: http://fnet.sourceforge.net/

    Please, submit your feature requests and feedbacks here:

    http://sourceforge.net/tracker/?group_id=253892&atid=1126921 

    Please, submit your bug reports here:

    http://sourceforge.net/tracker/?group_id=253892&atid=1126920

FNET brief information:
The FNET is a free, open source software project (under GNU GPLv3) for building embedded communication software using the Freescale processors.

The stack provides following protocols and services:
  * Supported platforms:
     - Reindeer - MCF5282 (M5282EVB).
     - Kirin2 - MCF52235 (M2235EVB).
     - Kirin3 - MCF52259 (M2259EVB).
     - Lasko - MCF51CN128 (TWR-MCF51CN).
     - Other Freescale platforms to be supported soon.
  * Available as:
     - Stand-alone version. No underlying RTOS is required, although it can be used with it.
  * Supported Compilers:
     - Freescale CodeWarrior for ColdFire Version 7.1.
     - Freescale CodeWarrior for Microcontrollers Version 6.2.
  * BSD-style Socket layer API
  * Protocols:
     - User Datagram Protocol (UDP).
     - Transmission Control Protocol (TCP).
     - Internet Protocol (IPv4).
     - Internet Control Message Protocol (ICMP).
     - Address Resolution Protocol (ARP).
  * Network Interfaces:
     - Ethernet interface.
     - Loopback interface.
  * Services:
     - HTTP/0.9 server. Both SSI and CGI are supported.
     - DHCP client.
     - TFTP client.
     - Static File System.
     - CFM Flash driver.
  * Tools:
     - GUI File System Generation Tool.
  * Applications:
     - TFTP Bootloader.
     - Shell Application.
  * Doxygen User Documentation.

 Changes in 0.5.0 (since last public version):
 - Added TFTP client.
 - Added Coldfire Flash Module (CFM) driver.
 - Added FNET TFTP Bootloader.
 - Fixed DHCP client.
 - Updated Shell Demo application. Application parameters are saved in the flash.
 - Improved Shell library. Added blocking-command and multi-word parameter features.
 - A lot of other changes and fixes.

Best regards,

Andrey Butok

[butok]

Who is “Hercules” :smileywink:

You can try to increase the value of the FNET_CFG_HEAP_SIZE parameter.

[albertolubeiro]

Hi andrey,

Hercules is a program than can work as a TCP/IP client server terminal.

I have already try to increase the value of that constant, but what happens is that i can send more times "hello world"

but sooner or later there is a moment that "malloc_max" is minor that "FNET_CFG_ETH_MTU*1.5". it is like the data are accumulating in the "heap" until there is no space.

¿how can i send data and next delete them from the heap?

Thank you very much Andrey.

Best regards

[butok]

Hi Alberto

>>how can i send data and next delete them from the heap?

You are not able to do it. It is deleted automatically (when sent to network, received ACK for TCP or closed socket).

I guess you have something wrong.

The only thing that come to my mind, after your words "After doing this manytimes" => Try to play with SO_LINGER option. You will  find examples of its usage in source code of every FNET TCP service (HTTP,TELNET etc.).

=> So unsent data will be deleted  immediately if set it to {1,0}. More description is here:

Embedded TCP/IP stack: closesocket

Embedded TCP/IP stack: linger Struct Reference

Hope it will help,

Andrey Butok

[TomE]

> You are not able to do it. It is deleted automatically (when sent to network,

> received ACK for TCP or closed socket).

> I guess you have something wrong.

Very likely. Lots of programs using TCP don't close properly but can leave the connection "hanging". There are lots of TCP implementations that make it too easy for the clients to do this too.

Another problem is that TCP stacks are required to have the sockets "hang around" for a while after a close as part of the protocol. if that "held open socket" keeps ALL of the socket data (including data buffers) then small systems run out of memory as you're seeing. I've written a TCP system where I separated the socket and buffer storage, so the sockets didn't take much memory in this state.

How big is an FNET socket? Does it contain 4k or 32k of ring-buffers or something?

Read up about the "TIME-WAIT" state here:

http://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation

Then follow the link in the above to this:

File:Tcp state diagram fixed.svg - Wikipedia, the free encyclopedia

Check the different paths marked "Active Close" and "Passive Close". The difference is that the end of the connection that STARTS the close goes to "Active Close" and gets hit with the "Time Wait Timeout" The other end goes through "Last ACK" and avoids the timeout. So you should arrange things so the CLIENT is the one that initiates the Close, and then the Server won't have this problem to deal with.

"Lingering" is the problem where a socket is closed BEFORE the data has been ACKed. Bad implementations can close without removing all data and this causes the stack to be stuck in other states. The LINGER option adds a timeout to force it out of these. You should play with these (using Zero values and also amsll numbers like 1 or 2) and see if it fixes your problem.

Another thing well worth doing is to add some debugging so the stack PRINTS what state it is in every time it changes state. That should tell you which states it is getting stuck in and will let you solve the problem properly.

Under Linux and Windoes the "Netstat" command shows what STATES the sockets are in, like this:

Proto  Local Address	Foreign Address	State
TCP	tevans-xp-vm:1055	localhost:1110	CLOSE_WAIT
TCP	tevans-xp-vm:1057	localhost:1110	CLOSE_WAIT
TCP	tevans-xp-vm:1075	192.168.0.7:netbios-ssn  ESTABLISHED
TCP	tevans-xp-vm:1079	192.168.0.162:netbios-ssn  ESTABLISHED

That shows two connections in "CLOSE_WAIT". Does FNET have an equivalent way of getting to the TCP socket state data?

Tom

[butok]

>> Does it contain 4k or 32k of ring-buffers or something?

FNET sockets contain only real data and do not create blank buffers => saves memory.

>> Does FNET have an equivalent way of getting to the TCP socket state data?

It does not have the "netstat" command. But it is  possible to implement it.

I more interested to hear, did the “linger” option help to Alberto’s application?

[TomE]

> FNET sockets contain only real data and do not create blank buffers => saves memory.

That's good. Alberto must have a lot of hung sockets or not much free memory to start with. Alberto, how many time is "manytimes"? How much memory do you have, and how much FREE memory do you have?

> I more interested to hear, did the “linger” option help to Alberto’s application?

I'd like to inspect the source to see whether the SO_LINGER options affect any code in the TIME_WAIT state.

Is there a way to download the code without running a Windows ".exe" program?

Aha! I found "git clone git://git.code.sf.net/p/fnet/gitcode fnet-gitcode" on the "Code" tab, and:

FNET - Embedded TCP/IP Stack / Code / [5e8fc9] /fnet_stack/stack/fnet_tcp.c

I wouldn't recommend SO_LINGER with a parameter of zero. From my quick reading it looks like it slams the socket closed and sends a RESET segment instead of sending a FIN. That might cause bad problems for the other end of the connection.

And in fnet_tcp_close(), SO_LINGER is ignored in FNET_TCP_CS_CODE_WAIT, so it won't get you out of there if that if that is the problem.

You can always change the value of FNET_TCP_TIME_WAIT from its current value in fnet_tcp.h to something smaller (but tis current value of 24 seconds isn't too bad).

#define FNET_TCP_TIME_WAIT (240/5) /* 240 x FNET_TCP_SLOWTIMO = 2 minutes/5 */

Tom

[albertolubeiro]

Hi Andrey and Tom,

First of all, thanks you so much for trying to help me, but I have to take a look to another issue during a couple of days.

When I have tried SO_LINGER, I'll tell you how it went.

Tom, I'll  also tell you how much memory do I have, and how much free memory do I have.

Thanks again!!!!!

Best regards

[albertolubeiro]

Hi Andrey and Tom,

Andrey, I have set SO_LINGER option with a value of 1 and seems it works fine. But as Tom says SO_LINGER is ignored unless it is a parameter of zero, in which case, sends a reset segment. Why it works fine now?

I have also tried to modify FNET_TCP_TIME_WAIT before seting the SO_LINGER option, but nothing has changed.

Tom, I have reduced the size of "FNET_CFG_HEAP_SIZE" from 30k to 6k cause I can not put it so big.

Now with the SO_LINGER option set, in the function "fnet_tcp_snd" I have:

"freespace" = 2048. This is always the same and not depends of the HEAP size.

"malloc_max" = has not always the same value but it is 5160 or 4872.

The data that I send is "char buffer[12]={"hola mundo 1"};"

Without the SO_LINGER option, "malloc_max" is decrementing until  "malloc_max < (long)"(FNET_CFG_ETH_MTU*1.5.

In that moment the client conects but i send nothing.

Thanks and best regards.

PD: Sorry for my english :smileywink:

[butok]

Hi Alberto,

It is good that it works for you now.

The FNET HTTP and Telnet servers have the linger option set to {1,4}, that corresponds to the 4 seconds.

By default, when the option is not set, it corresponds to the 24 seconds (big TCP/IP stacks has it set to 2 minutes).

When you have the extremely small heap-size, it is needed to decrease MTU value as much as possible.

BTW: I have not tested FNET with heap-size less than 10 KBs. It is nice to hear that it works with 2KBs.

Thanks,

Andrey

[albertolubeiro]

Hi Andrey,

>> By default, when the option is not set, it corresponds to the 24 seconds

Sorry but I still don´t understand. I don´t see where the option SO_LINGER is used instead the 24 seconds. There is one place, but linger time must be zero. "fnet_tcp_close"

¿Where take effect the linger time that I have put to 1?

>>I have not tested FNET with heap-size less than 10 KBs. It is nice to hear that it works with 2KBs.

"FNET_CFG_HEAP_SIZE" isn't 2k. It is 6k

The 2k buffer is "FNET_CFG_SOCKET_TCP_TX_BUF_SIZE", but i don't modified that.

Thanks!!!

[TomE]

> t see where the option SO_LINGER is used instead the 24 seconds.

24 seconds is the default FNET_TCP_TIME_WAIT. As Andrey says, the default for this in "normal computers" (PC, Linux) is 2 minutes, and that can chew up a lot of memory. 24 seconds is fine. It has nothing t do with the "linger time".

I googled for "tcp linger explanation" and came up with some interesting links:

Graceful Shutdown, Linger Options, and Socket Closure (Windows)

The above says how Windows programmers are meant to program with TCP sockets. Open it and scroll down to the "Client Side, Server Side" diagram. The bit that should surprise you is where the client has CLOSED from its end, but is still required to READ data from the other end AFTER the "close". This is normal TCP operation. You should be able to set up a connection which is "closed" in one direction, but still should be able to send data in the other end "for ever". This is useless, but a feature of the way TCP works. The problem is that most people don't know this and don't code that way so they don't "read after close" when they should and that leaves the connection "stuck".

The above link also has in the comments details that Windows TCP sockets simply don't seem to work properly in Windows 7 and that the LINGER options no longer work. So if you're using FNET and connecting to and from Windows, these problems might show up at the FNET end.

sockets - TCP option SO_LINGER (zero) - when it&amp;#39;s required - Stack Overflow

The above is a VERY good summary of TCP connection closing, problems the "bad solution of using a LINGER time of zero", and how to write your protocols properly to not require this.

From your original post:

> What i do is; when the "hercules" connect i send "hello world"

> and next I close the socket.

From the previous link, it is far better to have the client close the socket. Closing from the server end necessarily causes entry into TIME_WAIT. This should only take 24 seconds though, and then you should have the memory back. Have you waited for this time?

4.6 - What exactly does SO_LINGER do? (Page 1) / UNIX Socket FAQ / UNIX Socket FAQ

The above details how SO_LINGER works and what the three different settings (off, on-zero, on-non-zero) mean, or are MEANT to mean.

Andrey, I'm pretty sure FNET implements the "on and zero" the way it details (sends an immediate abort/RST), but does the FNET socket block in the "on and non-zero" case as detailed above, or does it do something else?

> But as Tom says SO_LINGER is ignored unless it is a parameter of zero,

Further down in that function SO_LINGER with a non-zero parameter sets the connection timer to that value. The Zero state does work in all non-TIME_WAIT states, and you shouldn't be in that state when you call fnet_tcp_close() anyway.

Tom

[albertolubeiro]

Hi Tom,

Yeah, I have also googled but I have no idea of ethernet and there are so many concepts..... bufff :smileyconfused:

I guess that reading over here and over there i will learn little by little.

Last two questions to finally understand the solution of my problem:

>>This should only take 24 seconds though, and then you should have the memory back. Have you waited for this time?

That was my problem, that I didn't wait the 24 seconds before trying to connect again, althought the 24 seconds were not those of  FNET_TCP_TIME_WAIT but of FNET_TCP_ABORT_INTERVAL. I have tested it changing FNET_TCP_ABORT_INTERVAL by another value in "fnet_tcp_close".

Why so much time to close the conection?

>>Further down in that function SO_LINGER with a non-zero parameter sets the connection timer to that value.

If SO_LINGER is set, the system blocks the process during CLOSE until it can transmit the data or until the end of the interval indicated by the L_LINGER member, whichever comes first.

At this point of the "fnet_tcp_close" the paremeter l_linger has the same purpose of FNET_TCP_ABORT_INTERVAL.

Does this mean that the system can`t transmit the data in less than 24 seconds? because if it were possible, it would not be necessary to wait for 24 seconds, would be?

Why putting "cb->tcpcb_timers.connection = 2", works fine?

Thanks you so much for your support, really.

Best regards.

[TomE]

> Why so much time to close the conection?

It isn't "why 24 seconds" but "why 2 minutes" as that's the original specification. And that I should know and should be able to find really fast. The definitive source document for "all things TCP" is RFC793:

http://tools.ietf.org/html/rfc793

The stack stays in TIME_WAIT for "2 * MSL" where that means the "Maximum Segment Lifetime", which is:

Page 81

MSL

  Maximum Segment Lifetime, the time a TCP segment can exist in

  the internetwork system. Arbitrarily defined to be 2 minutes.

Page 28

   TIME-WAIT - represents waiting for enough time to pass to be sure

  the remote TCP received the acknowledgment of its connection

  termination request.

Page 73

TIME-WAIT STATE

  The only thing that can arrive in this state is a

  retransmission of the remote FIN. Acknowledge it, and restart

  the 2 MSL timeout.

There are two reasons for this wait. First to guarantee the remote's FIN gets ACKed even if the first ACK gets lost. Second to prevent "old duplicates" from a previous session from causing problems with a new connection. Both these are detailed extremely well here:

TIME_WAIT and its design implications for protocols and scalable client server systems - Asynchronou...

Here's another take on the same thing, and the implications for NAT/PAT servers.:

TIME_WAIT and &amp;#8220;port reuse&amp;#8221; | Dvas0004&amp;#039;s Blog

But two MINUTES for the MSL? You didn't know the Internet was first implemented in 1981 on Carrier Snails, did you?

In fact the original specification says "For this specification the MSL is taken to be 2 minutes. This   is an engineering choice, and may be changed if experience indicates it is desirable to do so." But nobody has revisited that sentence in the last 42 years as the "real computers" all have so much memory now that this doesn't matter.. Except for web servers which have to deal with this still.

Tom

[IRTrans]

Hi,

I have got one question: I just started with the FNET TCP/IP Stack.

I will use it with either the MCF52235 or the MCF5282 CPU. As I am using Win7 x64 I need to use CodeWarrior 10.3. The projects for my Microcontroller are, however, only available for CW 7.2.

Is it possible to simply port them to CW 10.3?

Using CW 7.2 on a Virtual Windows XP is not an option because USB debugging and flashing is not working reliable with Virtual machine.

Thanks in advance, Marcus

[butok]

FNET 2.4.0 released (http://fnet.sourceforge.net):

- Added multiple-session support to the HTTP server.
- Added HW Checksum Calculation support by ENET module (for K60).
- New configuration parameters:
    - FNET_CFG_HTTP_SESSION_MAX defines maximum number of simultaneous user-session that can be handled by the HTTP server.
    - FNET_CFG_CPU_ETH_HW_TX_PROTOCOL_CHECKSUM enables/disables insertion of protocol checksum by Ethernet-module.
    - FNET_CFG_CPU_ETH_HW_TX_IP_CHECKSUM enables/disables insertion of IPv4 header checksum by Ethernet-module.
    - FNET_CFG_CPU_ETH_HW_RX_PROTOCOL_CHECKSUM enables/disables discard of frames with wrong protocol checksum by Ethernet-module.
    - FNET_CFG_CPU_ETH_HW_RX_IP_CHECKSUM enables/disables discard of frames with wrong IPv4 header checksum by Ethernet-module.
    - FNET_CFG_CPU_ETH_HW_RX_MAC_ERR enables/disables discard of frames with MAC layer errors by Ethernet-module.
- Other minor changes.

Best regards,
Andrey Butok

FNET Community

[gogreen]

Hi Andrey

I have downloaded the FNET shell demo (FNET TCP/IP stack) on Kinetis K60 using FLASH-based target selection with IAR v6.10 tool.

I have included all the shell demo project files in IAR tool project options.

I was able to compile, build and load it on TWR-K60 board.

But there was no output on serial port. I couldn't see any output on any of these serial ports mentioned below.

a. UART3 port (on PE Micro Terminal utility) of TWR-K60 board

b. UART5 port (on TeraTerm Serial Port Console) of TWR-SER board

Can you please help me resolve this problem?

Thanks

Samrat

[butok]

Hi Samrat,

>> IAR v6.10 tool.

I use the  IAR v6.50.2 tool.

FNET demos tested on TWR-K60N512 + TWR-SER bard.

By default, for K60 demo applications, is used UART 3 on TWR-SER board (controlled by FNET_CFG_CPU_SERIAL_PORT_DEFAULT ).

If you are still failed => most probably you have wrong jumper setting (for some reason they are always wrong on new boards, in my case). Please look into proper User Manual.

Best reagrds,

Andrey Butok

[gogreen]

Hi Andrey

I was able to run TCP/UDP benchmarking demo. I want to clarify my understanding w.r.t. this demo.

I would like to know what data is sent from FBench to the Microcontroller (say MK60N512). Does FBench send 0's because I receive 0's in  MK60N512?

Please correct my understanding of sending/receiving data if I'm wrong.

Thanks in advance

Samrat

[gogreen]

Thanks for your reply.

You are right, the hardware jumper settings were changed and it worked.

I want to implement modbus protocol over TCP/IP stack.

Can you please let me know the .C and .H files of the stack I need to look at for developing the Modbus protocol over TCP/IP ?

Thanks in advance

Samrat

[IRTrans]

Hi,

in the meantime I have ported the MFC52235 Shell Demo project to CW 10.4 - it was no big deal, in fact only the project / compiler settings need to be done. Everything is working fine and as it should.

I have got 2 questions regarding specific parts of the Stack:

1. DHCP discovery does not work correctly in some networks. In one network I was using for test it works fine and without any problems. In another network the DHCP discoveries time out without any result. After waiting for a minute after boot up the DHCP discovery works when triggered again manually via the shell. I already did a bit of tracing using wireshark - it looks like there are no DHCP packets going out over the network. HW is the MC52235EVB connected to a Netgear GBit PoE Switch.

2. We are using http SSI extensively. In general they are working fine, however it looks like it is not possible to return an empty SSI value. After returning an empty value the page is not parsed further. Is that behaviour by design?

Thanks in advance, Marcus

[IRTrans]

In the meantime I was able to track down the DHCP problem a bit. In fact it is no DHCP problem. It looks like an icompatibility between the Device/Stack and the switch. Connected to a different switch in the same network everything is fine. With the "problem" switch it takes around 1 minute until the link is really up. The LNK LED of the MCF device is on but no data is transferred. That also happens with a ping if a static IP is used.

Very strange.

Has anybody seen that already?

Marcus