TCP/IP Task dequeued with PART: Out of Blocks in MQX for KSDK 1.1.0

[bowe]

We are using MQX for KSDK 1.1.0 on a MK64FN1M0.  We have two HTTP servers running (one will eventually be used for SSL) and a Telnet server running.  The issue we are seeing is occasionally the K64 will quit responding to ethernet, but we know it is still running because we still have serial (debug) output and are receiving data over SPI from another processor.  This issue seems to happen more or less randomly, and does not seem to be related to the number of clients connected.  The K64 does not recover until a reset, so I do not believe it is a timeout issue.

When we connect via JTAG to the K64 after the problem occurs, the TCP/IP task has code "PART: Out of Blocks" and has been dequeued (which explains why it never recovers).  Note that we have increased the socket partition size as follows, but the problem still occurs:

#define RTCSCFG_SOCKET_PART_INIT 32

#define RTCSCFG_SOCKET_PART_GROW 1

#define RTCSCFG_SOCKET_PART_MAX RTCSCFG_SOCKET_PART_INIT

Is this the expected method of failure for an out of blocks error?  It seems that a more graceful mode of failure would be to drop connections/packets instead of take down the whole TCP/IP task.  It is still unclear to us if we are actually running out of blocks, or if blocks are not getting released properly (or if the partition getting corrupted).  Any ideas?

Thanks,

-Bowe

[Luis_Garabo]

Hi Bowe,

Are you adding enough PCBs for your application? The web_hvac demo application is a good example of how to implement this.

Try to add the next lines:

    _RTCSPCB_init = 8;

    _RTCSPCB_grow = 2;

    _RTCSPCB_max = 20;

    _RTCS_msgpool_init = 8;

    _RTCS_msgpool_grow = 2;

    _RTCS_msgpool_max  = 20;

    _RTCS_socket_part_init = 8;

    _RTCS_socket_part_grow = 2;

    _RTCS_socket_part_max  = 20;

These lines will ensure you to have at least 8 sockets available in the system. Remember that your server will already take one for each http server.

I hope this helps you.

Garabo

-----------------------------------------------------------------------------------------------------------------------

Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[bowe]

I have implemented the tweaks to the PCB partition, socket partition, and message pool you provided, and it is seeming more stable from what I can tell.  I will continue to test it over the next week and see (before it varied, but could take about 2 days to crash).  Currently, I have the following as far as partition usage over the last 2 days of running (with RTCSCFG_TCP_MAX_CONNECTIONS set to 8):

     Socket Partition:  8 Total Blocks, 0 Free, 8 Used, 2 Growth, 20 Limit

     PCB Partition:  18 Total Blocks, 1 Free, 17 Used, 2 Growth, 20 Limit

So if this does actually fix the problem, I have questions as to why.  If it ensures 8 sockets available (and we can expand up to 20, right?), then how do we ensure we will never try to open more than the 20 maximum sockets?  Is this controlled by setting RTCSCFG_TCP_MAX_CONNECTIONS, where we basically need to have a socket for each server (3 in my case of 2 HTTP servers and 1 Telnet server) to listen and a socket for each TCP connection, so I would potentially need 11 sockets?  If so, then given that the socket partition is still only 8 big, then I am not hitting my max TCP connections limit (or even hitting 5 TCP connections).  Why is the PCB Partition growing so much?  Should I be concerned that it might run out of room?  How should I tweak these settings if I want to increase the max TCP connections?

Thanks,

-Bowe

[Luis_Garabo · Accepted solution]

Hi Bowe,

The maximum number of sessions that your server can handle is set with this macro: HTTPSRV_EXAMPLE_SESSIONS. This macro will prevent to open more session that the one allowed by this macro.

For each HTTP server you need 1 socket that will be used to listen for new connections. Then each session will need one more socket.

The Packet Control Blocks are used within all protocol layers which deal with datagram-like chunks of data. It contains packet data. Each socket uses 1 PCB. and each PCB has two fragments. So in summary you will see two PCBs pointers per socket.

The extra PCB that you are seeing might be a dummy array to indicate the end of the array.

You should be calculate the amount of sockets and PCB that you will need according with the amount out max sessions that you will allow for your http servers.

I hope this answers your questions.

Have a great day,
Garabo

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[bowe]

Ah, thank you.  That is what I was looking for.  It looks like most parameters that are set 0 get set to a default value in httpsrv_set_params() in httpsrv_supp.c.  In the case of max sessions, HTTPSRVCFG_DEF_SES_CNT is the default, and is set to 2.

In our case, we had one http server set to 4 max sessions, the other http server set to 0 (which defaults to 2), and then the telnet session set to 2 max sessions.  So give 1 PCB for each server (2 http, 1 telnet) gives 3 PCBs.  Then 2 PCBs per session (2*4 http, 2*2 http, 2*2 telnet) gives 16 PCBs.  This gives a grand total of 19 PCBs, so our maximum of 20 PCBs should be adequate.  Please correct me if I am wrong.

Thanks,

-Bowe

[Luis_Garabo]

Hi Bowe,

Your maths are correct! Good luck with your implementation!

Regards,

Garabo

[bowe]

In looking at the partitions after a crash, it appears all blocks (and not just in socket partition, but all partitions) are "used", with an owner of 0x1FFFF.  I assume this is the maximum task ID, and indicates some type of an error/bug that is occurring to the blocks (probably upon allocation or release?).

-Bowe