Can't open HTTPSRV web pages. Explorer keeps in waiting

[kaiyueyang]

My chip is K60DN512. I was using MQX3.8 & HTTPD before, and it works well. I updated MQX to 4.2 and switched to HTTPSRV last week. Then I can't open the web pages any more. My project is using HTTPSRV, socket and ftp client. Socket and ftp client works well without HTTPSRV .

If I disable socket and ftp, the web pages can be opened. But the explorer will get a error page "can't connect the server" half the time.

I've traced the program. It seems stuck in httpsrv_server_task():

	/* Get socket with incoming connection (IPv4 or IPv6) */
	while (!connsock && server->run)
	{
	connsock = RTCS_selectset(&(server->sock_v4), 2, 250);
	}
	if (server->run)
	{
	struct sockaddr  remote_addr;
	unsigned short   length;

	new_sock = accept(connsock, (sockaddr *) &remote_addr, &length);
	connsock = 0;
	}
	else
	{
	break;
	}

Here is my rtcs init function:

uint_32 RTCS_Init(void)
{
    int_32            error;
    _enet_address     address;

    IPCFG_IP_ADDRESS_DATA ip_data;
    extern const HTTPSRV_CGI_LINK_STRUCT cgi_lnk_tbl[];
    extern const TFS_DIR_ENTRY tfs_data[];
    HTTPSRV_PARAM_STRUCT params;
    char index[20];

    /* runtime RTCS configuration */
    _RTCSPCB_init = 4;
    _RTCSPCB_grow = 2;
    _RTCSPCB_max = 10;
    _RTCS_msgpool_init = 4;
    _RTCS_msgpool_grow = 2;
    _RTCS_msgpool_max  = 10;
    _RTCS_socket_part_init = 4;
    _RTCS_socket_part_grow = 2;
    _RTCS_socket_part_max  = 10;
    _RTCSTASK_stacksize = 6000;
//    _int_install_unexpected_isr();

    //start rtcs
    error = RTCS_create();
    _IP_forward = TRUE;
    ENET_get_mac_address(DEMOCFG_DEFAULT_DEVICE, ENET_IPADDR, address);
    if (error != RTCS_OK)
    {
        printf("rtcs error1\n");
        return ERMD_RTCS_INIT_ERROR;
    }


    error = ipcfg_init_device (DEMOCFG_DEFAULT_DEVICE, address);
    ip_data.ip      = ENET_IPADDR;
    ip_data.mask    = ENET_IPMASK;
    ip_data.gateway = ENET_GATEWAY;
    error = ipcfg_bind_staticip (DEMOCFG_DEFAULT_DEVICE, &ip_data);
    if (error != RTCS_OK)
    {
        printf("rtcs error2\n");
        return ERMD_RTCS_INIT_ERROR;
    }
    //set ip address
    //set web page data
    error = _io_tfs_install("tfs:", tfs_data);
    if (error != RTCS_OK)
    {
        return ERMD_RTCS_INIT_ERROR;
    }
    /* Start HTTP server on each interface */
        _mem_zero(&params, sizeof(HTTPSRV_PARAM_STRUCT));
        snprintf(index, 20, "\\hardware.html");
        params.af = AF_INET;  //IPv4, IPv6 or from config.h
        params.root_dir = "tfs:";
        params.index_page = index;
        params.auth_table = auth_realms;

        /*
        ** If there is only one server listen on any IP address 
        ** so address can change in runtime (DHCP etc.).
        ** Otherwise we will use static IP for server.
        */
//        params.ipv4_address.s_addr = ENET_IPADDR;
        params.ipv4_address.s_addr = INADDR_ANY;
            /* Print active IPv4 address */
        /*
        ** Maximum default number of available sockets in RTCS is six.
        ** Maximum of two are used for HTTP server as listening sockets (one for IPv4 and one for IPv6).
        ** So we take two of sockets and use them for sessions.
        */
        params.max_ses = 2;
        params.cgi_lnk_tbl = (HTTPSRV_CGI_LINK_STRUCT*) cgi_lnk_tbl;

        /* There are a lot of static data in CGIs so we have to set large enough stack for script handler */
//        params.script_stack = 100;
        server = HTTPSRV_init(&params);
        if (server == NULL)
        {
            printf("HTTPSRV Init err\n");
        }



    return ERMD_OK;
}

[kaiyueyang]

I've found out what caused the problem. HTTPSRV_SERVER_STACK_SIZE in httpsrv_prv.h was set to 8500 by accident. When I modified it to 850, my server works well. But there's still a problem in my cgi receive function. I got a param->content_length as 1024. But when I run HTTPSRV_cgi_read(param->ses_handle, _tmp, 1024), I got a return as 864 or less. I'll research on it by myself first. Or maybe I'll open a new topic.

Thank you guys.

[kaiyueyang]

I used printf other than breakpoint to monitor the process.

Client should receive 3 files on loading a page, a html file, a css file and a js file.

After received 1 or 2 files , the chip will stuck in httpsrv_ses_process();

session->timeout should be 20000 given by httpsrv_ses_init()  as HTTPS@RVCFG_SES_TO I set in user_config.h.

I keep outputting the value of session->timeout. And sometime it would be 200.

So I set it to 20000 just at the head of httpsrv_ses_process();

Then it would cycle in case of HTTPSRV_SES_WAIT_REQ. And the returned value of httpsrv_readreq() was -1.

Then I printf the value of read in httpsrv_readreq()

read = recv(session->sock, buf+session->buffer.offset, HTTPSRVCFG_SES_BUFFER_SIZE-session->buffer.offset, 0);

I got the number 0x00001649 what is RTCSERR_TCP_TIMED_OUT.

Now I wonder was it right? And what's the reason of the strange change of session->timeout.

[dmitryalexeyev]

Looks like that's a bug in HTTPSRVs'  httpsrv_readreq function:

 /* Read data */
        read = recv(session->sock, buf+session->buffer.offset, HTTPSRVCFG_SES_BUFFER_SIZE-session->buffer.offset, 0);
        if ((read == RTCS_ERROR) || (read == 0))
        {
            if (RTCS_geterror(session->sock) == RTCSERR_TCP_CONN_CLOSING)
            {
                /* Client closed connection so we close it too */
                session->keep_alive = 0;
                session->state = HTTPSRV_SES_CLOSE;
            }
            break;
        }

Moved 'break' from line 11 into if {} block (after line 9) and it stopped dropping connections immediatelly.

( I connect to web server manually via telnet and receive immediate disconnection)

[karelm_]

Hi Dmitry,

this is not a bug. It is done like that intentionally. The socket is closed on server side in two cases:

1. Socket is closed by client.
2. Whole request is read.

Time for which HTTP server waits for request is defined by macro HTTPSRVCFG_RECEIVE_TIMEOUT. Its value is by default 50ms. If no request is received in 50ms (return value of read() is 0), session is closed to preserve resources. That is why you are disconnected from telnet terminal immediately after you connect. If you want to send requests from telnet, please increase value of this macro to at least 10 seconds (#define HTTPSRVCFG_RECEIVE_TIMEOUT 10000 in your user_config.h). You might also run into such problems on slow networks. If that is the case, increasing value to something like 250-500ms should resolve your problems. It is also possible to enable HTTP keep-alive to preserve the connection after response is sent. This can be done by defining HTTPSRVCFG_KEEPALIVE_ENABLED as 1. Session lifetime is then controlled by value of HTTPSRVCFG_KEEPALIVE_TO (200ms by default). Macro HTTPSRVCFG_SES_TO (20 seconds by default) sets timeout for sessions which does not use HTTP keep-alive but use i.e CGI.

Note: You have to recompile at least BSP and RTCS to apply your changes to macros in user_config.h

Edit:

This would be better, and it seems like a bug in HTTP server:

        /* Read data */
        read = recv(session->sock, buf+session->buffer.offset, HTTPSRVCFG_SES_BUFFER_SIZE-session->buffer.offset, 0);
        if ((read == RTCS_ERROR) || (read == 0))
        {
            uint_32 error;

            error = RTCS_geterror(session->sock);
            /* Client closed connection so we close it too */
            if (error == RTCSERR_TCP_CONN_CLOSING)
            {
                session->keep_alive = 0;
                session->state = HTTPSRV_SES_CLOSE;
            }
            /* 
             * If reading timed out, set number of read bytes to 0
             * to prevent session closing.
             */
            else if (error == RTCSERR_TCP_TIMED_OUT)
            {
               read = 0;
            }
            break;
        }

Thank you. You helped me to find a bug in HTTP server.

Best regards,

Karel

[kaiyueyang]

I captured data with wireshark. After client sent "GET / HTTP/1.1", MQX was supposed to send the web page. But I don't know where is it stuck.

[Martin_]

Put breakpoint to httpsrv_processreq() and debug into it to see how the processing of the HTTP request goes.

[kaiyueyang]

I set a breakpoint at the first line of httpsrv_processreq(), which is    "session->state = HTTPSRV_SES_RESP;".

It's very stange that sometime it will stuck at the line. I stepped the prog but it kept run this command.

But sometime it can move on. Then the program runs well.

I don't know whether this is a problem on Codewarrior Debugger.

[kaiyueyang]

Here is my socket summary:

before trying to open web pages:

after trying to open web pages:

Is this normal? What's the function of socket 0x1fff8f50? I thought client will connect to it but it seemed connected to 0x1fff90d0.

[karelm_]

Hi,

yes, this is completely normal. Socket 0x1FFF8F50 is listening on port 80 and accepts other connection requests from client. After accept() is called on this socket, new socket is created (0x1FFF90D0) and this socket is connected to the client. This allows HTTP server to process each request in new task while connections with clients are established in other task.

Best regards,

Karel

[kaiyueyang]

Since socket is normal, is it possible to be a tfs problem?

[karelm_]

Hi,

I have a few questions and remarks:

1. How many sockets you use in your custom application? You should further increase _RTCSPCB_max, _RTCS_msgpool_max and  _RTCS_socket_part_max if your application needs many sockets. In your current configuration HTTP server will use up to 3 sockets and FTP server will use 2 or more. So there are probably 5 left.

2. You should set value of variable script_stack according to requirements of your CGI scripts if you use them. Otherwise using CGI might cause stack overflow of HTTP server script handler task and server will stop to respond.

3. Set auth_table pointer only if your application uses authentication.

4. Return value of HTTPSRV_init() is unsigned integer, you should compare it to 0 rather than NULL.

Best Regards

Karel

[kaiyueyang]

I've increased _RTCSPCB_max, _RTCS_msgpool_max , _RTCS_socket_part_max and CGI scripts. It still can't work. I've even tried a plain web page without any cgi, can't work.

Here is the task summary when I try to open web pages

[Martin_]

in TAD you should check also: Stack Usage and (Lightweight) Memory Pools.

[kaiyueyang]

The stack used of these three tasks is under 20%. And I've used 86k of 108k memory(79%). Is it enough for httpsrv?

[kaiyueyang]

1.I only use 1 socket in my custom app.

2.My CGI doesn't send data throught url. So I don't know whether it will be affected by script_stack. I'll try it.

3.I need authentication in my application.

4.I'll do it.

Thank you very much.

I'm now in The National Day holiday. So I'll reply u after 8th.

[DavidS]

To add to Karel's post please have a look at following:

https://community.freescale.com/message/351405

Regards,

David