Implementing MQX Task Manager

[mohwaqas12]

Hello mqx users,

I have been using mqx to prototype my system. I have created many tasks that run and perform different kind of work. I need to have a task that runs as a supervisor to manage all running tasks. I need to monitor their error codes/health somehow  to manage them and to decide what to do in case something goes wrong (e.g un handled  interrupts ) and to take appropriate action maybe abort or restart them.

I went through MQXRM , and found out 2 functions, _task_get_error, _task_set_error, that are used to set or get error codes, but unfortunatly they work for currently active task.

I need to know any such mechanism  that i should follow to determine error codes of all tasks.

I believe since mqx do not has such function it should be like this

In Supervisor task:

1. Get Kernal Data
2. Iterate through task list
3. Determine Error Code of Each task
4. Decide what to do ,  (abort them, restart them) or do nothing
5. sleep for sometime

I do not yet to know this would work or not, this is what i assumed after seeing task structure in mqx soure code.

If this is how they can be managed , Do i have to disable interrupts or maybe disable preemption  while supervisor task is running.

Or maybe i could set up exception handlers for each task or use global events to indicate something has gone wrong to supervisor, to let it decide what to do.

Regards

[JuroV]

Hello,

your assumption is right and should work under assumption that during critical section when accessing kernel_data, no other task nor interrupt is accessing the kernel_data, too.

This is a way how to get directly to the kernel structure, however, it is not clean solution.

There is another solution, maybe a bit cleaner, but needs small hack. You can modify the _task_set_error function to set the error code for the task, but also to inform the supervisor task about the code (global static data- structured table {task_id, error_code}). Note that you must solve mutual access to the structure (task A wirtes, supertask reads).

[mohwaqas12]

Hello Juraj

Your proposed solution seems to be much better. Instead of global structure, what if i use a message queue in _task_set_error to send a message to supervisor queue, in this case supervisor would mostly be sleeping and only be awaken (with a high priority) when it receives  a message from task indicating a change in error code.

Supervisor task would check what to do if certain or unknown error codes are generated, performs necessary  steps and goes back to message waiting state.

What should be the case for un handled exceptions that may arise in task. Would this solution work, if i set exception handlers for tasks and send a message to supervisor queue from inside exception handlers.  In this case supervisor task would be able to detect exceptions and abort that task or restart it if necessary.

Regards

[JuroV]

Hello Muhammad,

your solution seems also fine, but needs some considerations. First consideration is the behaviour of _task_set_error, which  can be quite different from the previous one: _lwmsgq_send can block the caller task, i.e. _task_set_error can block the active task (=huge difference from previous behaviour, but this does not necessarily need to be an issue). The issue can be avoided by not setting any blocking flag in call of _lwmsgq_send fcn.

Next issue coming into mind is: what happens if the queue is full (and blocking is disallowed)? This can be practically limited by large message queue and high priority of supervisor task (higher than any task that call _task_set_error().

You can send another important messages instead of task error codes to the queue, also in the exception. The only limitiation is that you cannot _lwmsgq_send with blocking flag. In the supervisor call you can then restart the task.

[Teckna]

Hello,

Maybe some useful supervisor task manager functions could be: tick count, stack utilization, and so on.

What do you think about it?

Many thanks

Teckna

[JuroV]

Hi Teckna,

thank you for your input. I think that these features are quite interesting, but we are not going this way for now. We will always support developers to implement what they need with MQX, but we are not planning for now to add such "supervisor task manager" to MQX.