Implementing Dynamic Linker/Loader for Plugin Functionality in Cortex-M Microcontrollers with FreeRT

[mastupristi]

Hello

I am working on a firmware project involving RT117x. My development stack includes FreeRTOS, LittleFS, and the gcc-arm-none-eabi toolchain. I'm currently exploring the implementation of a dynamic linker/loader system to enable plugin functionality, where clients can independently develop and deploy their algorithms.

While I am aware of solutions like the Linkable Loadable Extensions (LLEXT) in Zephyr OS and dynamic linking approaches like udynlink, I am constrained to use FreeRTOS for this project, and thus, I am seeking alternative solutions within this environment.

Specifically, I am looking for insights on:

1. Plugin File Format and Loading: Considering the dynamic linker/loader approach, what file format and loading strategy would be optimal?
2. Function Entry Points Management: How to effectively manage function entry points in non-PIC firmware for plugin interaction?
3. Memory Allocation Strategies: Best practices for handling different memory types (DTCM and SRAM) for plugin code and data.
4. Compilation and Linking Requirements: Essential parameters for compiling and linking, including any necessary post-linking steps.
5. Security and Stability: Ensuring system security and stability when dynamically loading third-party plugins.

I'd appreciate any advice, experiences, or references to similar projects within the NXP ecosystem. Thank you for your support!

best regards

Max

[boe_identified]

Hi @mastupristi 

I just started looking at shared objects in ELF ET_DYN format ( *.so using linux) for Zephyr or FreeRTOS and came across this post.

Perhaps that ET_DYN / shared objects could be of some use to you?

Allows import and export, allows some introspection although there is no info on function (parameter) types as far as I can tell.

Plausibly you have made your decisions by now (yes, I am "a bit late to the party"); if so perhaps you have some pointers you would like to share ?

Kind regards

BoE

[mastupristi]

Hi @boe_identified 

I don't know ELF ET_DYN format. I tried googling something, but if you give me some links it might be better.

I started studying zephyr's llext with the idea of porting what I need

best regards

Max

[boe_identified]

Have not made notes of (most) bits and pieces I have gathered, but below a (very) brief summary of my (current) understanding, followed by some reasonably related and relevant links.

Summary

Info that can be found a shared object file (elf ET_DYN)

• names of symbols exported from the shared object 
  • for code and data

• names of symbols imported to the shared object 
  • for code and data

• info needed to load and use symbols
  • type info not included
  • notably number and type of function parameters must be known 

Unsurprisingly loading to be done in close cooperation with OS used; loading mechanism can be provided by OS.

Seems to be outside of FreeRTOS (primary) scope; while I have seen some pointers to rolling-your-own loader, I don't seem to have saved them.

I think (some) Lite-OS related OS's provide dynamic loading

QNX neutrino docs describe how

Links

• https://docs.oracle.com/cd/E19683-01/816-1386/chapter6-43405/index.html
  • Table 7-3 ELF File Identifiers

• https://en.wikipedia.org/wiki/Executable_and_Linkable_Format#File_header
  • Table "ELF header", offset 0x10, 2 bytes, e_type

• https://en.wikipedia.org/wiki/Dynamic_loading
  • https://en.wikipedia.org/wiki/Dynamic_loading#In_C/C++

"... Not all systems support dynamic loading. Unix-like operating systems such as macOS, Linux, and Solaris provide dynamic loading with the C programming language "dl" library. ..."

• https://stackoverflow.com/questions/50303305/elf-file-type-et-exec-and-et-dyn
  • gcc can produce shared objects, sometimes even (unexpectedly) by default 8-)

• https://en.wikipedia.org/wiki/LiteOS

• https://www.qnx.com/developers/docs/7.1/#com.qnx.doc.neutrino.sys_arch/topic/dll.html

[nxf77486]

Hello,

Thank you in advance for contacting NXP support.

Can you please help me clarifying a point you are willing to use freertos to allocate memory in a dynamic way like in the following post?

Or you are just looking to have an specific link file for the application?

[mastupristi]

Can you please help me clarifying a point you are willing to use freertos to allocate memory in a dynamic way like in the following post?

Or you are just looking to have an specific link file for the application?

I'll try to clarify further (although I think I explained it well enough in my previous post)

I have my own FW, but I have to provide the user with the ability to write, compile, and run its plugins.

What does this mean? The user should not be forced to compile the entire FW by adding its algorithms into the complete set of source files. Conversely, it should be possible to add user algorithms at runtime (the user will be provided with some kind of SDK).

My development stack includes:

• gcc-arm-none-eabi toolchain: The toolchain used to compile the FW, and presumably to be used to compile the plugin as well, unless ways are found with which to compile with other toolchains (such as: llvm clang, rust, etc.).
• FreeRTOS: The FW uses FreeRTOS, and the plugin will have to be allowed to use the APIs that FreeROTS provides, such as: task creation, task synchronization, semaphores, mutexes, dynamic memory, etc.
• LittleFS: The board has flash memory that will be used as mass storage, and it will be used to store plugins file to load.

Of course, the whole feature must also include the dynamic linker/loader (which will be contained in the FW).

Searching the Internet, I found a couple of projects:

• Linkable Loadable Extensions (LLEXT) in Zephyr OS (https://docs.zephyrproject.org/latest/services/llext/index.html Unfortunately, I am forced to use FreeRTOS, so I cannot use this solution. Besides, it's still experimental and not yet stable or even released
• Udynlink (https://github.com/bogdanm/udynlink the project is a bit old, and of "pre-alpha" quality. Other than that it seems very rigid and insufficiently flexible regarding the compilation process.
  There are a number of forks that I should study better.

so I repeat and argue the most important and urgent points that I need to know/understand:

1. Plugin File Format and Loading: In considering the implementation of a dynamic linker/loader for plugin functionality, I am prioritizing a file format that allows for introspection to properly load the necessary memories (both instructions and data). I aim to adopt a simple yet standard format, preferably one that's already been used in similar contexts, to avoid reinventing existing solutions. What file format and loading strategy would you recommend that aligns with these requirements?
2. Function Entry Points Management: In my plugin implementation scenario, unlike typical cases where plugins solely provide functionalities to the host, my plugins will also need to utilize APIs offered by the firmware. I've deduced that the plugins should have two function pointer tables: one for the functions they provide and another for the functions they use from the firmware. Hence, I believe the linker/loader should populate this second table. Additionally, the firmware must be designed to retrieve function pointers at runtime for invoking functions provided by the plugin. How can I effectively manage these function entry points and ensure seamless interaction between the firmware and the plugins?
3. Compilation and Linking Requirements: In light of the requirement for dual function pointer tables in my plugins (one for functions they provide and another for functions they use from the firmware) I am seeking guidance on the compilation process. I anticipate needing to use flags like -fPIC or -fPIE and link with options like --unresolved-symbols=ignore-in-object-files and --emit-relocs. However, I would like to know what additional parameters are necessary. Specifically, are the standard compiler and linker sufficient to 'create' these tables, or is there a need for any post-build steps? Should I anticipate the need to utilize existing tools or develop custom solutions for this purpose?
   In addition to understanding the compilation process for the plugins, I also need guidance on how to modify the compilation of my firmware. Specifically, how should I adjust the firmware's compilation process to enable or facilitate the retrieval of function pointers that the plugins require? Is there a particular approach or modification necessary to ensure smooth interaction between the firmware and the plugins, especially in the context of function pointer handling?

I hope this will be enough to clarify the use case and what I'm looking for

regards

Max

[nxf77486]

Hello,

Thank you in advance for clarifying your information an need I believe that in the Chapter 19 of the MCUXpresso documentation  you can find a root to your implementation. Please let me know if there is anything else where I can help you.

[mastupristi]

I read chapter 19, but it talks about how to configure MCUXpresso IDE to build executables with various options.

I can't figure out how this is going to help me do what I need to do.

Could you kindly point me to or cite the sections that you think are most useful?

In the meantime, I have made the decision to use the elf format for plugins. With this Question 1 should close. This will obviously force me to find an elf interpreter that is feasible in the baremetal context. Do you have any advice for me? Do you know any elf parser projects for microcontrollers (maybe on github)?

regards

Max

[nxf77486]

Hello,

Please give me a moment to verify if we have something we can offer for the .elf parser. I would let you know ASAP.

[mastupristi]

Hi @nxf77486,

do you have any news?

regards

Max