i.MX93 Shared memory access using UIO/mmap crashes (Segfaults) only when debugging with GDB

[Drehstromlampe]

Hello NXP and forum members,

we are working with the i.MX93 and want to use the shared memory to interact between the M33 and the A55 core(s), using a shared ringbuffer structure in this memory area.

The M33 can access the shared memory without problems, which does basically work on the A55 side, too, BUT: As soon as a connected GDB client tries to print the content of this shared memory, the program crashes with a segmentation fault:

Unable to handle kernel read from unreadable memory at virtual address ffff8000822e7000.

Which is strange, as the base of the shared memory block starts at 0x8F600000 and is mapped to virtual memory at FFFFF73E0000.

A bit more in detail:

We wanted to avoid to write a complete kernel driver for our needs, handling the memory access on user-space using the UIO drivers.

For this, we created two according entries in the device tree, defining the memory areas and assigning them to the UIO driver.

After mapping the memory area to a user-space area using mmap(), we can read and write the shared area without problems from both sides when not debugging/printing the memory areas using GDB.

Most IDEs and debugging frontends print local/function-local variables as soon as a breakpoint gets hit, leading to a quite unusable debugger for us, completely blocking our project at the moment

As described here https://stackoverflow.com/questions/654393/examining-mmaped-addresses-using-gdb , this problem was solved by adding a .access member to the vm_operations_struct in the according (custom) kernel driver.

But as far as I can tell, the according method seems to be implemented in the UIO driver used by us (CONFIG_HAVE_IOREMAP_PROT is enabled): https://github.com/phytec/linux-phytec-imx/blob/v6.6.52-2.2.0-phy/drivers/uio/uio.c 

We are using a SOM board by PHYTEC and are currently using their carrier board hardware "PhyBOARD Nash" until our own hardware is finished. Hence, we mostly use patches as "quick and dirty" in our yocto layer for the moment.

Attached to this post is a archive containing a minimal working app example, and a down-stripped version of our current yocto meta-layer.

What is pretty strange in my opinion, is that there are not much search results on the web about problems with UIO, mmap and gdb, but this seems to be a rather common way for accessing raw memory from user-space.

Do you have an idea, why the GDB seems to access wrong, unmapped/unaccessible memory areas or what could be causing our problems here?

Thanks in advance & best regards

Markus

[Bio_TICFSL]

Hello,

you have to fo with Phytec for better reference on this. A segmentation fault (segfault) during debugging indicates that the program attempted to access a memory location it was not permitted to access. This typically results in the operating system terminating the program with a SIGSEGV signal. Debugging tools like GDB can help pinpoint the cause.

Common causes of segmentation faults:

Dereferencing a null or uninitialized pointer: Attempting to access memory through a pointer that does not point to a valid memory location.

Out-of-bounds array access: Accessing an array element beyond its declared size.

Stack overflow: Excessive recursion or large local variables consuming all available stack space.

Writing to read-only memory: Attempting to modify a memory segment designated as read-only, such as the code segment. 

Use-after-free errors: Accessing memory after it has been deallocated.

• Obtain a backtrace: Once the segfault occurs, GDB will stop. Use the bt (backtrace) command to see the call stack leading up to the crash. This helps identify the function and line of code where the fault occurred. 
• Inspect variables and memory:
  Navigate through the stack frames using frame N (where N is the frame number from the backtrace) and use print variable_name to inspect the values of relevant variables, especially pointers, to identify potential issues like null or invalid addresses.
  Use memory analysis tools:

  Can be invaluable for detecting memory errors, including invalid memory accesses, uninitialized memory usage, and memory leaks, which can lead to segfaults.

  Regards

[Drehstromlampe]

Hello @Bio_TICFSL ,

thank you for your fast response.

Yes I am aware how a memory access fault could happen and the examples/causes you mentioned and described.

The GDB client itself seems to execute the wrong memory-read and causing the SIGSEGV crash, so this would not help us debug this issue, but seems to be causing it.

Perhaps you can you tell me, if in theory GDB should be able to access handled memory like we do (UIO + mmap)?

As unfortunately we can't test here at the moment with a "vanilla"/mainline kernel tree, as we are on the PHYTEC carrier board, we cannot check by ourselves if this is a PHYTEC branch or general (mainline) problem.

Best regards

Markus