isp_media_server segmentation fault at VIV_VIDEO_EVENT_SET_CROP or other mmap operations

[r3gl0]

I'm currently working with the NXP IMX8MPLUS and trying to enable the ISP for the MT9V024 camera sensor.

My current set up is following:

Yocto core image with IMAGE_INSTALL:append = " kernel-module-isp-vvcam gstreamer1.0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad v4l-utils isp-imx libtinyxml2 media-ctl fmt ".

Linux 5.15.71 Kirkstone
Is a headless build

Externally, I build:

- isp-imx-4.2.2.20.0
- isp-vvcam (lf-5.15.y_2.2.0)
and then I send the generated files to the board.

My current bottle neck is that when I run

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw,format=YUY2,width=752,height=480" ! queue ! videoconvert ! theoraenc ! oggmux ! filesink location=test_image.ogv

isp_media_server crashes in the VIV_VIDEO_EVENT_SET_CROP case. To be more specific it crashes here:

---(source isp-imx-4.2.2.20.0/mediacontrol/daemon/V4l2Event.cpp)--

case VIV_VIDEO_EVENT_SET_CROP: {
ALOGI("VIV_VIDEO_EVENT_SET_CROP"); // I add this for debug.

if (bufMap.find(event.addr) == bufMap.end()) {
bufMap[event.addr] = ::mmap(NULL, VIV_JSON_BUFFER_SIZE,
PROT_READ | PROT_WRITE, MAP_SHARED, eventDevice, event.addr);
}
struct viv_rect *rect = (struct viv_rect *)bufMap[event.addr]; // gdb points at this line.

// It is weird to fail here because v4l2-ctl --all -d /dev/video0 works well (check log)

-----

---
It also crashes when I perform any other operation like v4l2-ctl -d 0 -c viv_ext_ctrl='{<id>:<pipeline.s.dwe.onoff>;<enable>:false}'

Here i had heavily modify the code for testing purposes:

case VIV_VIDEO_EVENT_EXTCTRL: {
ALOGI("VIV_VIDEO_EVENT_EXTCTRL: Event received.");
ALOGI("VIV_VIDEO_EVENT_EXTCTRL: event.addr = 0x%llx", (long long)event.addr);
ALOGI("VIV_VIDEO_EVENT_EXTCTRL: eventDevice = %d", eventDevice);
ALOGI("VIV_VIDEO_EVENT_EXTCTRL: VIV_JSON_BUFFER_SIZE = %zu", (size_t)VIV_JSON_BUFFER_SIZE);

{

struct stat st;
if (fstat(eventDevice, &st) == -1) {
perror("fstat failed");
} else {
printf("eventDevice size: %lld bytes\n", (long long)st.st_size);
printf("Trying to mmap at offset: %lld\n", (long long)event.addr);
}
}
{

struct v4l2_requestbuffers req = {0};
req.count = 1;
req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
req.memory = V4L2_MEMORY_MMAP;

if (ioctl(eventDevice, VIDIOC_REQBUFS, &req) == -1) {
ALOGE("VIDIOC_REQBUFS failed: %s", strerror(errno));
} else {
ALOGI("Requested %u buffers, got %u", req.count, req.count);
}

struct v4l2_buffer buf = {0};
buf.index = 0;
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;

if (ioctl(eventDevice, VIDIOC_QUERYBUF, &buf) == 0) { // if request buffer succeded
ALOGI("Buffer offset from VIDIOC_QUERYBUF: 0x%x", buf.m.offset);
}

if (event.addr % getpagesize() != 0) {
ALOGE("event.addr is not page-aligned: 0x%llx", (long long)event.addr);
break;
}
if (fcntl(eventDevice, F_GETFD) == -1){
ALOGI("Invalid File descriptor");

}

void *test_map = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_SHARED, eventDevice, 0);
if (test_map == MAP_FAILED) {
ALOGE("Test mmap failed with fd=%d, errno=%d (%s)", eventDevice, errno, strerror(errno));
} else {
ALOGI("Test mmap succeeded. Address: %p", test_map);
munmap(test_map, 4096);
}

long page_size = getpagesize();
off_t small_offset = page_size; // try offset 4096
size_t small_size = page_size; // 4096 bytes

void* second_test_map = mmap(NULL, small_size, PROT_READ | PROT_WRITE, MAP_SHARED, eventDevice, small_offset);
if (second_test_map == MAP_FAILED) {
ALOGE("Second test mmap failed: fd=%d, offset=0x%llx, errno=%d (%s)",
eventDevice, (long long)small_offset, errno, strerror(errno));
} else {
ALOGI("Second test mmap succeeded. Address: %p (offset=0x%llx)",
second_test_map, (long long)small_offset);
munmap(second_test_map, small_size);
}

}

void *mapped_json_buffer = NULL; // Use a temporary pointer for mmap result

ALOGI("Calling mmap(NULL, %u, PROT_READ|PROT_WRITE, MAP_SHARED, %d, 0x%llx)",
VIV_JSON_BUFFER_SIZE, eventDevice, (long long)event.addr);

// Check if the address is already mapped in bufMap
auto it = bufMap.find(event.addr);
if (it == bufMap.end()) {
ALOGI("VIV_VIDEO_EVENT_EXTCTRL: event.addr not found in bufMap. Attempting mmap...");
// Not found, attempt to mmap
mapped_json_buffer = ::mmap(NULL, VIV_JSON_BUFFER_SIZE,
PROT_READ | PROT_WRITE, MAP_SHARED, eventDevice, event.addr);

if (mapped_json_buffer == MAP_FAILED) {
// CRITICAL ERROR: mmap failed
ALOGE("VIV_VIDEO_EVENT_EXTCTRL: FAILED to mmap memory for event.addr=0x%llx, fd=%d, size=%zu. errno=%d (%s)",
(long long)event.addr, eventDevice, (size_t)VIV_JSON_BUFFER_SIZE, errno, strerror(errno));
}
// mmap successful, store in bufMap
ALOGI("VIV_VIDEO_EVENT_EXTCTRL: mmap successful. Mapped address: 0x%p", mapped_json_buffer);
bufMap[event.addr] = mapped_json_buffer;
} else {
// Already mapped, retrieve from bufMap
ALOGI("VIV_VIDEO_EVENT_EXTCTRL: event.addr found in bufMap. Retrieving mapped address.");
mapped_json_buffer = it->second;
}

ALOGI("VIV_VIDEO_EVENT_EXTCTRL: Mapped buffer pointer: 0x%p", mapped_json_buffer);

// Double-check if the pointer is valid before using it
if (mapped_json_buffer == MAP_FAILED || mapped_json_buffer == NULL) {
ALOGE("VIV_VIDEO_EVENT_EXTCTRL: Mapped buffer is invalid (0x%p) after mmap/lookup. Skipping JSON parsing.", mapped_json_buffer);
}

Json::Reader reader;
// Now, safely use the mapped_json_buffer
std::string str = (char *)mapped_json_buffer; // This line crashes

---
The second test map is failed which makes me think that there is trying to map a empty space.

--------
This is my init sequence

cd
cd build-out/sdk
export LD_LIBRARY_PATH=$pwd:$LD_LIBRARY_PATH
export GST_DEBUG=4
systemctl stop imx8-isp.service
export ISP_LOG_LEVEL=8

rmmod vvcam-video
rmmod vvcam-dwe
rmmod vvcam-isp
rmmod imx8-media-dev.ko
insmod mt9v024.ko
insmod vvcam-isp.ko
insmod vvcam-dwe.ko
insmod vvcam-video.ko # this sets the previous drivers as media
insmod imx8-media-dev.ko
./isp_media_server CAMERA0&

and then one previously mentioned commands (gts or v4l2)
---
I have barely change the OS08a20_8M_10_1080p_hdr.xml and sensor_dwe_os08a20_1080P_config.json so i got my custom files, but i think it should be enough to get a image, even if its not good quality, i changed the resolution to 720x480p,
Files overview:

root@imx8mp:~/build-out/sdk# ls
MT9V024_8M_10_480p_linear.xml libadpcc.so libcam_device.so libhal.so libsom_ctrl.so
OS08a20_8M_10_1080p_hdr.xml libadpf.so libcam_engine.so libi2c_drv.so libversion.so
OS08a20_8M_10_1080p_linear.xml libaec.so libcameric_drv.so libibd.so libvom_ctrl.so
OS08a20_8M_10_4k_hdr.xml libaee.so libcameric_reg_drv.so libisi.so libvvdisplay_shared.so
OS08a20_8M_10_4k_linear.xml libaf.so libcim_ctrl.so libjsoncpp.so mt9v024.drv
Sensor0_Entry.cfg libaflt.so libcommon.so libjsoncpp.so.1.9.0 mt9v024.ko
Sensor0_Entry_mt9v024.cfg libahdr.so libcppnetlib-client-connections.so libjsoncpp.so.21 os08a20.drv
Sensor0_Entry_os08a20.cfg libappshell_ebase.so libcppnetlib-client-connections.so.0 libmedia_server.so os08a20.ko
Sensor1_Entry.cfg libappshell_hal.so libcppnetlib-client-connections.so.0.13.0 libmim_ctrl.so ov2775.ko
Sensor1_Entry_mt9v024.cfg libappshell_ibd.so libcppnetlib-server-parsers.so libmipi_drv.so run.sh
Sensor1_Entry_os08a20.cfg libappshell_oslayer.so libcppnetlib-server-parsers.so.0 libmom_ctrl.so start_isp.sh
VSI_Monitor.cfg libavs.so libcppnetlib-server-parsers.so.0.13.0 libmt9v024.so test_image.ogv
basler-camera-driver-vvcam.ko libawb.so libcppnetlib-uri.so libmt9v024.so.1 video_test
dewarp_config libawdr3.so libcppnetlib-uri.so.0 libmt9v024.so.1.0.0 vvcam-dwe.ko
imx8-media-dev.ko libbase64.so libcppnetlib-uri.so.0.13.0 libos08a20.so vvcam-isp.ko
isp_media_server libbufferpool.so libdewarp_hal.so libos08a20.so.1 vvcam-video.ko
liba2dnr.so libbufsync_ctrl.so libebase.so libos08a20.so.1.0.0 vvext
liba3dnr.so libcam_calibdb.so libfpga.so liboslayer.so
\root@imx8mp:~/build-out/sdk# ls dewarp_config/
sensor_dwe_bypass_480P_config.json sensor_dwe_os08a20_1080P_config.json sensor_dwe_os08a20_4K_config.json
root@imx8mp:~/build-out/sdk# cat dewarp_config/sensor_dwe_bypass_480P_config.json
{
"dewarpConfigArray" :[
{
"source_image":{
"width" : 752,
"height" : 480
},

"?dewarpType": "LENS_CORRECTION, FISHEYE_EXPAND, SPLIT_SCREEN",
"dewarpType": "FISHEYE_DEWARP",

"scale": {
"roix" : 0,
"roiy" : 0,
"factor" : 1.0
},

"split": {
"horizon_line" : 240,
"vertical_line_up" : 367,
"vertical_line_down": 367
},

"bypass" : true,
"hflip" : false,
"vflip" : false,

"camera_matrix" : [ 3.9177894938743270e+003, 0., 1.9716642989691159e+003, 0.,3.9177894938743270e+003, 1.1024451486734267e+003, 0., 0., 1. ],
"distortion_coeff": [ -1.3019040624278982e-001, 2.2621318183135611e-001,1.2109476203374344e-003, 9.4317229120640680e-004,-7.1053958306120402e-001 ],
"perspective" : [1.0, 0, 0, 0, 1, 0, 0, 0, 1]

}
]

}
root@imx8mp:~/build-out/sdk# cat Sensor0_Entry.cfg
name="mt9v024"
drv = "mt9v024.drv"
mode= 0

[mode.0]
xml = "MT9V024_8M_10_480p_linear.xml"
dwe = "dewarp_config/sensor_dwe_bypass_480P_config.json"root@imx8mp:~/build-out/sdk#

---
This is how my media looks like (I recalled I tried another isp project and I had media1 as well, but that was in yocto scarthgap, so I dont know if should be the same here):

root@imx8mpmk5:~/build-out/sdk# media-ctl -p -d /dev/media0
Media controller API version 5.15.71

Media device information

driver mxc-md
model FSL Capture Media Device
serial
bus info
hw revision 0x0
driver version 5.15.71

Device topology
- entity 1: mxc-mipi-csi2.0 (8 pads, 1 link)
type Node subtype V4L flags 0
device node name /dev/v4l-subdev2
pad0: Sink
<- "mt9v024 2-0048":0 [ENABLED,IMMUTABLE]
pad1: Sink
pad2: Sink
pad3: Sink
pad4: Source
pad5: Source
pad6: Source
pad7: Source

- entity 10: mt9v024 2-0048 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev3
pad0: Source
[fmt:unknown/0x0]
-> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]

root@imx8mpm:~/build-out/sdk# media-ctl -p -d /dev/media1
Failed to enumerate /dev/media1 (-2)

----

I hope someone can give me a hint of how to further debug this or point me which component is generating this issue.

[Bio_TICFSL]

Hello

with the https://docs.nxp.com/bundle/AN13712/page/topics/start_isp_media_server.html

maintain the extracted isp-imx in your repository for customized driver development. However, the diff tool, did not see the difference of .so files during the migration from isp-imx-4.2.2.6.0 to isp-imx-4.2.2.11.0 . Thus, they stayed the same as isp-imx-4.2.2.6.0 and faced the same issue as described in IMX8M-Plus Camera Integration and using ISP.

Regards

[r3gl0 · Accepted solution]

Hey, I think that isp-imx-4.2.2.20.0 does not have the mentioned migration issue.

I fixed it by utilizing:
./build-all-isp.sh release partial V4L2
instead of
./build-all-isp.sh release partial

I was reading a manual from yocto scarthgap which default mode is V4L2, so I was building in native mode instead of V4L2, since in isp-imx-4.2.2.20.0 the default mode is "native sensor".