topic Re: gst-launch-1.0 returns: Internal data stream error in i.MX Graphics

gst-launch-1.0 returns: Internal data stream error

btarnowski — Thu, 06 Oct 2022 21:20:11 GMT

Hi all. I'm working with imx8mm and camera sensor os02g10.
I added the Rockchip driver for this sensor as a patch.
It supports only RAW10 pixelformat. I can capture data from the sensor with command:

v4l2-ctl -d /dev/video0 --verbose --set-fmt-video=width=1920,height=1080,pixelformat=BG10 --stream-mmap --stream-count=1 --stream-to=raw10.raw

The output file needs to be processed to get correct size and raw natural green image (before White Balancing). It has to be done with PC computer with using this command:

gst-launch-1.0 -v filesrc location=path_to_file_from_camera_sensor.raw blocksize=42923008 ! "video/x-bayer,format=bggr,width=1920,height=1080,framerate=1/1" ! bayer2rgb ! videoconvert ! avenc_mjpeg ! filesink location=frame.jpeg

So the data from camera sensor looks correct but the target device (not PC) has issue with gst-launch-1.0. The goal is to get this command to work:

gst-launch-1.0 v4l2src num-buffers=1 ! jpegenc ! filesink location=picture.jpg

There is a my log for this command (patch added to have additional messages):

root@abcd:~# gst-launch-1.0 v4l2src num-buffers=1 ! jpegenc ! filesink location=/tmp/test.jpg
[ 161.334441] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_open
[ 161.340830] mx6s-csi 32e20000.csi_bridge: v4l_subdev: mxc_mipi-csi.0
[ 161.347230] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high request.
[ 161.353969] os02g10 3-003d: in function: os02g10_s_power
[ 161.359313] os02g10 3-003d: in function: os02g10_runtime_resume
[ 161.365272] os02g10 3-003d: in function: __os02g10_power_on
[ 161.370922] os02g10 3-003d: mux_oe_gpio - dir 1
[ 161.381504] os02g10 3-003d: pwdn_gpio - dir 1
[ 161.386003] os02g10 3-003d: mux_sel_gpio - dir 1
[ 161.395694] os02g10 3-003d: reset_gpio - dir 1
[ 161.410247] os02g10 3-003d: OS02G10_REG_SOFTWARE_RESET
[ 161.415915] m01_b_os02g10 3-003d: return value: 0
[ 161.420762] mxc_mipi-csi.0: return value: 0
[ 161.425074] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_enable
[ 161.431531] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.437764] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.444582] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.451750] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_init
[ 161.458176] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_querycap
[ 161.465522] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_close
[ 161.472011] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.478216] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.485041] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.492207] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_disable
[ 161.498760] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_deinit
[ 161.505330] os02g10 3-003d: in function: os02g10_s_power
[ 161.510686] os02g10 3-003d: in function: os02g10_runtime_suspend
[ 161.511644] m01_b_os02g10 3-003d: return value: 0
[ 161.516723] os02g10 3-003d: in function: __os02g10_power_off
[ 161.521523] mxc_mipi-csi 32e30000.mipi_csi: in function: mipi_csis_stop_stream
[ 161.527212] mxc_mipi-csi.0: return value: 0
[ 161.538735] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high release.
[ 161.617586] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_open
[ 161.623974] mx6s-csi 32e20000.csi_bridge: v4l_subdev: mxc_mipi-csi.0
[ 161.630373] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high request.
[ 161.637110] os02g10 3-003d: in function: os02g10_s_power
[ 161.642447] os02g10 3-003d: in function: os02g10_runtime_resume
[ 161.648403] os02g10 3-003d: in function: __os02g10_power_on
[ 161.654048] os02g10 3-003d: mux_oe_gpio - dir 1
[ 161.664367] os02g10 3-003d: pwdn_gpio - dir 1
[ 161.668752] os02g10 3-003d: mux_sel_gpio - dir 1
[ 161.677952] os02g10 3-003d: reset_gpio - dir 1
[ 161.692776] os02g10 3-003d: OS02G10_REG_SOFTWARE_RESET
[ 161.698437] m01_b_os02g10 3-003d: return value: 0
[ 161.703283] mxc_mipi-csi.0: return value: 0
[ 161.707602] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_enable
[ 161.714064] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.720266] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.727074] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.734232] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_init
[ 161.740647] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_querycap
[ 161.748024] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_close
[ 161.754493] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.760696] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.767518] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.774690] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_disable
[ 161.781247] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_deinit
[ 161.787801] os02g10 3-003d: in function: os02g10_s_power
[ 161.793164] os02g10 3-003d: in function: os02g10_runtime_suspend
[ 161.794252] m01_b_os02g10 3-003d: return value: 0
[ 161.799217] os02g10 3-003d: in function: __os02g10_power_off
[ 161.804027] mxc_mipi-csi 32e30000.mipi_csi: in function: mipi_csis_stop_stream
[ 161.809705] mxc_mipi-csi.0: return value: 0
[ 161.821233] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high release.
Setting pipeline to PAUSED ...
[ 161.852442] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_open
[ 161.861400] mx6s-csi 32e20000.csi_bridge: v4l_subdev: mxc_mipi-csi.0
[ 161.867790] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high request.
[ 161.874537] os02g10 3-003d: in function: os02g10_s_power
[ 161.879880] os02g10 3-003d: in function: os02g10_runtime_resume
[ 161.885849] os02g10 3-003d: in function: __os02g10_power_on
[ 161.891499] os02g10 3-003d: mux_oe_gpio - dir 1
[ 161.902085] os02g10 3-003d: pwdn_gpio - dir 1
[ 161.906463] os02g10 3-003d: mux_sel_gpio - dir 1
[ 161.916000] os02g10 3-003d: reset_gpio - dir 1
[ 161.930715] os02g10 3-003d: OS02G10_REG_SOFTWARE_RESET
[ 161.936325] m01_b_os02g10 3-003d: return value: 0
[ 161.941245] mxc_mipi-csi.0: return value: 0
[ 161.945569] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_enable
[ 161.952066] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.958303] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.965256] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.972434] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_init
[ 161.978838] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_querycap
[ 161.985842] mx6s_capture.c in function: mx6s_vidioc_enum_input
[ 161.991844] mx6s_capture.c in function: mx6s_vidioc_enum_input
[ 161.997733] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_g_std - input argument a=0x0
[ 162.006459] mxc_mipi-csi.0: return value: -515
[ 162.011021] mx6s_capture.c in function: mx6s_vidioc_g_input
[ 162.017438] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_enum_fmt_vid_cap

Pipeline is live and does not need PREROLL ...

[ 162.027911] mx6s-csi 32e20000.csi_bridge: v4l2_subdev: mxc_mipi-csi.0
Pipeline is PREROLLED ...

[ 162.035677] os02g10 3-003d: in function: os02g10_enum_mbus_code

Setting pipeline to PLAYING ...

[ 162.043934] m01_b_os02g10 3-003d: return value: 0
New clock: GstSystemClock

[ 162.051586] mxc_mipi-csi 32e30000.mipi_csi: supported format0 by mipi-csi driver: 0x2008
[ 162.051590] mxc_mipi-csi 32e30000.mipi_csi: supported format1 by mipi-csi driver: 0x2007
[ 162.062267] mxc_mipi-csi 32e30000.mipi_csi: supported format2 by mipi-csi driver: 0x3001
[ 162.070637] mxc_mipi-csi 32e30000.mipi_csi: supported format3 by mipi-csi driver: 0x3007
[ 162.079002] mxc_mipi-csi.0: return value: 0
[ 162.091640] mx6s_captuere.c in function format_by_mbus
[ 162.096794] mx6s_captuere.c: mbus_code 3007
[ 162.101029] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_enum_fmt_vid_cap
[ 162.108713] mx6s-csi 32e20000.csi_bridge: v4l2_subdev: mxc_mipi-csi.0
[ 162.115179] os02g10 3-003d: in function: os02g10_enum_mbus_code
[ 162.121121] os02g10 3-003d: invalid argument
[ 162.125414] m01_b_os02g10 3-003d: return value: -22
[ 162.130395] mxc_mipi-csi.0: return value: -22
[ 162.134859] mx6s-csi 32e20000.csi_bridge: No more fmt
[ 162.139961] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_g_pixelaspect
[ 162.147408] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_g_selection
[ 162.154652] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_g_selection
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.

[ 162.162785] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_close
[ 162.171658] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 162.180697] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface

Additional debug info:../git/libs/gstine0/GstV4l2Src:v4l2src0:
/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipel

[ 162.190366] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 162.200397] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_disable
[ 162.209794] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_deinit
[ 162.219209] os02g10 3-003d: in function: os02g10_s_power
[ 162.227400] os02g10 3-003d: in function: os02g10_runtime_suspendd (-4)

streaming stopped, reason not-negotiate
Execution ended after 0:00:00.126697071
Setting pipeline to NULL ...

[ 162.228012] m01_b_os02g10 3-003d: return value: 0
[ 162.236290] os02g10 3-003d: in function: __os02g10_power_off
[ 162.243926] mxc_mipi-csi 32e30000.mipi_csi: in function: mipi_csis_stop_stream
Freeing pipeline ...
[ 162.243947] mxc_mipi-csi.0: return value: 0
[ 162.243957] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high release.
root@abcd:~#

So the main issue is connected with (short version):

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.136944375

Any hints, any help ?
What is wrong ?

Re: gst-launch-1.0 returns: Internal data stream error

joanxie — Fri, 07 Oct 2022 04:56:58 GMT

what bsp version do you use? did you change any source code? current bsp shouldn't support raw10 data as default

Re: gst-launch-1.0 returns: Internal data stream error

btarnowski — Fri, 07 Oct 2022 08:29:01 GMT

Linux version 5.10.72-lts-5.10.y+gbea7af8cc939

and I added format definition for RAW10 :

static struct mx6s_fmt formats[] = { { .name = "UYVY-16", .fourcc = V4L2_PIX_FMT_UYVY, .pixelformat = V4L2_PIX_FMT_UYVY, .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8, .bpp = 2, }, { .name = "YUYV-16", .fourcc = V4L2_PIX_FMT_YUYV, .pixelformat = V4L2_PIX_FMT_YUYV, .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8, .bpp = 2, }, { .name = "YUV32 (X-Y-U-V)", .fourcc = V4L2_PIX_FMT_YUV32, .pixelformat = V4L2_PIX_FMT_YUV32, .mbus_code = MEDIA_BUS_FMT_AYUV8_1X32, .bpp = 4, }, { .name = "RAWRGB8 (SBGGR8)", .fourcc = V4L2_PIX_FMT_SBGGR8, .pixelformat = V4L2_PIX_FMT_SBGGR8, .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8, .bpp = 1, + }, { + .name = "RAWRGB10 (SBGGR10)", + .fourcc = V4L2_PIX_FMT_SBGGR10, + .pixelformat = V4L2_PIX_FMT_SBGGR10, + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, + .bpp = 2, } };

and here

@@ -322,6 +322,10 @@ static const struct csis_pix_format mipi_csis_formats[] = { .code = MEDIA_BUS_FMT_SBGGR8_1X8, .fmt_reg = MIPI_CSIS_ISPCFG_FMT_RAW8, .data_alignment = 8, + }, { + .code = MEDIA_BUS_FMT_SBGGR10_1X10, + .fmt_reg = MIPI_CSIS_ISPCFG_FMT_RAW10, + .data_alignment = 16, } };

and also I changed resolution in function:

static void csi_init_interface(struct mx6s_csi_dev *csi_dev)

imag_para = (1080 << 16) | 1920;

Re: gst-launch-1.0 returns: Internal data stream error

btarnowski — Fri, 07 Oct 2022 09:37:22 GMT

The complete replay above.
Linux version 5.10.72-lts-5.10.y+gbea7af8cc939

Re: gst-launch-1.0 returns: Internal data stream error

joanxie — Thu, 13 Oct 2022 08:27:34 GMT

confirmed that jpegenc doesn't support bggr as input

Re: gst-launch-1.0 returns: Internal data stream error

Udhay_ — Thu, 22 Dec 2022 23:49:25 GMT

can you able to stream using RAW10?

Re: gst-launch-1.0 returns: Internal data stream error

btarnowski — Thu, 29 Dec 2022 18:08:14 GMT

RAW10 works for IMX8MM with gst, example below.
gst-launch-1.0 -vvv v4l2src device=/dev/video0 num-buffers=1 \! "video/x-bayer,width=1920,height=1080,framerate=(fraction)25/1,format=(string)bggr" \! bayer2rgb ! pngenc ! filesink location=test.png

Thera are patches and bbappned for yocto.

Manifest for Yocto :

"meta-freescale": { "branch": "hardknott", "rev": "469d6c958c76ea235b3d3c1527e797ce3a7392e3", "url": "https://github.com/Freescale/meta-freescale" }, "meta-freescale-3rdparty": { "branch": "hardknott", "rev": "f8150f3b37cb83cba1f9e2378e57bb63e02d4610", "url": "https://github.com/Freescale/meta-freescale-3rdparty" }, "meta-freescale-distro": { "branch": "hardknott", "rev": "e6daa26ba1f748326546063d63a085ae671827d9", "url": "https://github.com/Freescale/meta-freescale-distro" }, "meta-imx": { "branch": "hardknott-5.10.72-2.2.0", "rev": "refs/tags/rel_imx_5.10.72_2.2.0", "url": "https://source.codeaurora.org/external/imx/meta-imx" }, "meta-openembedded": { "branch": "hardknott", "rev": "f44e1a2b575826e88b8cb2725e54a7c5d29cf94a", "url": "https://github.com/openembedded/meta-openembedded" }, "poky": { "branch": "hardknott", "rev": "269265c00091fa65f93de6cad32bf24f1e7f72a3", "url": "https://git.yoctoproject.org/git/poky" }

Build Configuration: BB_VERSION = "1.50.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "ubuntu-20.04" TARGET_SYS = "aarch64-poky-linux" MACHINE = "imx8mmevk" DISTRO_VERSION = "5.10-hardknott" TUNE_FEATURES = "aarch64 armv8a crc cortexa53 crypto" TARGET_FPU = "" meta-poky = "HEAD:269265c00091fa65f93de6cad32bf24f1e7f72a3" meta-oe meta-multimedia meta-python = "HEAD:f44e1a2b575826e88b8cb2725e54a7c5d29cf94a" meta-freescale = "HEAD:469d6c958c76ea235b3d3c1527e797ce3a7392e3" meta-freescale-3rdparty = "HEAD:f8150f3b37cb83cba1f9e2378e57bb63e02d4610" meta-freescale-distro = "HEAD:e6daa26ba1f748326546063d63a085ae671827d9" meta-bsp meta-sdk meta-ml = "HEAD:a38040cc159f0ef9fc177358932c2c75257c59e9" meta-gnome meta-networking meta-filesystems = "HEAD:f44e1a2b575826e88b8cb2725e54a7c5d29cf94a"