gst-launch-1.0 returns: Internal data stream error

[btarnowski]

Hi all. I'm working with imx8mm and camera sensor os02g10.
I added the Rockchip driver for this sensor as a patch.
It supports only RAW10 pixelformat. I can capture data from the sensor with command:

v4l2-ctl -d /dev/video0 --verbose --set-fmt-video=width=1920,height=1080,pixelformat=BG10 --stream-mmap --stream-count=1 --stream-to=raw10.raw

The output file needs to be processed to get correct size and raw natural green image (before White Balancing). It has to be done with PC computer with using this command:

gst-launch-1.0 -v filesrc location=path_to_file_from_camera_sensor.raw blocksize=42923008 ! "video/x-bayer,format=bggr,width=1920,height=1080,framerate=1/1" ! bayer2rgb ! videoconvert ! avenc_mjpeg ! filesink location=frame.jpeg

So the data from camera sensor looks correct but the target device (not PC) has issue with gst-launch-1.0. The goal is to get this command to work:

gst-launch-1.0 v4l2src num-buffers=1 ! jpegenc ! filesink location=picture.jpg

There is a my log for this command (patch added to have additional messages):

root@abcd:~# gst-launch-1.0 v4l2src num-buffers=1 ! jpegenc ! filesink location=/tmp/test.jpg
[ 161.334441] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_open
[ 161.340830] mx6s-csi 32e20000.csi_bridge: v4l_subdev: mxc_mipi-csi.0
[ 161.347230] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high request.
[ 161.353969] os02g10 3-003d: in function: os02g10_s_power
[ 161.359313] os02g10 3-003d: in function: os02g10_runtime_resume
[ 161.365272] os02g10 3-003d: in function: __os02g10_power_on
[ 161.370922] os02g10 3-003d: mux_oe_gpio - dir 1
[ 161.381504] os02g10 3-003d: pwdn_gpio - dir 1
[ 161.386003] os02g10 3-003d: mux_sel_gpio - dir 1
[ 161.395694] os02g10 3-003d: reset_gpio - dir 1
[ 161.410247] os02g10 3-003d: OS02G10_REG_SOFTWARE_RESET
[ 161.415915] m01_b_os02g10 3-003d: return value: 0
[ 161.420762] mxc_mipi-csi.0: return value: 0
[ 161.425074] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_enable
[ 161.431531] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.437764] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.444582] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.451750] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_init
[ 161.458176] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_querycap
[ 161.465522] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_close
[ 161.472011] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.478216] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.485041] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.492207] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_disable
[ 161.498760] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_deinit
[ 161.505330] os02g10 3-003d: in function: os02g10_s_power
[ 161.510686] os02g10 3-003d: in function: os02g10_runtime_suspend
[ 161.511644] m01_b_os02g10 3-003d: return value: 0
[ 161.516723] os02g10 3-003d: in function: __os02g10_power_off
[ 161.521523] mxc_mipi-csi 32e30000.mipi_csi: in function: mipi_csis_stop_stream
[ 161.527212] mxc_mipi-csi.0: return value: 0
[ 161.538735] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high release.
[ 161.617586] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_open
[ 161.623974] mx6s-csi 32e20000.csi_bridge: v4l_subdev: mxc_mipi-csi.0
[ 161.630373] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high request.
[ 161.637110] os02g10 3-003d: in function: os02g10_s_power
[ 161.642447] os02g10 3-003d: in function: os02g10_runtime_resume
[ 161.648403] os02g10 3-003d: in function: __os02g10_power_on
[ 161.654048] os02g10 3-003d: mux_oe_gpio - dir 1
[ 161.664367] os02g10 3-003d: pwdn_gpio - dir 1
[ 161.668752] os02g10 3-003d: mux_sel_gpio - dir 1
[ 161.677952] os02g10 3-003d: reset_gpio - dir 1
[ 161.692776] os02g10 3-003d: OS02G10_REG_SOFTWARE_RESET
[ 161.698437] m01_b_os02g10 3-003d: return value: 0
[ 161.703283] mxc_mipi-csi.0: return value: 0
[ 161.707602] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_enable
[ 161.714064] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.720266] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.727074] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.734232] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_init
[ 161.740647] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_querycap
[ 161.748024] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_close
[ 161.754493] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.760696] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.767518] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.774690] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_disable
[ 161.781247] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_deinit
[ 161.787801] os02g10 3-003d: in function: os02g10_s_power
[ 161.793164] os02g10 3-003d: in function: os02g10_runtime_suspend
[ 161.794252] m01_b_os02g10 3-003d: return value: 0
[ 161.799217] os02g10 3-003d: in function: __os02g10_power_off
[ 161.804027] mxc_mipi-csi 32e30000.mipi_csi: in function: mipi_csis_stop_stream
[ 161.809705] mxc_mipi-csi.0: return value: 0
[ 161.821233] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high release.
Setting pipeline to PAUSED ...
[ 161.852442] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_open
[ 161.861400] mx6s-csi 32e20000.csi_bridge: v4l_subdev: mxc_mipi-csi.0
[ 161.867790] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high request.
[ 161.874537] os02g10 3-003d: in function: os02g10_s_power
[ 161.879880] os02g10 3-003d: in function: os02g10_runtime_resume
[ 161.885849] os02g10 3-003d: in function: __os02g10_power_on
[ 161.891499] os02g10 3-003d: mux_oe_gpio - dir 1
[ 161.902085] os02g10 3-003d: pwdn_gpio - dir 1
[ 161.906463] os02g10 3-003d: mux_sel_gpio - dir 1
[ 161.916000] os02g10 3-003d: reset_gpio - dir 1
[ 161.930715] os02g10 3-003d: OS02G10_REG_SOFTWARE_RESET
[ 161.936325] m01_b_os02g10 3-003d: return value: 0
[ 161.941245] mxc_mipi-csi.0: return value: 0
[ 161.945569] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_enable
[ 161.952066] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 161.958303] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface
[ 161.965256] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 161.972434] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_init
[ 161.978838] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_querycap
[ 161.985842] mx6s_capture.c in function: mx6s_vidioc_enum_input
[ 161.991844] mx6s_capture.c in function: mx6s_vidioc_enum_input
[ 161.997733] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_g_std - input argument a=0x0
[ 162.006459] mxc_mipi-csi.0: return value: -515
[ 162.011021] mx6s_capture.c in function: mx6s_vidioc_g_input
[ 162.017438] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_enum_fmt_vid_cap

Pipeline is live and does not need PREROLL ...

[ 162.027911] mx6s-csi 32e20000.csi_bridge: v4l2_subdev: mxc_mipi-csi.0
Pipeline is PREROLLED ...

[ 162.035677] os02g10 3-003d: in function: os02g10_enum_mbus_code

Setting pipeline to PLAYING ...

[ 162.043934] m01_b_os02g10 3-003d: return value: 0
New clock: GstSystemClock

[ 162.051586] mxc_mipi-csi 32e30000.mipi_csi: supported format0 by mipi-csi driver: 0x2008
[ 162.051590] mxc_mipi-csi 32e30000.mipi_csi: supported format1 by mipi-csi driver: 0x2007
[ 162.062267] mxc_mipi-csi 32e30000.mipi_csi: supported format2 by mipi-csi driver: 0x3001
[ 162.070637] mxc_mipi-csi 32e30000.mipi_csi: supported format3 by mipi-csi driver: 0x3007
[ 162.079002] mxc_mipi-csi.0: return value: 0
[ 162.091640] mx6s_captuere.c in function format_by_mbus
[ 162.096794] mx6s_captuere.c: mbus_code 3007
[ 162.101029] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_enum_fmt_vid_cap
[ 162.108713] mx6s-csi 32e20000.csi_bridge: v4l2_subdev: mxc_mipi-csi.0
[ 162.115179] os02g10 3-003d: in function: os02g10_enum_mbus_code
[ 162.121121] os02g10 3-003d: invalid argument
[ 162.125414] m01_b_os02g10 3-003d: return value: -22
[ 162.130395] mxc_mipi-csi.0: return value: -22
[ 162.134859] mx6s-csi 32e20000.csi_bridge: No more fmt
[ 162.139961] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_g_pixelaspect
[ 162.147408] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_g_selection
[ 162.154652] mx6s-csi 32e20000.csi_bridge: in function: mx6s_vidioc_g_selection
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.

[ 162.162785] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_close
[ 162.171658] mx6s-csi 32e20000.csi_bridge: in function: csihw_reset
[ 162.180697] mx6s-csi 32e20000.csi_bridge: in function: csi_init_interface

Additional debug info:../git/libs/gstine0/GstV4l2Src:v4l2src0:
/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipel

[ 162.190366] mx6s-csi 32e20000.csi_bridge: in function: csi_dmareq_rff_disable
[ 162.200397] mx6s-csi 32e20000.csi_bridge: in function: csi_clk_disable
[ 162.209794] mx6s-csi 32e20000.csi_bridge: in function: mx6s_csi_deinit
[ 162.219209] os02g10 3-003d: in function: os02g10_s_power
[ 162.227400] os02g10 3-003d: in function: os02g10_runtime_suspendd (-4)

streaming stopped, reason not-negotiate
Execution ended after 0:00:00.126697071
Setting pipeline to NULL ...

[ 162.228012] m01_b_os02g10 3-003d: return value: 0
[ 162.236290] os02g10 3-003d: in function: __os02g10_power_off
[ 162.243926] mxc_mipi-csi 32e30000.mipi_csi: in function: mipi_csis_stop_stream
Freeing pipeline ...
[ 162.243947] mxc_mipi-csi.0: return value: 0
[ 162.243957] mx6s-csi 32e20000.csi_bridge: csi v4l2 busfreq high release.
root@abcd:~#

So the main issue is connected with (short version):

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.136944375

Any hints, any help ?
What is wrong ?

[btarnowski · Accepted solution]

RAW10 works for IMX8MM with gst, example below. 
gst-launch-1.0 -vvv v4l2src device=/dev/video0 num-buffers=1 \! "video/x-bayer,width=1920,height=1080,framerate=(fraction)25/1,format=(string)bggr" \! bayer2rgb ! pngenc ! filesink location=test.png

Thera are patches and bbappned for yocto.

Manifest for Yocto :

    "meta-freescale": {
        "branch": "hardknott",
        "rev": "469d6c958c76ea235b3d3c1527e797ce3a7392e3",
        "url": "https://github.com/Freescale/meta-freescale"
    },
    "meta-freescale-3rdparty": {
        "branch": "hardknott",
        "rev": "f8150f3b37cb83cba1f9e2378e57bb63e02d4610",
        "url": "https://github.com/Freescale/meta-freescale-3rdparty"
    },
    "meta-freescale-distro": {
        "branch": "hardknott",
        "rev": "e6daa26ba1f748326546063d63a085ae671827d9",
        "url": "https://github.com/Freescale/meta-freescale-distro"
    },
    "meta-imx": {
        "branch": "hardknott-5.10.72-2.2.0",
        "rev": "refs/tags/rel_imx_5.10.72_2.2.0",
        "url": "https://source.codeaurora.org/external/imx/meta-imx"
    },
    "meta-openembedded": {
        "branch": "hardknott",
        "rev": "f44e1a2b575826e88b8cb2725e54a7c5d29cf94a",
        "url": "https://github.com/openembedded/meta-openembedded"
    },
    "poky": {
        "branch": "hardknott",
        "rev": "269265c00091fa65f93de6cad32bf24f1e7f72a3",
        "url": "https://git.yoctoproject.org/git/poky"
    }

Build Configuration:
BB_VERSION           = "1.50.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "ubuntu-20.04"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "imx8mmevk"

DISTRO_VERSION       = "5.10-hardknott"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53 crypto"
TARGET_FPU           = ""


meta-poky            = "HEAD:269265c00091fa65f93de6cad32bf24f1e7f72a3"
meta-oe              
meta-multimedia      
meta-python          = "HEAD:f44e1a2b575826e88b8cb2725e54a7c5d29cf94a"
meta-freescale       = "HEAD:469d6c958c76ea235b3d3c1527e797ce3a7392e3"
meta-freescale-3rdparty = "HEAD:f8150f3b37cb83cba1f9e2378e57bb63e02d4610"
meta-freescale-distro = "HEAD:e6daa26ba1f748326546063d63a085ae671827d9"
meta-bsp             
meta-sdk             
meta-ml              = "HEAD:a38040cc159f0ef9fc177358932c2c75257c59e9"
meta-gnome           
meta-networking      
meta-filesystems     = "HEAD:f44e1a2b575826e88b8cb2725e54a7c5d29cf94a"

[btarnowski]

Linux version 5.10.72-lts-5.10.y+gbea7af8cc939

and I added format definition for RAW10 :

 static struct mx6s_fmt formats[] = {
 	{
 		.name		= "UYVY-16",
		.fourcc		= V4L2_PIX_FMT_UYVY,
 		.pixelformat	= V4L2_PIX_FMT_UYVY,
 		.mbus_code	= MEDIA_BUS_FMT_UYVY8_2X8,
 		.bpp		= 2,
 	}, {
 		.name		= "YUYV-16",
		.fourcc		= V4L2_PIX_FMT_YUYV,
 		.pixelformat	= V4L2_PIX_FMT_YUYV,
 		.mbus_code	= MEDIA_BUS_FMT_YUYV8_2X8,
 		.bpp		= 2,
 	}, {
 		.name		= "YUV32 (X-Y-U-V)",
		.fourcc		= V4L2_PIX_FMT_YUV32,
 		.pixelformat	= V4L2_PIX_FMT_YUV32,
 		.mbus_code	= MEDIA_BUS_FMT_AYUV8_1X32,
 		.bpp		= 4,
 	}, {
 		.name		= "RAWRGB8 (SBGGR8)",
		.fourcc		= V4L2_PIX_FMT_SBGGR8,
 		.pixelformat	= V4L2_PIX_FMT_SBGGR8,
 		.mbus_code	= MEDIA_BUS_FMT_SBGGR8_1X8,
 		.bpp		= 1,
+	}, {
+		.name		= "RAWRGB10 (SBGGR10)",
+		.fourcc	        = V4L2_PIX_FMT_SBGGR10,
+		.pixelformat	= V4L2_PIX_FMT_SBGGR10,
+		.mbus_code	= MEDIA_BUS_FMT_SBGGR10_1X10,
+		.bpp		= 2,
 	}
 };

and here

@@ -322,6 +322,10 @@ static const struct csis_pix_format mipi_csis_formats[] = {
 		.code = MEDIA_BUS_FMT_SBGGR8_1X8,
 		.fmt_reg = MIPI_CSIS_ISPCFG_FMT_RAW8,
 		.data_alignment = 8,
+	}, {
+		.code = MEDIA_BUS_FMT_SBGGR10_1X10,
+		.fmt_reg = MIPI_CSIS_ISPCFG_FMT_RAW10,
+		.data_alignment = 16,
 	}
 };

and also I changed resolution in function:

static void csi_init_interface(struct mx6s_csi_dev *csi_dev)

imag_para = (1080 << 16) | 1920;

[Udhay_]

can you able to stream using RAW10?

[joanxie]

what bsp version do you use? did you change any source code? current bsp shouldn't support raw10 data as default

[btarnowski]

The complete replay above.
Linux version 5.10.72-lts-5.10.y+gbea7af8cc939

[joanxie]

confirmed that jpegenc doesn't support bggr as input