Use kmssink with elcdif on IMX8MQ processor

[jemish_1990]

Hi All,

We are working product based on IMX8MQ (MIMX8MQ6DVAJZAA) processor.

• In our board we are using two displays.
• We are using LCD via eLCDIF + MIPI and HDMI via DCSS.
• In our case currently we are rendering video on LCD using glimagesink and HDMI using kmssink.
• We display preview captured via camera on LCD. In that usecase we observed some horizontal flickering on LCD screen.
• To debug this issue we use LCD using DCSS (which is driven by kmssink). And we don't see issue.
• In imx8mq release note I see that kmssink is used only with DCSS and glimagesink with eLCDIF.

Is there any limitation that we can't use kmssink with eLCDIF?

Is there any way I can mitigate this limitation? Is it possible to use kmssink with eLCDIF?

Thanks,

Jemish

[joanxie]

why don't you use glimagesink or waylandsink to replace kmssink? I suggest that you can use glimagesink

[jemish_1990]

Hi Joanxie

I have already written in my question that we are using "glimagesink" for LCD. But with that we observed some horizontal flickering on LCD screen. Attaching video for your reference.

But we drive LCD via DCSS using kmsink we don't see these artifacts.

If there is any limitation that we can't use kmssink with eLCDIF then it's fine  but I want to know the reason behind it. Why can't we use kmssink with eLCDIF?

Thanks,

Jemish

[jemish_1990]

Hi NXP team,

Any update on this?

Thanks,

Jemish

[gary_bisson]

Hi,

There's nothing preventing you from using kmssink SW-wise on eLCDIF.

We've been supporting kmssink on our boards and it works fine. Couple of things to note:

- drm driver name is mxsfb-drm, you might need to specify it to the kmssink as parameter

- this kms driver doesn't support scaling, unlike DCSS, so you also might need to specify 'can-scale=false'

Regards,

Gary

[jemish_1990]

Hi Gary,

Thanks for you input. I tried your suggestion on same day you provided. And replied back on that day. Something went wrong on this thread. And somehow I don't see my reply so today I am posting again.

Unfortunately your suggestion didn't work for me and I am still getting following error. 

# gst-launch-1.0 filesrc location=animation-1.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! kmssink driver-name="mxsfb-drm" can-scale=false
Setting pipeline to PAUSED ...
====== VPUDEC: 4.4.5 build on Jul 9 2020 09:54:35. ======
wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Jul 9 2020 09:54:03)
vpulib: 1.1.1
firmware: 1.1.1.65535
Pipeline is PREROLLING ...

====== AIUR: 4.4.5 build on Jul 9 2020 09:54:29. ======
Core: MPEG4PARSER_06.16.01 build on Dec 11 2018 03:04:17
file: /usr/lib/lib_mp4_parser_arm_elinux.so
------------------------
Track 00 [video_0] Enabled
Duration: 0:00:03.500000000
Language: eng
Mime:
video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)avc, width=(int)1280, height=(int)800, framerate=(fraction)24/1, codec_data=(buffer)01640020ffe1001967640020acd9405
------------------------
------------------------
Track 01 [audio_0] Enabled
Duration: 0:00:03.568000000
Language: eng
Mime:
audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)2298, stream-format=(string)raw, codec_data=(buffer)119056e500
------------------------
ERROR: from element /GstPipeline:pipeline0/GstQueue:queue0: Internal data stream error.
Additional debug info:
gstqueue.c(1566): gst_queue_loop (): /GstPipeline:pipeline0/GstQueue:queue0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

Do you have any further suggestions?

Thanks,

Jemish

[gary_bisson · Accepted solution]

Hi,

Please try to use videotestsrc first:

gst-launch-1.0 videotestsrc ! kmssink driver-name="mxsfb-drm" can-scale=false

Is this working? If so, then it is because your video resolution doesn't match the display. As said, it cannot scale, therefore resolutions have to match or you need to scale it via another plugin.

Regards,

Gary

[jemish_1990]

Hi Gary,

Thanks for you input. Test pattern works for me with minor change in command.

gst-launch-1.0 videotestsrc ! kmssink driver-name="mxsfb-drm" can-scale=false force-modesetting=true.

Yes I am aware the eLCDIF doesn't support scaling so I kept MP4 video of same resolution as LCD (1280x800). Still I am getting same error .

ERROR: from element /GstPipeline:pipeline0/GstAiurDemux:aiurdemux0: GStreamer encountered a general stream error.

To understand error in detail I enabled log level and I am getting following message:

aiurdemux aiurdemux.c:3421:aiurdemux_push_pad_buffer: Pad audio push error type -1

basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<videoconvert0> transform could not transform video/x-raw, format=(string)NV12, width
=(int)1280, height=(int)800, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flippe
d/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)24000/1001 in anything we support

aiurdemux aiurdemux.c:3421:aiurdemux_push_pad_buffer: Pad video push error type -4

As per my understanding eLCDIF doesn't support color conversion also. That's the reason I am getting the above error. So I have to convert video in way that eLCDIF supports. Correct me if I am wrong.

Do you have any further suggestion?

Thanks,

Jemish

[gary_bisson]

Hi,

Yes you might have to do a colorspace conversion, try adding a videoconvert plugin before the kmssink one.

Regards,

Gary

[jemish_1990]

Hi Gary,

Sorry for late reply!

Thanks a lot for your valuable inputs. We have concluded this task.

We can close this thread now.

Thanks,

Jemish

[gary_bisson]

Hi,

Glad to hear it works! Please make sure to mark the answer that resolved the issue as best answer.

Thanks,

Gary

[joanxie]

what bsp version do you use? try to use the latest 5.4.24 version, I tested kmssink successfully just now

[jemish_1990]

Hi Gary,

Thanks for you input. I tried command as you suggested but it still failed to start video and gives an error.

I am pasting log for command.Do you have further suggestion?

gst-launch-1.0 filesrc location=animation-1.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! kmssink driver-name="mxsfb-drm" can-scale=false
Setting pipeline to PAUSED ...
====== VPUDEC: 4.4.5 build on Jul 9 2020 09:54:35. ======
wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Jul 9 2020 09:54:03)
vpulib: 1.1.1
firmware: 1.1.1.65535
Pipeline is PREROLLING ...

====== AIUR: 4.4.5 build on Jul 9 2020 09:54:29. ======
Core: MPEG4PARSER_06.16.01 build on Dec 11 2018 03:04:17
file: /usr/lib/lib_mp4_parser_arm_elinux.so
------------------------
Track 00 [video_0] Enabled
Duration: 0:00:03.500000000
Language: eng
Mime:
video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)avc, width=(int)1280, height=(int)800, framerate=(fraction)24/1, codec_data=(buffer)01640020ffe1001967640020acd9405
------------------------
------------------------
Track 01 [audio_0] Enabled
Duration: 0:00:03.568000000
Language: eng
Mime:
audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)2298, stream-format=(string)raw, codec_data=(buffer)119056e500
------------------------
ERROR: from element /GstPipeline:pipeline0/GstQueue:queue0: Internal data stream error.
Additional debug info:
gstqueue.c(1566): gst_queue_loop (): /GstPipeline:pipeline0/GstQueue:queue0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

Thanks,

Jemish