h.265 decoding on i.MX8MP

wellu · ‎08-12-2025

We have two boards, one with i.MX8MP and another with i.MX95 and we try to decode h.265 encoded RTP stream with GStreamer pipeline.

MX95 based decoding works and we can see the video. Same pipeline with MX8MP stalls and video is not shown at all. Pipeline seems to hang forever waiting for something to happen. We have pinpointed the issue to the actual h.265 decoding part of the pipeline.

So the question is that what are the differences between MX8MP and MX95 capabilities regarding h.265 video decoding? The actual RTP stream decoding works and we use the same exact videostream and pipeline on both setups.

We are using v4l2h265dec on both boards but tried using vpudec too on MX8MP with similar results.

We have not yet analysed the actual video stream further.

Also, it is a bit unclear whether we should use v4l2h265dec or vpudec on MX8MP. There seems to be mixed instructions out there. Based on tests on some other videos vpudec is not so good as v4l2h265dec

What are the limits regarding eg. resolution, framerate, profiles and versions for hardware accelerated h.265 video decoding on i.MX8MP?

Based on the datasheet MX8MP should handle 1080p60 HEVC/H.265 Main, Main 10 (up to level 5.1)

Zhiming_Liu · ‎08-12-2025

Hi @wellu

Can you share a short h.265 video for testing?

I tested h265 decoding on EVK with L6.12.20, the display is normal.

oot@imx8mpevk:~# $GSTL filesrc location=Big_Buck_Bunny_1080_10s_30MB.mp4 typefind=true ! video/quicktime ! aiurdemux ! h265parse ! queue max-size-time=0 ! v4l2h265dec ! autovideosink
Setting pipeline to PAUSED ...
 
====== V4L2DEC: 1.26.0 build on May  7 2025 08:46:20. ======
Pipeline is PREROLLING ...
 
====== AIUR: 4.10.0 build on May 10 2025 01:33:22. ======
        Core: MPEG4PARSER_06.22.14  build on Apr 28 2025 08:50:53
file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm_elinux.so.3.2
------------------------
    Track 00 [video_0] Enabled
        Duration: 0:00:10.000000000
        Language: und
    Mime:
        video/x-h265, parsed=(boolean)true, alignment=(string)au, stream-format=(string)hev1, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, codec_data=(buffer)01216000000090000000000078f000fcfdf8f800000f04200001001840010c01ffff216000000300900000030000030078959809210001002b420101216000000300900000030000030078a003c08010e596566924caf01010000003001000000301e08022000100074401c172b4624027000107784e0105ffffffffffffff732ca2de09b51747dbbb55a4fe7fc2fc4e7832363520286275696c642031353129202d20322e362b34392d3732313933373664653432613a5b57696e646f77735d5b47434320372e332e305d5b3634206269745d20386269742b3130626974202d20482e3236352f4845564320636f646563202d20436f7079726967687420323031332d3230313820286329204d756c7469636f7265776172652c20496e63202d20687474703a2f2f783236352e6f7267202d206f7074696f6e733a2063707569643d31303530313131206672616d652d746872656164733d33206e756d612d706f6f6c733d3820777070206e6f2d706d6f6465206e6f2d706d65206e6f2d70736e72206e6f2d7373696d206c6f672d6c6576656c3d322062697464657074683d3820696e7075742d6373703d31206670733d33302f3120696e7075742d7265733d31393230783130383020696e7465726c6163653d3020746f74616c2d6672616d65733d30206c6576656c2d6964633d3020686967682d746965723d31207568642d62643d30207265663d34206e6f2d616c6c6f772d6e6f6e2d636f6e666f726d616e6365206e6f2d7265706561742d6865616465727320616e6e657862206e6f2d617564206e6f2d68726420696e666f20686173683d30206e6f2d74656d706f72616c2d6c6179657273206f70656e2d676f70206d696e2d6b6579696e743d3235206b6579696e743d32353020676f702d6c6f6f6b61686561643d3020626672616d65733d3420622d61646170743d3220622d707972616d696420626672616d652d626961733d302072632d6c6f6f6b61686561643d3235206c6f6f6b61686561642d736c696365733d34207363656e656375743d3430207261646c3d30206e6f2d696e7472612d72656672657368206374753d3634206d696e2d63752d73697a653d382072656374206e6f2d616d70206d61782d74752d73697a653d33322074752d696e7465722d64657074683d312074752d696e7472612d64657074683d31206c696d69742d74753d302072646f712d6c6576656c3d322064796e616d69632d72643d302e3030206e6f2d7373696d2d7264207369676e68696465206e6f2d74736b6970206e722d696e7472613d30206e722d696e7465723d30206e6f2d636f6e73747261696e65642d696e747261207374726f6e672d696e7472612d736d6f6f7468696e67206d61782d6d657267653d33206c696d69742d726566733d33206c696d69742d6d6f646573206d653d33207375626d653d33206d6572616e67653d35372074656d706f72616c2d6d76702077656967687470206e6f2d77656967687462206e6f2d616e616c797a652d7372632d70696373206465626c6f636b3d303a302073616f206e6f2d73616f2d6e6f6e2d6465626c6f636b2072643d34206e6f2d6561726c792d736b69702072736b6970206e6f2d666173742d696e747261206e6f2d74736b69702d66617374206e6f2d63752d6c6f73736c657373206e6f2d622d696e747261206e6f2d73706c697472642d736b697020726470656e616c74793d30207073792d72643d322e3030207073792d72646f713d312e3030206e6f2d72642d726566696e65206e6f2d6c6f73736c65737320636271706f6666733d3020637271706f6666733d302072633d61627220626974726174653d32353932302071636f6d703d302e3630207170737465703d342073746174732d77726974653d302073746174732d726561643d30206970726174696f3d312e3430207062726174696f3d312e33302061712d6d6f64653d312061712d737472656e6774683d312e303020637574726565207a6f6e652d636f756e743d30206e6f2d7374726963742d6362722071672d73697a653d3332206e6f2d72632d677261696e2071706d61783d36392071706d696e3d30206e6f2d636f6e73742d766276207361723d31206f7665727363616e3d3020766964656f666f726d61743d352072616e67653d3020636f6c6f727072696d3d32207472616e736665723d3220636f6c6f726d61747269783d32206368726f6d616c6f633d3020646973706c61792d77696e646f773d30206d61782d636c6c3d302c30206d696e2d6c756d613d30206d61782d6c756d613d323535206c6f67322d6d61782d706f632d6c73623d38207675692d74696d696e672d696e666f207675692d6872642d696e666f20736c696365733d31206e6f2d6f70742d71702d707073206e6f2d6f70742d7265662d6c6973742d6c656e6774682d707073206e6f2d6d756c74692d706173732d6f70742d727073207363656e656375742d626961733d302e3035206e6f2d6f70742d63752d64656c74612d7170206e6f2d61712d6d6f74696f6e206e6f2d686472206e6f2d6864722d6f7074206e6f2d6468647231302d6f707420616e616c797369732d72657573652d6c6576656c3d35207363616c652d666163746f723d3020726566696e652d696e7472613d3020726566696e652d696e7465723d3020726566696e652d6d763d30206e6f2d6c696d69742d73616f206374752d696e666f3d30206e6f2d6c6f77706173732d64637420726566696e652d6d762d747970653d3020636f70792d7069633d3180
------------------------
Redistribute latency...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:09.934119686
Setting pipeline to NULL ...
Total showed frames (301), playing for (0:00:09.934072185), fps (30.300).
Freeing pipeline ...

Best Regards,
Zhiming

wellu · ‎08-12-2025

Hi,

unfortunately I cannot share the actual video stream as it is a network dump in .pcap format which has some confidential information in it. We are sending the contents of the file using tcpreplay command line tool to replicate the same video stream on both platforms we are evaluating.

I can confirm that I can succesfully decode Big_Buck_Bunny_1080_10s_30MB.mp4 from a file source using h265parse and v4l2h265dec elements using a GStreamer pipeline so h.265 decoding generally works. It is now something in our video stream that i.MX 95 handles just fine but i.MX 8M Plus refuses to decode.

Here's the pipeline that works on i.MX 95 but does not display anything on i.MX 8M Plus. Pipeline is identical and stream too as it is replayed identically. Command simply hangs on "Redistribute latency..." messages and nothing gets displayed. Other end is surely sending the RTP stream over network with correct IP etc.

root@imx8mp-var-dart:~# gst-launch-1.0 udpsrc port=4010 caps="application/x-rtp,media=video,encoding-name=H265,payload=96,clock-rate=90000" ! rtph265depay ! h265parse ! v4l2h265dec ! queue ! waylandsink
Setting pipeline to PAUSED ...

====== V4L2DEC: 1.24.7 build on Oct 23 2024 09:43:13. ======
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Redistribute latency...
Redistribute latency...

If I simply push frames to fakesink the pipeline seems to work ok which means we can actually get the RTP stream decoded.

root@imx8mp-var-dart:~# gst-launch-1.0 udpsrc port=4010 caps="application/x-rtp,media=video,encoding-name=H265,payload=96,clock-rate=90000" ! rtph265depay ! h265parse ! fakesink                      
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Redistribute latency...
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:09.924096959
Setting pipeline to NULL ...
Freeing pipeline ...

To me it seems that v4l2h265dec element on i.MX 8M Plus is the culprit here but there are no error messages to understand this problem further.

i.MX 95 and i.MX 8M Plus surely are different architecture and have different HW decoders so this kind of makes sense.

wellu · ‎08-13-2025

Some more tests. I can decode the RTP stream on MX8MP and create a Matroska container. This command simply decodes RTP frames and wraps h.265 stream as-is inside .mkv.

root@imx8mp-var-dart:~# gst-launch-1.0 udpsrc port=4010 caps="application/x-rtp,media=video,encoding-name=H265,payload=96,clock-rate=90000" ! rtph265depay ! h265parse ! matroskamux ! filesink location=capture.mkv 
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Redistribute latency...
Redistribute latency...
0:00:31.6 / 99:99:99.

So reading and decoding RTP works and we get a proper Matroska file. No errors on MX8MP.

Resulting file is then playable e.g on Debian workstation without errors and video is seen with proper colors and resolution. Note that on Debian we need to use vah265dec as v4l2h265dec is not available. So yes, surely a different decoder here.

# gst-launch-1.0 filesrc location=capture.mkv ! matroskademux ! h265parse ! vah265dec ! autovideosink

When I try to play this Matroska file on MX8MP I get no display.

root@imx8mp-var-dart:~# gst-launch-1.0 filesrc location=capture.mkv ! matroskademux ! h265parse ! v4l2h265dec ! waylandsink
Setting pipeline to PAUSED ...

====== V4L2DEC: 1.24.7 build on Oct 23 2024 09:43:13. ======
Pipeline is PREROLLING ...
Redistribute latency...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.942468705
Setting pipeline to NULL ...
Total showed frames (0), playing for (0:00:00.942440329), fps (0.000).
Freeing pipeline ...
root@imx8mp-var-dart:~#

I also tried vpudec in place of v4l2h265dec with same results i.e no video. In the end both elements try to pass h.265 decoding to MX8MP hardware.

ffprobe gives me this information from the capture.mkv file and h.265 stream contained in it:

Input #0, matroska,webm, from 'capture.mkv':
  Metadata:
    encoder         : GStreamer matroskamux version 1.24.7
    creation_time   : 2025-08-13T06:57:07.463061Z
  Duration: N/A, start: 6.140000, bitrate: N/A
  Stream #0:0(eng): Video: hevc (Main), yuv420p(tv, unknown/bt709/unknown), 1280x800, SAR 1:1 DAR 8:5, 30 fps, 30 tbr, 1k tbn (default)
    Metadata:
      title           : Video

So the file is playable on Debian workstation and on i.MX 95 but gives no display on i.MX 8M Plus.

So the question really is that what makes some h.265 non-playable on MX8MP that do play on MX95 and other platforms?

Zhiming_Liu · ‎08-13-2025

Hi @wellu

The i.MX8MP and i.MX95 has different VPU design and driver. Please try to enable GST debug to check if there is any useful log. If you can share the .mkv file, this will be helpful. As if we can't reproduce this issue, it's hard to give some useful suggestion or solution.

Best Regards,
Zhiming

wellu · ‎08-13-2025

Can I send you a link privately for the .mkv file?

Zhiming_Liu · ‎08-13-2025

Yes, of course, please share it in private message.

wellu · ‎08-14-2025

Any news about the issue?

Zhiming_Liu · ‎08-15-2025

Hi,

The i.MX8MP and i.MX95 have different VPUs and drivers, so their compatibility with code stream issues varies.

Here is error log from ffprobe: Could not find ref with POC -362

ffprobe -i capture3.mkv -show_streams -show_frames -loglevel debug 2> 2.txt

AVFormatContext @ 0x13f004a60] Opening 'capture3.mkv' for reading
[file @ 0x6000019e4150] Setting default whitelist 'file,crypto,data'
[matroska,webm @ 0x13f004a60] Format matroska,webm probed with size=2048 and score=100
st:0 removing common factor 1000000 from timebase
[matroska,webm @ 0x13f004a60] Before avformat_find_stream_info() pos: 576 bytes read:32768 seeks:0 nb_streams:1
[hevc @ 0x13f005040] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x13f005040] Decoding VPS
[hevc @ 0x13f005040] Main profile bitstream
[hevc @ 0x13f005040] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x13f005040] Decoding SPS
[hevc @ 0x13f005040] Main profile bitstream
[hevc @ 0x13f005040] Decoding VUI
[hevc @ 0x13f005040] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x13f005040] Decoding PPS
[matroska,webm @ 0x13f004a60] All info found
[matroska,webm @ 0x13f004a60] After avformat_find_stream_info() pos: 4481 bytes read:32768 seeks:0 frames:1
Input #0, matroska,webm, from 'capture3.mkv':
  Metadata:
    encoder         : GStreamer matroskamux version 1.24.7
    creation_time   : 2025-08-13T06:29:37.529959Z
  Duration: N/A, start: 0.382000, bitrate: N/A
  Stream #0:0(eng), 1, 1/1000: Video: hevc (Main), 1 reference frame, yuv420p(tv, unknown/bt709/unknown, left), 1280x800, 0/1, SAR 1:1 DAR 8:5, 30 fps, 30 tbr, 1k tbn (default)
      Metadata:
        title           : Video
[hevc @ 0x13f005040] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x13f005040] Decoding VPS
[hevc @ 0x13f005040] Main profile bitstream
[hevc @ 0x13f005040] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x13f005040] Decoding SPS
[hevc @ 0x13f005040] Main profile bitstream
[hevc @ 0x13f005040] Decoding VUI
[hevc @ 0x13f005040] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x13f005040] Decoding PPS
Processing read interval id:0 start:N/A end:N/A
[hevc @ 0x13f005040] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x13f005040] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x13f005040] Decoding SEI
[hevc @ 0x13f005040] Skipped PREFIX SEI 6
[hevc @ 0x13f005040] Format yuv420p chosen by get_format().
[hevc @ 0x13f005040] Could not find ref with POC -362
[hevc @ 0x13f005040] Output frame with POC 0/-360.
[hevc @ 0x13f005040] Decoded frame with POC 0/-360.
[hevc @ 0x13f005040] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x13f005040] Output frame with POC 0/-358.
[hevc @ 0x13f005040] Decoded frame with POC 0/-358.

Dump the frame information, then the frames are P frames, there is no IDR frame. Can you check the rtsp source setting?

ffprobe -i capture3.mkv -show_frames -show_packets -print_format json > 1.txt

Best Regards,
Zhiming

wellu · ‎08-15-2025

Hi,

video is partial h.265 over RTP and we have no control over the sending end. It very well might be that missing IDR frames cause i.MX 8M Plus decoder to fail where as i.MX 95 and Debian workstation work. It was just wrapped inside Matroska for testing without touching (e.g recoding) the h.265 stream itself.

We just don't know the exact reason but missing IDR frames could be it. As far as I know the h.265 decoding is done in Hantro G2 on MX8MP and other platforms use very different driver code and HW indeed.

We try to get a new stream with proper IDR frames in it and test again. I'll inform you once this is done and thanks for the help so far.

Zhiming_Liu · ‎08-14-2025

Hi,

The internal team is checking this issue. This may take some time.

Best Regards,
Zhiming

wellu · ‎08-18-2025

This is still open as we are unable to pinpoint the exact reason why the file is not decodable on MX8MP but works on MX95 and e.g Debian workstation.

There are surely differences between video decoding pipeline but MX8MP should be able to decode h.265 in general.

Stream has keyframes in it but as the stream is RTP there are no guarantees that the h.265 stream starts with a keyframe. We need to be able to start midstream too. We did some experiments and modified the file so that it begins with a keyframe but the results are the same: MX8MP cannot decode the stream where as MX95 can.

For us h.265 decoding in MX8MP and MX95 is just a black box. We feed in the bytes and bytes should get decoded. In MX8MP case it just does not work and no errors can be pinpointed.

Zhiming_Liu · ‎08-18-2025

Hi @wellu
I use below command to repair .mkv you provided, then repaired stream can work on i.MX8MP.

ffmpeg -i capture3.mkv -c:v libx265 -crf 23 -preset fast -c:a copy repaired.mkv

ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, matroska,webm, from 'capture3.mkv':
  Metadata:
    encoder         : GStreamer matroskamux version 1.24.7
    creation_time   : 2025-08-13T06:29:37.529959Z
  Duration: N/A, start: 0.382000, bitrate: N/A
  Stream #0:0(eng): Video: hevc (Main), yuv420p(tv, unknown/bt709/unknown), 1280x800, SAR 1:1 DAR 8:5, 30 fps, 30 tbr, 1k tbn, 30 tbc (default)
    Metadata:
      title           : Video
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> hevc (libx265))
Press [q] to stop, [?] for help
[hevc @ 0x5f891b95e940] Could not find ref with POC -362
x265 [info]: HEVC encoder version 3.5+1-f0c1022b6
x265 [info]: build info [Linux][GCC 11.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 32 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 5 / wpp(13 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
x265 [info]: Keyframe min / max / scenecut / bias  : 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 15 / 4 / 0
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-23.0 / 0.60
x265 [info]: tools: rd=2 psy-rd=2.00 rskip mode=1 signhide tmvp fast-intra
x265 [info]: tools: strong-intra-smoothing lslices=5 deblock sao
Output #0, matroska, to 'repaired.mkv':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0(eng): Video: hevc, yuv420p(tv, unknown/bt709/unknown, progressive), 1280x800 [SAR 1:1 DAR 8:5], q=2-31, 30 fps, 1k tbn (default)
    Metadata:
      title           : Video
      encoder         : Lavc58.134.100 libx265
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=  856 fps=379 q=30.6 Lsize=     965kB time=00:00:28.43 bitrate= 278.1kbits/s speed=12.6x
video:956kB audio:0kB subtitle:0kB other streams:0kB global headers:2kB muxing overhead: 0.939328%
x265 [info]: frame I:      4, Avg QP:20.74  kb/s: 2778.42
x265 [info]: frame P:    168, Avg QP:22.44  kb/s: 738.77
x265 [info]: frame B:    684, Avg QP:30.09  kb/s: 144.64
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 0.6% 0.0% 0.0% 0.0% 99.4%

encoded 856 frames in 2.20s (389.23 fps), 273.56 kb/s, Avg QP:28.55

The difference of frames:

--- capture3.json	2025-08-19 11:39:43.045729390 +0900
+++ repaired.json	2025-08-19 11:39:28.132640482 +0900
@@ -13,7 +13,7 @@
             "coded_width": 1280,
             "coded_height": 800,
             "closed_captions": 0,
-            "has_b_frames": 0,
+            "has_b_frames": 2,
             "sample_aspect_ratio": "1:1",
             "display_aspect_ratio": "8:5",
             "pix_fmt": "yuv420p",
@@ -21,12 +21,13 @@
             "color_range": "tv",
             "color_primaries": "bt709",
             "chroma_location": "left",
+            "field_order": "progressive",
             "refs": 1,
             "r_frame_rate": "30/1",
             "avg_frame_rate": "30/1",
             "time_base": "1/1000",
-            "start_pts": 382,
-            "start_time": "0.382000",
+            "start_pts": 0,
+            "start_time": "0.000000",
             "disposition": {
                 "default": 1,
                 "dub": 0,
@@ -43,22 +44,25 @@
             },
             "tags": {
                 "language": "eng",
-                "title": "Video"
+                "title": "Video",
+                "ENCODER": "Lavc58.134.100 libx265",
+                "DURATION": "00:00:28.533000000"
             }
         }
     ],
     "format": {
-        "filename": "capture3.mkv",
+        "filename": "repaired.mkv",
         "nb_streams": 1,
         "nb_programs": 0,
         "format_name": "matroska,webm",
         "format_long_name": "Matroska / WebM",
-        "start_time": "0.382000",
-        "size": "3320178",
+        "start_time": "0.000000",
+        "duration": "28.533000",
+        "size": "988301",
+        "bit_rate": "277096",
         "probe_score": 100,
         "tags": {
-            "encoder": "GStreamer matroskamux version 1.24.7",
-            "creation_time": "2025-08-13T06:29:37.529959Z"
+            "ENCODER": "Lavf58.76.100"
         }
     }
 }

The core error log with capture3.mkv should be this :

codecparsers_h265 gsth265parser.c:1578:gst_h265_parser_identify_nalu_hevc: Can't parse, buffer has too small size 3959, offset 3959

Best Regards,
Zhiming

wellu · ‎08-18-2025

We can re-encode and "fix" the stream with e.g ffmpeg but then it is not the same stream again. As said the original stream opens e.g using i.MX95, Debian workstation, it opens up in Mac, different browsers and so on but fails to open on MX8MP.

Stream comes over RTP from 3rd party camera source and MX8MP needs to decode it. Now it fails and we don't know the real problem. There might be something we can adjust in the sending end but complete re-encoding is not possible as the camera is live source and minimal latency is needed. Camera source has h.265 encoder in it which compresses the videostream and MX8MP should decode it.

We can play other h.265 videos just fine on MX8MP.

Now, I think the problem is how MX8MP's Hantro G2 handles h.265 decompression but we are not sure.

Zhiming_Liu · ‎08-19-2025

Hi @wellu

I opened the capture.mkv with VLC/Windows Media Player in x86 Windows11 PC, it also can't be played.

Different platforms have different levels of tolerance for errors in streaming. The fact that the same issue occurs on Windows 11 PCs indicates that the primary issue lies within the RTSP stream itself, rather than in the specific hardware platform.

Is that possible to adjust the camera encoder side?

Best Regards,
Zhiming

wellu · ‎08-19-2025

We can most likely do some changes in the sending end but I was hoping we could pinpoint the exact problem in the stream so we could simply adjust that setting. Now we just know it does not work but the exact problem is unknown.

For example, re-encoding the stream with ffmpeg means that at least ffmpeg is able to decode the stream. Also MX95 was able to decode it just fine with GStreamer pipeline. I can decode and show it in Debian using GStreamer too. Matroska file also plays inline in Mac Chrome browser.

So yes, there is something in the stream that makes it unusable in MX8. Different platforms have different decoders and in MX8MP it is Hantro G2 that is responsible for h.265 decoding.

Thanks for the pointers, I really appreciate the debugging from NXP side.

Zhiming_Liu · ‎08-19-2025

Hi,

Is that possible to get the camera stream from beginning? Although I don't think the POC error is a core error, having a stream from the beginning for debugging would reduce a lot of other interference.

Best Regards,
Zhiming

wellu · ‎08-19-2025

RTP stream might be ongoing when MX8MP starts decoding so we might skip the real start. However, this is the nature of RTP streams and any kind of live streaming. We tested this also on MX95 and it had no problems decoding the stream even if it would miss the real start.

Zhiming_Liu · ‎08-20-2025

Hi @wellu

The issue could from NALU size, as the error log about each frame is about NALU size.

And i compared the NALU size between capture2.mkv and repaired.mkv. The NALU in repaired frame is smaller than original frame. So you could try to adjust camera setting to decrease NALU size, like adjust GOP size or using lower bitrate.

Best Regards,
Zhiming