topic Re: Freescale's hardware accelerated h264 codec in i.MX Processors

Freescale's hardware accelerated h264 codec

rostislav — Mon, 03 Jun 2013 12:57:56 GMT

I installed the Freescale's codecs following i.MX Linux Mulimedia Framework User's Guide.

While trying to use a h264 codec we get the following error:

0:11.053 PExternalT...0x2f2be420 GStreamer mfw_gst_h264_getbuffer: >>DECODER: Error -4 in allocating the Framebuffer[0]

Could anybody point to a document that may help to interpret codec errors? What the "Error -4" means?

I printed that question in my old thread, but I have doubts because the old thread status assumed answered. That is a reason I reprinted the question starting new thread.

Re: Freescale's hardware accelerated h264 codec

LeonardoSandova — Mon, 03 Jun 2013 16:28:14 GMT

Hi Rostislav,

Seems that error comes when trying to allocate memory. Could you post the gstreamer pipeline? Also attached the produced log when the pipeline is run with the parameter '--gst-debug=*mfw*:5'

BTW, I do not know about the error number (-4), not really informative :smileysad:

Leo

Re: Freescale's hardware accelerated h264 codec

rjongbloed — Tue, 04 Jun 2013 01:25:25 GMT

This is a test script which pretty much does exactly what the application does. Substituting x264 and running on a normal PC, this pipeline does work. On the embedded system, bang!

#!/bin/sh

ENC="mfw_vpuencoder codec-type=2"

DEC="mfw_h264decoder"

#ENC="x264enc"

#DEC="ffdec_h264"

gst-launch \

"videotestsrc" ! \

"ffmpegcolorspace" ! \

"video/x-raw-yuv,format=(fourcc)I420,width=352,height=288,framerate=(fraction)90000/3000" ! \

"$ENC" ! \

"rtph264pay" "mtu=1444" "pt=107" ! \

"application/x-rtp,media=video,payload=107,clock-rate=90000,encoding-name=H264" ! \

"rtph264depay" ! \

"$DEC" ! \

"ffmpegcolorspace" ! \

"ximagesink" "display=:0.0"

and this is the output:

H264D_ARM11_02.06.01 build on Jul 12 2011 13:38:27.

MFW_GST_H264_DECODER_PLUGIN 2.0.3-1-179-e630aa8d build on Dec 26 2011 16:04:09.

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX53

VPU Version: firmware 1.4.41; libvpu: 5.3.2

MFW_GST_VPU_ENCODER_PLUGIN 2.0.3-1-179-e630aa8d build on Dec 26 2011 15:58:19.

Pipeline is PREROLLING ...

Caught SIGSEGV accessing address 0x180f

#0 0x2ad1d976 in ?? ()

#1 0x2ad31722 in ?? ()

Spinning. Please run 'gdb gst-launch 16089' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

^CCaught interrupt -- handling interrupt.

Interrupt: Stopping pipeline ...

ERROR: pipeline doesn't want to preroll.

Setting pipeline to NULL ...

the SEGV is disappointing!

Re: Freescale's hardware accelerated h264 codec

LeonardoSandova — Tue, 04 Jun 2013 16:18:51 GMT

Try:

+ ENC="vpuenc"

+ DEC="vpudec"

+ mfw_ipucsc instead of ffmpegcolorspace

+ mfw_v4lsink instead of ximagesink

Re: Freescale's hardware accelerated h264 codec

rjongbloed — Tue, 04 Jun 2013 23:30:41 GMT

Unfortunately, I get

WARNING: erroneous pipeline: no element "vpuenc"

and when I do a gst-inspect, I get:

lucid@lucid-desktop:~/projects$ gst-inspect | grep vpu
mfw_vpuencoder: mfw_vpuencoder: Freescale: Hardware (VPU) Encoder
mfw_vpudecoder: mfw_vpudecoder: Freescale: Hardware (VPU) Decoder

is there something else we should install?

Re: Freescale's hardware accelerated h264 codec

LeonardoSandova — Wed, 05 Jun 2013 16:32:20 GMT

Oh Sorry, you are on MX53. Can you try those codecs (mfw_*) ? There is nothing more you need to install. You got the HW accelerated gstreamer elements you need to use.

Leo

Re: Freescale's hardware accelerated h264 codec

rjongbloed — Thu, 06 Jun 2013 00:21:14 GMT

Thank you, that has got us a bit further. Now there seems to be a transparency issue.

Do you have any additional documentation on the mfw_v4lsink?

The Linux_Multimedia_Framework_User_Guide.pdf file I have doesn't say much about it.

Re: Freescale's hardware accelerated h264 codec

rjongbloed — Fri, 07 Jun 2013 00:51:10 GMT

I have dug into this further. Going back to the gst-launch script, there is nothing appearing on the screen and there are occasional

[WARN] VPU mutex couldn't be locked before timeout expired

messages appearing. And when I do the ^C, I get

ERROR: pipeline doesn't want to preroll.

which implies something is stuck. While I am not really an expert in gstreamer, the pipeline looks OK and does work on a normal PC, with changes to the codecs etc. So, it has to be something subtle with the mfw_* stuff. I have attached a full trace log file.

Any ideas?

Re: Freescale's hardware accelerated h264 codec

rostislav — Fri, 07 Jun 2013 12:24:58 GMT

Leonardo,

Could you please recommend some documents regarding the codecs?

Re: Freescale's hardware accelerated h264 codec

LeonardoSandova — Fri, 07 Jun 2013 15:23:12 GMT

Hi Robert,

The only information you can get comes from either the document you pointed out or typing gst-inspect to the specific element.

Re: Freescale's hardware accelerated h264 codec

LeonardoSandova — Fri, 07 Jun 2013 15:26:41 GMT

By codecs you mean freescale gstreamer elements? If yes, as I told Robert, only the Multimedia User Guide and gst-inspect from the command line are the sources of info.

leo

Re: Freescale's hardware accelerated h264 codec

LeonardoSandova — Fri, 07 Jun 2013 15:28:57 GMT

Robert, you issue will be escalated. I ran out of ideas.

Leo

Re: Freescale's hardware accelerated h264 codec

rostislav — Tue, 11 Jun 2013 15:35:15 GMT

Leonardo,

Could you please involve anybody who is related to the freescale gstreamer elements development. We need to find a solution for the problem.

Re: Freescale's hardware accelerated h264 codec

LeonardoSandova — Tue, 11 Jun 2013 15:59:34 GMT

Your issue has been pushed to an internal query and it is currently being investigate.

Some observations from my side:

1. Do you observe the same issue without the (de)pay elements? Please construct a pipeline with just filesrc, enc, dec and filesink.

2. Why do you encode then decode at the same pipeline? your board should be streaming ( encoding + rtppaying) and your host should be doing the playback (decoding + depaying) but you have everything on a single pipeline. I know on PC works fine, but I do not understand the purpose of that pipeline.

Leo

Re: Freescale's hardware accelerated h264 codec

rjongbloed — Wed, 12 Jun 2013 01:03:25 GMT

1. Yes that works.But why does the (de)pay elements work fine when using codecs on a normal x86 PC?

2. This is for test purposes only. Imagine that it is really two pipelines on two different machines. As in:

gst-launch \

"videotestsrc" ! "mfw_ipucsc" ! \

"video/x-raw-yuv,format=(fourcc)I420,width=352,height=288,framerate=(fraction)90000/3000" ! \

"mfw_vpuencoder codec-type=2" ! \

"rtph264pay" "mtu=1444" "pt=107" ! \

packets are sent via RTP on a UDP socket to another machine

gst-launch \

"application/x-rtp,media=video,payload=107,clock-rate=90000,encoding-name=H264" ! \

"rtph264depay" ! \

"mfw_vpudecoder" ! \

"mfw_ipucsc" ! \

"mfw_v4lsink"

All I did for testing was not do the packets over the wire bit.

Re: Freescale's hardware accelerated h264 codec

LeonardoSandova — Wed, 12 Jun 2013 15:26:36 GMT

For point 2, have you run those two pipelines (on separate machines)?

Leo

Re: Freescale's hardware accelerated h264 codec

rostislav — Wed, 12 Jun 2013 15:43:01 GMT

Both pipelines run on the same machine.

Re: Freescale's hardware accelerated h264 codec

rjongbloed — Thu, 13 Jun 2013 00:13:15 GMT

I only have one machine to play with, so I have not tried that mode.

However, the principle of having an encoder and and a decoder running in a consecutive pipeline rather than two parallel pipelines should be a valid test.

As it works without the (de)pay stages, I am guessing there is an issue with getting the NALU's in a format that mfw_decoder likes. Is there any information on that?

Do you actually know of anyone that has a working pipeline for receiving RTP packets?

Re: Freescale's hardware accelerated h264 codec

LeonardoSandova — Thu, 13 Jun 2013 16:07:34 GMT

Hi Robert.

Yes, the pipeline you are running is valid, that is no doubt. on MX53 I have not seen any pipelines like the ones you are seeing. On MX6, there are some pipelines I have executed but using a RTP bin element, but I do not know what issues you will face on your setup; take a look here. BTW, if you got streaming working, any other issue is stopping you to advance?

Leo

Re: Freescale's hardware accelerated h264 codec

juangutierrez — Thu, 13 Jun 2013 23:10:13 GMT

I have tried next pipelines in iMx53 and they are working fine.

These are based on Leo's examples. So, the rtppay/depay is working correctly

# SOURCE FROM PC

gst-launch-0.10 gstrtpbin name=rtpbin \

videotestsrc ! ffmpegcolorspace ! "video/x-raw-yuv,format=(fourcc)I420,width=352,height=288,framerate=(fraction)90000/3000" ! x264enc byte-stream=true ! queue ! rtph264pay ! rtpbin.send_rtp_sink_0 \

rtpbin.send_rtp_src_0 ! udpsink port=5000 host=10.112.98.90 \

rtpbin.send_rtcp_src_0 ! udpsink port=5001 sync=false async=false host=10.112.98.90 \

udpsrc port=5005 ! rtpbin.recv_rtcp_sink_0

#SINK FROM IMX53

gst-launch -v gstrtpbin name=rtpbin \

udpsrc caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=H264" buffer-size=500000 \

port=5000 ! rtpbin.recv_rtp_sink_0 \

rtpbin. ! queue max-size-buffers=1000000 ! rtph264depay ! queue max-size-buffers=1000000 ! mfw_vpudecoder ! "mfw_v4lsink" \

udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0 \

rtpbin.send_rtcp_src_0 ! udpsink port=5005 sync=false async=false