Hi all,
I'm testing mfw_vpudeocder plugin on MX27 platform. But it failed to work.
VPU Version: firmware 2.2.5; libvpu: 3.3.2.
The linux kernel is 2.6.19.2. And the mfw_vpuencoder works well.
Following is the my test pipeline:
gst-launch-0.10 -v --gst-debug=mfw_vpudecoder:5 videotestsrc is-live=1 pattern=snow ! 'video/x-raw-yuv,format=(fourcc)I
420,width=352,height=288,framerate=25/1' ! mfw_vpuencoder codec-type=std_avc width=352 height=288 framerate=25 ! mfw_vpudecod
er codec-type=std_avc ! fakesink
Here I use fakesink instead of mfw_v4lsink, since I have no LCD on hand.
Or mfw_vpudecoder MUST to work with mfw_v4lsink?
There are two error message reported by the debug outputs :
1. mfw_vpudecoder mfw_gst_vpu_decoder.c:882:mfw_gst_vpudec_copy_data: >>VPU_DEC: vpu_DecGetBitstreamBuffer returned zero space do a reset
2. mfw_vpudecoder mfw_gst_vpu_decoder.c:1748:mfw_gst_vpudec_vpu_init: >>VPU_DEC: vpu_DecGetInitialInfo failed. Error code is -10 handle 104624
Anyone know what's the reason? Or how can I fix it and make mfw_vpudecoder works? Or is there anything wrong with my "libvpu.so"?
Note that if I set mfw_vpudecoder's property "loopback" to "true", no such error message is reported.
For your information, the detailed debug outputs are enclosed as follows.
0:00:01.417778948 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:180:mfw_gst_vpudec_set_property: >>VPU_DEC: codec=2 |
Setting pipeline to PAUSED ...
0:00:01.501541654 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3346:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State: Null to Ready
Product Info: i.MX27
VPU Version: firmware 2.2.5; libvpu: 3.3.2
MFW_GST_VPU_DECODER_PLUGIN build on Feb 2 2012 15:20:13.
0:00:01.550218572 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3470:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State Change 0xa for VPU returned 1
0:00:01.551200677 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3571:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State unhandled next 0xa
Product Info: i.MX27
VPU Version: firmware 2.2.5; libvpu: 3.3.2
MFW_GST_VPU_ENCODER_PLUGIN build on Feb 2 2012 15:20:16.
0:00:01.582081278 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3376:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State: Ready to Paused
0:00:01.582774887 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3470:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State Change 0x13 for VPU returned 1
0:00:01.583197217 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3571:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State unhandled next 0x13
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)352, height=(int)288, framerate=(fraction)25/1, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
0:00:01.630560076 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3459:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State: Transition to Playing new clock 2458872
0:00:01.631409023 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3470:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State Change 0x1c for VPU returned 1
0:00:01.632360451 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3571:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State unhandled next 0x1c
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)352, height=(int)288, framerate=(fraction)25/1, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)352, height=(int)288, framerate=(fraction)25/1, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)352, height=(int)288, framerate=(fraction)25/1, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)352, height=(int)288, framerate=(fraction)25/1, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:src: caps = video/x-h264, height=(int)288, width=(int)352, framerate=(fraction)25/1
0:00:01.773515940 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:759:mfw_gst_vpudec_vpu_open: >>VPU_DEC: codec=2 |
0:00:01.818350677 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:830:mfw_gst_vpudec_vpu_open: >>VPU_DEC: Setting file play mode to 0 |
0:00:01.820493910 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:1752:mfw_gst_vpudec_vpu_init: >>VPU_DEC: min buffer count= 3 |
0:00:01.821309324 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:1755:mfw_gst_vpudec_vpu_init: >>VPU_DEC: InitialInfo picWidth: 352, picHeight: 288, frameRate: 0 |
/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)352, height=(int)288, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, crop-top-by-pixel=(int)0, crop-left-by-pixel=(int)0, crop-right-by-pixel=(int)0, crop-bottom-by-pixel=(int)0, num-buffers-required=(int)5
0:00:01.919564285 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:732:mfw_gst_vpudec_FrameBufferInit: >>VPU_DEC: Use Non-DR mode. |
0:00:01.920372405 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:735:mfw_gst_vpudec_FrameBufferInit: >>VPU_DEC: buffer:0, type:FB_TYPE_HW |
0:00:01.921341503 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:735:mfw_gst_vpudec_FrameBufferInit: >>VPU_DEC: buffer:1, type:FB_TYPE_HW |
0:00:01.922318947 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:735:mfw_gst_vpudec_FrameBufferInit: >>VPU_DEC: buffer:2, type:FB_TYPE_HW |
0:00:01.923294811 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:735:mfw_gst_vpudec_FrameBufferInit: >>VPU_DEC: buffer:3, type:FB_TYPE_HW |
0:00:01.924258721 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:735:mfw_gst_vpudec_FrameBufferInit: >>VPU_DEC: buffer:4, type:FB_TYPE_HW |
0:00:01.925261653 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:1997:mfw_gst_vpudec_vpu_init: >>VPU_DEC: VPU Initialization complete |
0:00:01.998142781 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3707:mfw_gst_vpudec_setcaps: >>VPU_DEC: set framerate nu 25 de 1 |
0:00:01.999006014 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3715:mfw_gst_vpudec_setcaps: >>VPU_DEC: Time per frame 40000000 |
0:00:01.999956992 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3718:mfw_gst_vpudec_setcaps: >>VPU_DEC: Frame Rate = 25 |
0:00:02.000909473 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3720:mfw_gst_vpudec_setcaps: >>VPU_DEC: Input Width is 352 |
0:00:02.001874736 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3722:mfw_gst_vpudec_setcaps: >>VPU_DEC: Input Height is 288 |
0:00:02.002798195 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3741:mfw_gst_vpudec_setcaps: >>VPU_DEC: Assuming parser input using file play mode |
/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:sink: caps = video/x-h264, height=(int)288, width=(int)352, framerate=(fraction)25/1
0:00:02.890958722 1601 0x1194d0 ERROR | mfw_vpudecoder mfw_gst_vpu_decoder.c:882:mfw_gst_vpudec_copy_data: >>VPU_DEC: vpu_DecGetBitstreamBuffer returned zero space do a reset |
0:00:02.891782256 1601 0x1194d0 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:1354:mfw_gst_vpudec_reset: >>VPU_DEC: Reset close and reopen using same previously allocated memory |
0:00:02.892745339 1601 0x1194d0 ERROR | mfw_vpudecoder mfw_gst_vpu_decoder.c:1748:mfw_gst_vpudec_vpu_init: >>VPU_DEC: vpu_DecGetInitialInfo failed. Error code is -10 handle 104624 |
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2582): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming task paused, reason error (-5)
Execution ended after 1262809399 ns.
Setting pipeline to PAUSED ...
0:00:02.899099474 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3465:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State unhandled 0x23
0:00:02.899534587 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3470:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State Change 0x23 for VPU returned 1
0:00:02.899934436 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3479:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State: Playing to Paused
Setting pipeline to READY ...
0:00:02.903812782 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3465:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State unhandled 0x1a
/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:src: caps = NULL
/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:sink: caps = NULL
0:00:02.907580226 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3470:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State Change 0x1a for VPU returned 1
0:00:02.908171730 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3487:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State: Paused to Ready
0:00:02.908536166 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3190:mfw_gst_vpudec_cleanup: >>>>VPU_DEC: Cleanup frame started 0 |
(gst-launch-0.10:1601): GStreamer-CRITICAL **: gst_mini_object_unref: assertion `GST_IS_MINI_OBJECT (mini_object)' failed
0:00:02.941636842 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3240:mfw_gst_vpudec_cleanup: >>>>VPU_DEC: Before close frame_started=0 vpu busy 0 |
0:00:02.942492106 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3248:mfw_gst_vpudec_cleanup: >>>>VPU_DEC: After close |
0:00:02.962042256 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3310:mfw_gst_vpudec_cleanup: >>>>VPU_DEC: End Cleanup |
/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:src: caps = NULL
/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:sink: caps = NULL
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = NULL
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = NULL
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = NULL
Setting pipeline to NULL ...
0:00:03.003003835 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3465:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State unhandled 0x11
0:00:03.004044587 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3470:mfw_gst_vpudec_change_state: |
>>VPU_DEC: State Change 0x11 for VPU returned 1
>>VPU_DEC: State: Ready to Null
0:00:03.005817970 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:381:vpu_mutex_lock: >>VPU_DEC: lock mutex is NULL cnt=0 |
0:00:03.006836918 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3190:mfw_gst_vpudec_cleanup: >>>>VPU_DEC: Cleanup frame started 0 |
0:00:03.007787820 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:3310:mfw_gst_vpudec_cleanup: >>>>VPU_DEC: End Cleanup |
vpu_uninit
Freeing pipeline ...
0:00:03.025310301 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:4029:mfw_gst_vpudec_finalize: >>VPU_DEC: class finalized. |
0:00:03.026125264 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:4029:mfw_gst_vpudec_finalize: >>VPU_DEC: class finalized. |
0:00:03.111815414 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:4329:mfw_gst_vpudec_vpu_finalize: >>VPU_DEC: Destructor - final cleanup |
0:00:03.112613910 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:4338:mfw_gst_vpudec_vpu_finalize: >>VPU_DEC: Cleanup already done before destructor time |
0:00:03.113565188 1601 | 0x170b8 DEBUG | mfw_vpudecoder mfw_gst_vpu_decoder.c:4346:mfw_gst_vpudec_vpu_finalize: >>VPU_DEC: vpu instance 0x10f820 destroyed. |
Regards,
Jie
已解决! 转到解答。
seems you are using an old BSP that with the kernel is 2.6.19.2. Please try the i.MX27PDK BSP. You can download it from here: LDK_IMX27_R1_1
Thanks for your advice.
I have managed to make mfw_vpudecoder work well with LDK_IMX27_R1_1.
Now both mfw_vpuencoder and mfw_vpudecoer work well separately.
However, when I tried to run two gstreamer pipelines (one for encoding, and the other for decoding) simultaneously, the kernel outputs "VPU has already been opened." for the pipeline which is run a bit later.
In kernel, I find that the vpu driver (mxc_vpu.c) limits that only one open is allowed by keeping "open_count". Does this mean that multiple simultaneous encoder and decoder pipelines are impossible due to the kernel driver limit?
And if so, is it possible to fix it by modifying vpu driver? And how?
Any suggestion is appreciated.
As I remember you can have one encoder and one decoder instance simultaneously.
If you gst-inspect mfw_vpudecoder you will find a "loopback" parameter
You should use loopback=true
Please, let me if you was able to run the loopback test
The loopback test is OK.
But I need to run encoder and decoder simultaneously in two different threads or processes.
When I tried to do this, I got such errors.
Anyone succeeds to do this?
yes, but when you set you your decoder tread you will use loopback= true in order to tell decoder to not block VPU.
So, loopback=true is only to not block VPU for decoding. You can use it with 2 separated encoder & decoder scenario.
(as I remember I was able to do that sometime ago...)
Hi Daiane,
Although the loopback test is ok, I still got errors and failed to set up two different gstreamer pipelines (one is mfw_vpuencoder, the othrer is mfw_vpudecoder), even if I set "loopback=true" for mfw_vpudecoder.
Here is the pipeline I used to test loopback:
gst-launch-0.10 -v videotestsrc is-live=1 ! 'video/x-raw-yuv,format=(fourcc)I420,width=352,height=288,framerate=25/1' ! mfw_vpuencoder codec-type=std_avc width=352 height=288 framerate=25 ! mfw_vpudecoder codec_type=std_avc loopback=true min_latency=true ! mfw_v4lsink sync=false
Here the "loopback=true" is set and the loopback pipeline works well.
Then I tried to test two encoder/decoder instances with a PC (192.168.1.222) and a MX27 board (192.168.1.254). Here are the pipelines.
On PC (192.168.1.222):
Sender:
gst-launch -v filesrc location=./BEYOND-AMANI.mp4 ! qtdemux ! rtph264pay ! udpsink host=192.168.1.254 port=5434
Receiver:
udpsrc port=5434 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)MP4V-ES, profile-level-id=(string)2, config=(string)000001b002000001b59113000001000000012000c888800cd0b042414103, payload=(int)96, ssrc=(uint)3686574436, clock-base=(uint)1422970018, seqnum-base=(uint)63140' ! rtpmp4vdepay ! decodebin ! xvimagesink sync=false
And on MX27 (192.168.1.254)
Receiver (run in one terminal):
gst-launch-0.10 -v udpsrc port=5434 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name= (string)H264, sprop-parameter-sets=(string)\"Z0LAM6tAWh7QgAAAAwCAAAAZR4wZUA\\=\\=\\,aM48gA\\=\\=\", payload=(int)96, ssrc=(uint)2316533906, clock-base=(uint)2197594482, seqnum-base=(uint)15288' ! rtph264depay ! mfw_vpudecoder codec-type=std_avc min_latency=true loopback=true ! mfw_v4lsink
Sender (run in another terminal):
gst-launch-0.10 -v videotestsrc is-live=1 ! 'video/x-raw-yuv,format=(fourcc)I420,width=352,height=288,framerate=25/1' ! mfw_vpuencoder codec-type=0 width=352 height=288 framerate=25 loopback=false ! queue ! rtpmp4vpay ! udpsink host=192.168.1.222 port=5434
Then when I started the sender pipeline on the MX27 board after the receiver had been started, I got the following outputs:
Setting pipeline to PAUSED ...
Can't open /dev/mxc_vpu
IOSystemInit() failure.
ERROR: Pipeline doesn't want to pause.
Setting pipeline to NULL ...
Freeing pipeline ...
And as you see, the sender pipeline on MX27 failed to work although the "loopback=true" is set in the receiver pipeline.
Actually, in the vpu driver (kernel/driver/mxc/vpu/mxc_vpu.c), the vpu_open() method is defined as follows (both in MX27 ADS ltib and PDK ltib):
static int vpu_open(struct inode *inode, struct file *filp)
{
if (open_count++ == 0) {
filp->private_data = (void *)(&vpu_data);
vpu_hardware_enable();
} else {
printk(KERN_ERR "VPU has already been opened.\n");
return -EACCES;
}
return 0;
}
It seems that the vpu device (/dev/mxc_vpu) ONLY allows one open by keeping the open count with the "open_count" variable. So if the receiver has already opened the vpu device, there is no more chance for the sender pipeline to open the same vpu device for sending video stream, because the kernel will simply return "-EACCES" in this case. And this situation can't be solved by simply setting "loopback=true" property of mfw_vpudecoder.
And also on page 3-9 of the file "i.MX27 Linux Multimedia Framework User's Guide" (Document Number: 924-76396,by Freescale, 2009-11), it says that "The current i.MX27 BSP implementation does not support multiple encoder or decoder instances running at the same time. This is not a limitation in the VPU hardware but in the VPU driver."
So, both for ADS and PDK ltib for MX27, it seems that it is not possible to set up two vpu (mfw_vpuencoder and mfw_vpudecoder) pipelines without modifying the kernel vpu driver. How could it be possible to achieve it by simply setting "loopback=true" property of mfw_vpudecoder? Would you please try it on a real MX27 platform and see if it really works?
FYI, my VPU firmware version is 2.2.5 and the vpu library (libvpu) version is 3.3.2, and the kernel is 2.6.19.2. (Note the vpu_open() mtethod in kernel vpu driver of LTIB PDK is the same as the one in my kernel.)
I´m not sure I still have imx27 at office, if i find it I can try. But i will back to office only next Wednesday.
Do you have the VPU api doc? You can enter a SR and ask for this doc, i may help you to implement multi instance encoder/decoder
sorry. I don´t have.
I would suggest you to start a new discussion asking for help on implementing this.
Maybe someone could help you, and this discussion was already marked as "answered"
Message was edited by: Daiane Angolini