Hello
I am new to gstreamer and in our application we need to capture the video using and transmit it through Network using I.MX6Q board.
I am having I.MX6Q SDP board which contains the MIPI and parallel camera. For start i want to capture the video using parallel camera and i want to encode (as H264 mp4 file) it using VPU and save that .mp4 file on the SD card.
I am new to the gsteamer , and i want to know that it can be possible to encode the MIPI captured data in to .h264 format and stored it in SD card as .mp4 file?
Which gstreamer plugin will be required and what is the command to perform encoding operation?
I have already build rootfs using gst-fsl-plugin and i have gst-inspect | grep mfw command and it gives me the following plugin .
isink.imx: mfw_isink: IPU-based video sink
aacdec.imx: mfw_aacdecoder: aac audio decoder
ipucsc.imx: mfw_ipucsc: IPU-based video converter
mp3dec.imx: mfw_mp3decoder: mp3 audio decoder
v4lsink.imx: mfw_v4lsink: v4l2 video sink
mp3enc.imx: mfw_mp3encoder: mp3 audio encoder
audiopeq.imx: mfw_audio_pp: audio post equalizer
mpeg2dec.imx: mfw_mpeg2decoder: mpeg2 video decoder
vorbisdec.imx: mfw_vorbisdecoder: vorbis audio decoder
amrdec.imx: mfw_amrdecoder: amr audio decoder
h264.imx: mfw_h264decoder: h264 video decoder
v4lsrc.imx: mfw_v4lsrc: v4l2 based camera src
mpeg4dec.imx: mfw_mpeg4aspdecoder: mpeg4 video decoder
Please help me.
where is the gstreamer's plugin help document ?
Thanks,
J.
There some documents on the community which can help you. As a starting point you can use this one:GStreamer i.MX6 Pipelines and for your specific need checkGStreamer i.MX6 Encoding
In the other hand, the element to do hardware encoding is called 'vpuenc' and it is not listed on the log you post. Make sure you have installed it otherwise install it like this
for cmd in prep scbuild scdeploy; do ./ltib -m $cmd -gst-fsl-plugins; done
Leo
Hi Leo,
Thanks for your answer.
I have rechecked and vpuenc and vpudec both are installed.
I have refer the link and i tried to do encoding using
# Parallel (J9 port):
modprobe ov5642_camera
modprobe mxc_v4l2_capture
gst-launch mfw_v4lsrc ! queue ! vpuenc codec=0 ! matroskamux ! filesink location=output.mkv sync=false
When i tried to copy the file on my pc from board and try to play the encoded file on my PC using VLC player and it does not play.
Actually i want to encode the .mp4 with H264 as encoder. So i want muxer as mp4. How i can achieve this? what is the gst command to encode .mp4 file ?
Regards,
J.
Hello
I want to capture the video using parallel camera available on i.MX6Q SDP board and encode it with H.264 encoder and Mp4 as demuxer. I want to store that .mp4 file on SD card.
I have tried with following gst command but it is not working.
gst-launch mfw_v4lsrc fps-n=15 capture-mode=4 ! queue ! vpuenc codec=6 ! mp4mux ! filesink location=out.mp4 sync=false
I have tried to play .mp4 file on board using below command
gst-launch playbin2 uri=file:///out.mp4
it does not display anything either on PC or on board.
SO it looks that Mp4mux is not working.
After that i have tried to encode it using matroskamux using below command.
gst-launch mfw_v4lsrc fps-n=15 capture-mode=4 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=output.mkv sync=false
and try to play it using play2bin command and it played fine on board but it fails to play on PC using VLC media player.
Can anyone help me to get the .mp4 encoding working with mp4mux pipeline?
Please help on this issue.
Thanks,
J.
Hi John,
On the PC, can you try other players? mplayer, totem are good ones, for example. Also, create an encoded file (no container)
gst-launch mfw_v4lsrc fps-n=15 capture-mode=4 ! queue ! vpuenc codec=6 ! filesink location=output.codec6 sync=false
And play it on your host. Also, set capture modes, just to validate this is not a camera error.
Leo
Hello Leo,
Thanks for ur answer. I was busy show could not check the suggestion provided by you.
Anyway i have tired to runt the command
gst-launch mfw_v4lsrc fps-n=15 capture-mode=4 ! queue ! vpuenc codec=6 ! filesink location=output.codec6 sync=false
It has capture the file and try to play file using gplay on board it only play one frame only.
Also when i tried to capture the Mp4 file using H.264 codec
gst-launch mfw_v4lsrc fps-n=15 capture-mode=4 ! queue ! vpuenc codec=6 ! mp4mux ! filesink location=out.mp4 sync=false
i have got the file out.mp4 which is not playing either using gplay on board or any other video player on PC.
i am attaching for your reference.
Please help me to get this mp4 file working with H264 as encoder.
Regards,
J.
Hi, John
I think it is one known issue, there is some compatibility issue between vpuenc/h264 and mp4mux.
because mp4mux need to encoder plugin provide config data(SPS/PPS) explicitly and it only identify avc1 format.
so you can try another mp4 mux, such as 'ffmux_mp4' instead of 'mp4mux'.
Regard
Eagle
Hi Peng, if you have a link to learn more about this issue, please post it.
Thanks
Hi,
we had the same problems on an IMX53 device.
The plugin mp4mux needs the "codec_data" from the encoder plugin to create the avcC Box in the mp4 file.
This can be achieved by setting "h264-byte-stream" to "0".
But the mfw_vpuencoder has a bug in the function, which creates the "codec_data" from the SPS/PPS data.
Apply the following patch to mfw_gst_vpu_encoder.c
This worked for us.
Regards,
Uwe
--- a/gst-fsl-plugin-2.0.3/src/video/vpu_enc/src/mfw_gst_vpu_encoder.c
+++ b/gst-fsl-plugin-2.0.3/src/video/vpu_enc/src/mfw_gst_vpu_encoder.c
@@ -82,7 +82,7 @@ static MfwGstVPU_Enc *vpuenc_global_ptr = NULL;
#define VPU_PIC_TYPE ((vpu_enc->codec == STD_AVC) ? ((vpu_enc->outputInfo->picType>>1)&0x3) : ((vpu_enc->outputInfo->picType)&0x3) )
#define VPU_PIC_TYPE_IDR ( (vpu_enc->codec == STD_AVC) ? (!(vpu_enc->outputInfo->picType&0x1)) : (VPU_PIC_TYPE == 0))
#define VPU_PIC_TYPE_I ( VPU_PIC_TYPE_IDR || (VPU_PIC_TYPE == 0))
-#define NALU_HEADER_SIZE 5
+#define NALU_HEADER_SIZE 4
#define AVC_NALU_START_CODE 0x00000001
#ifdef SWAP
@@ -1390,19 +1390,19 @@ GstBuffer *
mfw_gst_vpuenc_packetize_avchdr (guint8 * sps_data, guint sps_size,
guint8 * pps_data, guint pps_size)
{
guint data_size = 6 + (2 + (sps_size - NALU_HEADER_SIZE)) +
1 + (2 + (pps_size - NALU_HEADER_SIZE));
GstBuffer *codec_data = gst_buffer_new_and_alloc (data_size);
guint8 *data = GST_BUFFER_DATA (codec_data);
gint8 i = 0;
/* unsigned int(8) configurationVersion = 1; */
data[i++] = 1;
/* unsigned int(8) AVCProfileIndication; */
- data[i++] = sps_data[NALU_HEADER_SIZE];
- /* unsigned int(8) profile_compatibility; */
data[i++] = sps_data[NALU_HEADER_SIZE + 1];
- /* unsigned int(8) AVCLevelIndication; */
+ /* unsigned int(8) profile_compatibility; */
data[i++] = sps_data[NALU_HEADER_SIZE + 2];
+ /* unsigned int(8) AVCLevelIndication; */
+ data[i++] = sps_data[NALU_HEADER_SIZE + 3];
/* bit(6) reserved = '111111'b;
* unsigned int(2) lengthSizeMinusOne; */
data[i++] = 0xFF; /* lengthSizeMinusOne = 3 */
@@ -1411,7 +1411,7 @@ mfw_gst_vpuenc_packetize_avchdr (guint8 * sps_data, guint sps_size,
data[i++] = 0xE1; /* numOfSequenceParameterSets = 1 */
/* unsigned int(16) sequenceParameterSetLength ; */
data[i++] = 0;
data[i++] = sps_size - NALU_HEADER_SIZE;
/* bit(8*sequenceParameterSetLength) sequenceParameterSetNALUnit; */
memcpy (data + i, sps_data + NALU_HEADER_SIZE, sps_size - NALU_HEADER_SIZE);
i += sps_size - NALU_HEADER_SIZE;
is this a real bug? fsl-gst-plugin 3.0.7 still has not applied this patch, but if it is indeed a bug, FSL should be notified somehow?
Hi, Leo
I just find one similar issue discussed in : https://community.freescale.com/thread/304415
but seems not answered completely either.
Regard
Eagle.
Hello Leo,
Is there any update on this? We are still facing the issue to encode .Mp4 file using gst plugin.
Your help will be appreciated.
Regards,
J.
Hi John.
Before we escalate the problem, try
1. Specify the device property on the source element (make sure it points to your camera)
2. Change the capture-mode property, try lowering the values
3. Are you using a NFS system? if yes, try to use a SD mounted system, just to avoid any network problem.
4. Enable debug flags, just to be sure there is nothing wrong on the encoding and muxing taks (--gst-debug=*enc*:2,*mux*:2)
Just to validate your camera is working try
gst-launch mfw_v4lsrc <place the same properties here> ! mfw_v4lsink
Leo