1. Enable QtMultimedia 1) In “source/meta-qt5/recipes-qt/qt5/qtmultimedia_git.bb”, add “gstreamer” at the end of “PACKAGECONFIG ??=” . For example on L3.14.52 bsp, build x11 backend: PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'alsa', 'alsa', '', d)} \ ${@bb.utils.contains('DISTRO_FEATURES', 'pulseaudio', 'pulseaudio', '', d)} \ gstreamer" 2) Build qtmultimedia: bitbake qtmultimedia 3) Copy the built package to target file system. For example : cp -a ./build-x11/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/qtmultimedia/5.5.0+gitAUTOINC+46a83d5b86-r0/image/usr/ /usr export DISPLAY=:0 Video example application located at : /usr/share/qt5/examples/multimedia/video/qmlvideo 2. Enable Qt Logging Info export QT_LOGGING_RULES=<category>[.type]=ture|false For example: Turn on all debug logging: export QT_LOGGING_RULES=*.debug=true Turn on all logging level of all multimedia module: export QT_LOGGING_RULES=qt.multimedia.*=true 3. Performance Checking Gstreamer: overlaysink fps=51.585 gst-launch-1.0 playbin uri=file:///H264_AVC_1080p_30fps_27Mbps_mp3.avi video-sink=”overlaysink sync=false” Gstreamer: glimagesink fps=43.850 gst-launch-1.0 playbin uri=file:///H264_AVC_1080p_30fps_27Mbps_mp3.avi video-sink="glimagesink sync=false" Qtmultimedia: fps=45.201 ./qmlvideo -url file:///H264_AVC_1080p_30fps_27Mbps_mp3.avi -hide-perf Current qt5 multimedia has a special video node for imx6 at "/src/plugins/videonode/imx6" which uses GPU DirectVIV to accelerate the video rendering. So the rendering performance is almost same as the glimagesink. For check the free run frame rate, the qt multimedia source need change a little bit to set sink sync as false and calculate the real video rendering rate. Set sync as false: In "/src/plugins/gstreamer/mediaplayer/qgstreamerplayersession.cpp" QGstreamerPlayerSession::playbinNotifySource(): - g_object_set(G_OBJECT(self->m_videoSink), "sync", !self->m_isLiveSource, NULL); + g_object_set(G_OBJECT(self->m_videoSink), "sync", false/!self->m_isLiveSource/, NULL); Get fps: In "/src/gsttools/qgstvideorenderersink.cpp" +#include <QTime> +gint g_frame_showed; +QTime g_time; + bool QVideoSurfaceGstDelegate::start(GstCaps *caps) … + g_frame_showed = 0; + g_time.restart(); … void QVideoSurfaceGstDelegate::stop() … + printf(">>>>> fps=%f:\n", (gdouble)g_frame_showed*1000/g_time.elapsed()); … bool QVideoSurfaceGstDelegate::handleEvent(QMutexLocker *locker) … const bool rendered = m_activeRenderer->present(m_surface, buffer); + g_frame_showed++; … Or use the attached patch: qtmultimedia_fps_check.diff 4. Streaming Playing: HTTP RTP RTSP HTTP: ./qmlvideo -url http://10.192.225.205/streaming/http/H264_HP13_1280x720_29_2850_AAC_LC_44_64_NobodyMTV.mp4 RTSP: ./qmlvideo -url rtsp://10.192.225.205/MPEG4_SP1_720x576_30fps_282_AMRNB_TVC.mp4 RTP: QT mediaplayer use gstreamer playbin as the backend, playbin don’t support rtp address as uri. So QT mediaplayer can’t support RTP playing directly. 5. Audio Recording /usr/share/qt5/examples/multimedia/audiorecorder/audiorecorder This is a widget application, not a QML application. It use QAudioRecorder as its background worker. By default it will use pulsesrc and lamemp3enc to recorder audio as mkv file at user home folder with name of clip_0001.mkv, clip_0002.mkv, and so on. If need special pipeline for recording, example use imxmp3enc to record mp3 files, we need change the function GstElement *QGstreamerCaptureSession::buildEncodeBin in “src/plugins/gstreamer/mediacapture/qgstreamercapturesession.cpp” as well as QGstreamerMediaContainerControl(QObject *parent) in “src/plugins/gstreamer/mediacapture/qgstreamermediacontainercontrol.cpp” 6. Camera Widgets Application: “/usr/share/qt5/examples/multimediawidgets/camera/camera” QML Application: “/usr/share/qt5/examples/multimedia/declarative-camera/declarative-camera” Qt Multimedia has some bugs on L3.14.52 with QCamera for recording. Use the attached patches: 0001-gstreamer-fix-camerabin-not-negotiated-error.patch 0001-QCamera-can-t-set-recording-container-format-video-a.patch 0002-QtCamera-change-default-encoding-parameters-to-use-N.patch If you want use overlaysink as the preview sink, then use following patch: 0003-use-overlaysink-as-the-preview-sink.patch Both applications need to set environment variable QT_GSTREAMER_CAMERABIN_VIDEOSRC to use imxv4l2src If camera device is other than /dev/video0, set environment variable QT_GSTREAMER_CAMERABIN_VIDEOSRC_DEVICE to the right camera device, such as "/dev/video1" , for example: export QT_GSTREAMER_CAMERABIN_VIDEOSRC=imxv4l2src export DISPLAY=:0 export QT_GSTREAMER_CAMERABIN_VIDEOSRC_DEVICE="/dev/video1" By default, the “post-preview” feature of camerabin was disabled by the patches provided, since the imxv4l2src can’t work with “post-preview” internal “videoconvert” element. To enable “post-preview” feature, one need uncomment following line //g_object_set(G_OBJECT(m_camerabin), POST_PREVIEWS_PROPERTY, TRUE, NULL); in “/qtmultimedia/src/plugins/gstreamer/camerabin/camerabinsession.cpp”, Function: CameraBinSession::CameraBinSession(GstElementFactory *sourceFactory, QObject *parent) And Change “videoconvert” to “imxvideoconvert_g2d” or “imxvideoconvert_ipu” or “imxvideoconvert_pxp” in line : csp = gst_element_factory_make ("videoconvert", "preview-vconv"); of file “/gstreamer1.0-plugins-bad/gst-libs/gst/basecamerabinsrc/gstcamerabinpreview.c” function: GstCameraBinPreviewPipelineData *gst_camerabin_create_preview_pipeline (GstElement * element, GstElement * filter)
記事全体を表示