Getting Started with RTSP and AP1302 using GStreamer

[#1]

Introduction.

i.MX processors are featured in multimedia applications, usually using cameras and/or displays. There are customers interested in streaming capabilities, their environment can provide network access and that's when streaming protocols come in handy.

There are different protocols that offer distinct requirements and benefits, in this article we will explore RTSP (Real Time Streaming Protocol) which was developed for audio/video transport and works over RTP. RTSP is adopted by amongst many applications, for surveillance cameras.

We will review RTSP use cases in a flexible setup, for customers exploring client and server capabilities, using latest camera release AP1302 and libcamera pipelines.

1. Hardware Setup.

• iMX95 19x19 EVK. Power supply connected to J5, USB type-C debug cable to J31, AP1302 XRPI-CAM-MINISAS to J13, Ethernet cable to J11.
• FRDM-IMX8MPLUS. Power supply connected to J2, USB type-C debug cable to J19, HDMI cable to J16, Ethernet cable to J5.
• Ethernet cables must be connected to network router or switch that can assign them IP.
• SD/eMMC flashed with Yocto Linux Factory LF_6.12.20-2.0.0 images.

Board selection can vary depending on customer's inventory, if there are plans for different processors, please create a support ticket.

Camera setup is done as follows

2. Software setup.

It's advised to add the following line to your config file so the RTSP server plug-in of gstreamer is built and deployed.

$ nano conf/local.conf

IMAGE_INSTALL:append = " gstreamer1.0-rtsp-server"

We can confirm that our gstreamer has RTSP server capabilities with the following command.

$ gst-inspect-1.0 | grep rtsp
pcapparse: irtspparse: IRTSPParse
rtsp: rtpdec: RTP Decoder
rtsp: rtspsrc: RTSP packet receiver
rtspclientsink: rtspclientsink: RTSP RECORD client

Server will use example binaries from gstreamer, with the following patch test-uri and test-launch tools are built, the mentioned patch is included in the next release, future releases won't need it.

$ bitbake imx-image-multimedia
$ mkdir ../sources/meta-imx/meta-imx-bsp/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server/
$ cp ~/Public/0001-YOCIMX-9113-rtsp-examples-install-test-launch-and-te.patch ../sources/meta-imx/meta-imx-bsp/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server/
$ nano ../sources/meta-imx/meta-imx-bsp/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.26.0.bb

FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"

SRC_URI = "https://gstreamer.freedesktop.org/src/${PNREAL}/${PNREAL}-${PV}.tar.xz \
file://0001-rtsp-examples-install-test-launch-and-te.patch \
"

…

-Dexamples=enabled \

$ bitbake imx-image-multimedia

You can deploy them as you prefer, this document uses scp.

$ find tmp/ -iname test-launch
$ scp tmp/work/armv8a-poky-linux/gstreamer1.0-rtsp-server/1.26.0/build/examples/test-launch root@<Server IP>:~
$ scp tmp/work/armv8a-poky-linux/gstreamer1.0-rtsp-server/1.26.0/build/examples/test-uri root@<Server IP>:~

3. Streaming MP4.

In this document, we use scp to copy a downloaded MP4 from host.

$ scp NXP.MP4 root@<Server IP>:~

Server board will be the one with the camera connected, it needs to change the execution permissions of the example binaries.

root@server $ chmod a+x test-*

Start server, there are three '/' due to URL path and root path. test-uri tool is used for serving saved files.

root@server $ ./test-uri file:///root/nxp.mp4

Client server will be the one streaming in the display/monitor, start streaming MP4 file on client. playbin is used for its simplicity and solution for audio and video playback, URI is defined by gstreamer example binaries, port 8554 is designated from standard.

root@client $ SERVERIP=<Server IP>
root@client $ gst-launch-1.0 playbin uri=rtsp://$SERVERIP:8554/test

4. Stream video test.

SMPTE Color Bars will be streamed as part of the video test. Video streaming is achieved using test-launch tool, it supports gstreamer pipelines between (" - "), notice that inserts spaces and requires a stream name.

root@server $ ./test-launch "( videotestsrc ! v4l2h264enc ! rtph264pay name=pay0 pt=96 )"
root@client $ gst-launch-1.0 playbin uri=rtsp://$SERVERIP:8554/test

5. Stream AP1302 camera.

i.MX95EVK will stream frames captured by AP1302, we use a gstreamer pipeline based on libcamera library, encoded to H264 and packetizing them to RTSP with default name.

root@server $ ./test-launch "( libcamerasrc camera-name="${CAMERA0}" ! video/x-raw, width=1280, height=800, format=YUY2 ! queue ! v4l2h264enc ! rtph264pay name=pay0 pt=96 )"

FRDM-i.MX8M-Plus will stream the frames from the server URI, depacketizing and decoding the frames.

root@client $ gst-launch-1.0 rtspsrc location=rtsp://$SERVERIP:8554/test latency=100 ! rtph264depay ! h264parse ! v4l2h264dec ! autovideosink

Conclusion.

Multimedia use cases can be explored with example binaries from gstreamer rtsp server plug-ins, video test output, camera pipelines, and MP4 files, customer are encouraged to explore both client and server capabilities upon their applications.

Then explore other streaming option like HTTP, for instance. Also using the secure version RTSPS.

References.

[1] https://www.nxp.com/docs/en/user-guide/UG10163.pdf 
[2] https://imxdev.gitlab.io/tutorial/i.MX_6Q_RTSP_video_and_camera_stream/ 

Patch was provided by SW development team. Remember that the patches below are provided as is, they were developed under LF6.12.20. When using them you are subject to the record of acceptance and legal compliance from EULA agreement in your Yocto build.