imx6q rstp stream and h264 decoding

ddejean · ‎03-20-2015

Hi everyone,

Our team is currently trying to play a RTSP stream with H264 encoded video with a Wandboard Quad running Android and we are facing some issues.

For information our wandboard quad is running the Freescale Kitkat 4.4 android port.

We first tried to diffuse with RTSP a 1080p movie (https://peach.blender.org/trailer-page/) using gstreamer (gst-rtsp-server-1.0/examples/test-mp4 ) on a computer and read it using the mediaframework on the wandboard and it worked perfectly.

Now we're trying to read a stream coming from a VLC rtsp server which casts the computer screen:

cvlc screen:// --screen-fps=30.000000 --screen-width=1920 --screen-height=1080 --live-caching=300 '--sout=#transcode{vcodec=h264,venc=x264{profile=high,preset=ultrafast,tune=zerolatency},fps=30,acodec=none}:rtp{sdp=rtsp://:8554/display}' --sout-keep

And we read it from a small android app which uses the media framework:

mediaPlayer = new MediaPlayer();

try {

mediaPlayer.setDataSource(context, Uri.parse("rtsp://192.168.2.26:8554/display"));

mediaPlayer.setOnPreparedListener(this);

mediaPlayer.prepare();

if (surface != null) {

mediaPlayer.setSurface(surface);

}

mediaPlayer.start();

} catch (IOException e) {

e.printStackTrace();

}

And the display stays black, there's no picture. Activating traces in the media framework we get the attached traces (see below or trace-1080p.txt attached file).

Then if we force VLC to scale to 720 it seems to work a bit better: ie we get an image, we don't have so much error traces but the display freezes after some time.

Does someone have any idea why we're not able to play this ? Any input/help would be welcome

V/MediaPlayer( 2445): constructor

V/MediaPlayer( 2445): setListener

D/MediaPlayer( 2445): Couldn't open file on client side, trying server side

V/MediaPlayer( 2445): setDataSource(rtsp://192.168.2.26:8554/display)

V/IMediaDeathNotifier( 2445): getMediaPlayerService

V/MediaPlayerService( 1436): Client(2) constructor

V/MediaPlayerService( 1436): Create new client(2) from pid 2445, uid 10045,

V/MediaPlayerService( 1436): setDataSource(rtsp://192.168.2.26:8554/display)

V/MediaPlayerService( 1436): player type = 4

V/AudioSink( 1436): AudioOutput(13)

V/NuPlayer( 1436): kWhatSetAudioSink

V/NuPlayer( 1436): kWhatSetDataSource

V/MediaPlayerService( 1436): setDataSource

V/MediaPlayer( 2445): MediaPlayer::setAudioStreamType

V/MediaPlayer( 2445): setVideoSurfaceTexture

V/MediaPlayerService( 1436): [2] setVideoSurfaceTexture(0x0)

V/MediaPlayer( 2445): prepare

V/MediaPlayerService( 1436): [2] setAudioStreamType(3)

V/MediaPlayerService( 1436): [2] prepareAsync

V/MediaPlayerService( 1436): [2] notify (0x2c61dde0, 200, 701, 0)

V/MediaPlayer( 2445): message received msg=200, ext1=701, ext2=0

W/MediaPlayer( 2445): info/warning (701, 0)

V/MediaPlayer( 2445): callback application

V/MediaPlayer( 2445): back from callback

I/MyHandler( 1436): connection request completed with result 0 (Success)

V/ARTSPConnection( 1436): request: 'DESCRIBE rtsp://192.168.2.26:8554/display RTSP/1.0

V/ARTSPConnection( 1436): Accept: application/sdp

V/ARTSPConnection( 1436): User-Agent: stagefright/1.2 (Linux;Android 4.4.2)

V/ARTSPConnection( 1436): CSeq: 1

V/ARTSPConnection( 1436):

V/ARTSPConnection( 1436): '

I/ARTSPConnection( 1436): status: RTSP/1.0 200 OK

V/ARTSPConnection( 1436): line: 'Server: VLC/2.0.3'

V/ARTSPConnection( 1436): line: 'Date: Fri, 20 Mar 2015 12:31:42 GMT'

V/ARTSPConnection( 1436): line: 'Content-Type: application/sdp'

V/ARTSPConnection( 1436): line: 'Content-Base: rtsp://192.168.2.26:8554/display'

V/ARTSPConnection( 1436): line: 'Content-Length: 456'

V/ARTSPConnection( 1436): line: 'Cache-Control: no-cache'

V/ARTSPConnection( 1436): line: 'Cseq: 1'

I/MyHandler( 1436): DESCRIBE completed with result 0 (Success)

I/ASessionDescription( 1436): v=0

I/ASessionDescription( 1436): o=- 15615828936265859616 15615828936265859616 IN IP4 ddejean-dev1

V/ASessionDescription( 1436): adding 'o=' => '- 15615828936265859616 15615828936265859616 IN IP4 ddejean-dev1'

I/ASessionDescription( 1436): s=Unnamed

V/ASessionDescription( 1436): adding 's=' => 'Unnamed'

I/ASessionDescription( 1436): i=N/A

V/ASessionDescription( 1436): adding 'i=' => 'N/A'

I/ASessionDescription( 1436): c=IN IP4 0.0.0.0

V/ASessionDescription( 1436): adding 'c=' => 'IN IP4 0.0.0.0'

I/ASessionDescription( 1436): t=0 0

V/ASessionDescription( 1436): adding 't=' => '0 0'

I/ASessionDescription( 1436): a=tool:vlc 2.0.3

V/ASessionDescription( 1436): adding 'a=tool' => 'vlc 2.0.3'

I/ASessionDescription( 1436): a=recvonly

V/ASessionDescription( 1436): adding 'a=recvonly' => '(null)'

I/ASessionDescription( 1436): a=type:broadcast

V/ASessionDescription( 1436): adding 'a=type' => 'broadcast'

I/ASessionDescription( 1436): a=charset:UTF-8

V/ASessionDescription( 1436): adding 'a=charset' => 'UTF-8'

I/ASessionDescription( 1436): a=control:rtsp://192.168.2.26:8554/display

V/ASessionDescription( 1436): adding 'a=control' => 'rtsp://192.168.2.26:8554/display'

I/ASessionDescription( 1436): m=video 0 RTP/AVP 96

V/ASessionDescription( 1436): new section 'video 0 RTP/AVP 96'

I/ASessionDescription( 1436): b=RR:0

V/ASessionDescription( 1436): adding 'b=RR' => '0'

I/ASessionDescription( 1436): a=rtpmap:96 H264/90000

V/ASessionDescription( 1436): adding 'a=rtpmap:96' => 'H264/90000'

I/ASessionDescription( 1436): a=fmtp:96 packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKNoB4AiflwEQAAADABAAAAMDyPGDKg==,aM4PyA==;

V/ASessionDescription( 1436): adding 'a=fmtp:96' => 'packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKNoB4AiflwEQAAADABAAAAMDyPGDKg==,aM4PyA==;'

I/ASessionDescription( 1436): a=control:rtsp://192.168.2.26:8554/display/trackID=0

V/ASessionDescription( 1436): adding 'a=control' => 'rtsp://192.168.2.26:8554/display/trackID=0'

V/MyHandler( 1436): No duration found, assume live stream

V/avc_utils( 1436): frame_crop = (0, 0, 0, 4), cropUnitX = 2, cropUnitY = 2

V/avc_utils( 1436): sample aspect ratio = 1 : 1

I/APacketSource( 1436): dimensions 1920x1080

V/MyHandler( 1436): track #1 URL=rtsp://192.168.2.26:8554/display/trackID=0

V/ARTSPConnection( 1436): request: 'SETUP rtsp://192.168.2.26:8554/display/trackID=0 RTSP/1.0

V/ARTSPConnection( 1436): Transport: RTP/AVP/UDP;unicast;client_port=16538-16539

V/ARTSPConnection( 1436): User-Agent: stagefright/1.2 (Linux;Android 4.4.2)

V/ARTSPConnection( 1436): CSeq: 2

V/ARTSPConnection( 1436):

V/ARTSPConnection( 1436): '

I/ARTSPConnection( 1436): status: RTSP/1.0 200 OK

V/ARTSPConnection( 1436): line: 'Server: VLC/2.0.3'

V/ARTSPConnection( 1436): line: 'Date: Fri, 20 Mar 2015 12:31:42 GMT'

V/ARTSPConnection( 1436): line: 'Transport: RTP/AVP/UDP;unicast;client_port=16538-16539;server_port=38481-38482;ssrc=38E2C772;mode=play'

V/ARTSPConnection( 1436): line: 'Session: 5714247c19670b2f;timeout=60'

V/ARTSPConnection( 1436): line: 'Content-Length: 0'

V/ARTSPConnection( 1436): line: 'Cache-Control: no-cache'

V/ARTSPConnection( 1436): line: 'Cseq: 2'

I/MyHandler( 1436): SETUP(1) completed with result 0 (Success)

I/MyHandler( 1436): server specified timeout of 60 secs.

W/MyHandler( 1436): Missing 'source' field in Transport response. Using RTSP endpoint address.

W/MyHandler( 1436): Server picked an odd RTP port, it should've picked an even one, we'll let it pass for now, but this may break in the future.

V/MyHandler( 1436): successfully poked holes.

V/ARTSPConnection( 1436): request: 'PLAY rtsp://192.168.2.26:8554/display RTSP/1.0

V/ARTSPConnection( 1436): Session: 5714247c19670b2f

V/ARTSPConnection( 1436): User-Agent: stagefright/1.2 (Linux;Android 4.4.2)

V/ARTSPConnection( 1436): CSeq: 3

V/ARTSPConnection( 1436):

V/ARTSPConnection( 1436): '

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 180

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 227

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 242

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 168

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 194

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 166

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 189

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 266

I/ARTSPConnection( 1436): status: RTSP/1.0 200 OK

V/ARTSPConnection( 1436): line: 'Server: VLC/2.0.3'

V/ARTSPConnection( 1436): line: 'Date: Fri, 20 Mar 2015 12:31:42 GMT'

V/ARTSPConnection( 1436): line: 'RTP-Info: url=rtsp://192.168.2.26:8554/display/trackID=0;seq=16099;rtptime=2754592522'

V/ARTSPConnection( 1436): line: 'Range: npt=41.485533-'

V/ARTSPConnection( 1436): line: 'Session: 5714247c19670b2f;timeout=60'

V/ARTSPConnection( 1436): line: 'Content-Length: 0'

V/ARTSPConnection( 1436): line: 'Cache-Control: no-cache'

V/ARTSPConnection( 1436): line: 'Cseq: 3'

I/MyHandler( 1436): PLAY completed with result 0 (Success)

V/MyHandler( 1436): Range: npt=41.485533-

V/MyHandler( 1436): streamInfo[1] = url=rtsp://192.168.2.26:8554/display/trackID=0;seq=16099;rtptime=2754592522

V/MyHandler( 1436): track #1: rtpTime=2754592522 <=> npt=41.49

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 23

V/AAVCAssembler( 1436): Access unit complete (8 nal units)

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 18

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 23

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 14

V/ARTPConnection( 1436): receiving RTP

V/OMX ( 1436): OnEmptyBufferDone buffer=0x2bea7f00

V/ACodec ( 1436): [OMX.Freescale.std.video_decoder.avc.v3.hw-based] onOMXEmptyBufferDone 0x2bea7f00

V/ACodec ( 1436): [OMX.Freescale.std.video_decoder.avc.v3.hw-based] Needs to copy input data for buffer 0x2bea7f00. (0x2be7fe08 != 0x2bea7f90)

V/ACodec ( 1436): [OMX.Freescale.std.video_decoder.avc.v3.hw-based] calling emptyBuffer 0x2bea7f00 w/ codec specific data

V/OMX ( 1436): OnEmptyBufferDone buffer=0x2bea7f00

V/ACodec ( 1436): [OMX.Freescale.std.video_decoder.avc.v3.hw-based] onOMXEmptyBufferDone 0x2bea7f00

V/MyHandler( 1436): video track doesn't have enough data yet. (0.00 secs buffered)

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 177

V/AAVCAssembler( 1436): Access unit complete (8 nal units)

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 243

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 209

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 137

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 149

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 188

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 178

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): addSingleNALUnit of size 252

V/MyHandler( 1436): onAccessUnitComplete track 0

V/MyHandler( 1436): storing accessUnit, no time established yet

V/MyHandler( 1436): video track doesn't have enough data yet. (0.00 secs buffered)

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): sequence length 1

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): sequence length 1

V/AAVCAssembler( 1436): sequence length 2

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): sequence length 1

V/AAVCAssembler( 1436): sequence length 2

V/AAVCAssembler( 1436): sequence length 3

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): sequence length 1

V/AAVCAssembler( 1436): sequence length 2

V/AAVCAssembler( 1436): sequence length 3

V/AAVCAssembler( 1436): sequence length 4

V/ARTPConnection( 1436): receiving RTP

[...]

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/MyHandler( 1436): video track doesn't have enough data yet. (0.00 secs buffered)

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/MyHandler( 1436): video track doesn't have enough data yet. (0.00 secs buffered)

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/MyHandler( 1436): video track doesn't have enough data yet. (0.00 secs buffered)

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/AAVCAssembler( 1436): packetLost (expected 16299)

V/AAVCAssembler( 1436): Not the sequence number I expected

V/MyHandler( 1436): video track doesn't have enough data yet. (0.00 secs buffered)

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/MyHandler( 1436): video track doesn't have enough data yet. (0.00 secs buffered)

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/ARTPConnection( 1436): receiving RTP

V/AAVCAssembler( 1436): Not the sequence number I expected

V/MyHandler( 1436): video track doesn't have enough data yet. (0.00 secs buffered)

Original Attachment has been moved to: trace-1080p.txt.zip

imx6q rstp stream and h264 decoding

imx6q rstp stream and h264 decoding

Android

Graphics & Display

i.MX6Quad

Multimedia