MJPEG streaming with Android 5.1 Extended Media Codec package (i.MX6)

[rljones]

I'm having some trouble using the android 5.1 extended multimedia codec package with our AOSP 5.1 build. I've very closely followed the instructions in the release notes to install, build, and test the added components without error. I've even been able to verify certain added components are working as intended, such as the added AAC audio support. 

My problem begins with attempting to use MJPEG video sources (originally failed to play a camera mjpeg video stream and am now trying to simplify). I've attempted using the instructions in the release notes to test using a network video stream, but Android's gallery3d app fails to play the video. I've tried using various incarnations of the following commands to no avail:

# stream from host

vlc -vvv stream_file_name --sout udp://224.0.1.1:1234

# play on android device

am start -n com.android.gallery3d/com.android.gallery3d.app.MovieActivity -d udp://224.0.1.1:1234

Here is a snippet from the logcat of the device after failing to play the video stream:

D/AndroidRuntime( 8810): >>>>>> START com.android.internal.os.RuntimeInit uid 0 <<<<<<
D/AndroidRuntime( 8810): CheckJNI is ON
E/cutils-trace( 8810): Error opening trace file: No such file or directory (2)
E/memtrack( 8810): Couldn't load memtrack module (No such file or directory)
E/android.os.Debug( 8810): failed to load memtrack module: -2
D/AndroidRuntime( 8810): Calling main entry com.android.commands.am.Am
I/ActivityManager(  532): START u0 {dat=http://rjones:1234 flg=0x10000000 cmp=com.android.gallery3d/.app.MovieActivity} from uid 0 on display 0
V/WindowManager(  532): addAppToken: AppWindowToken{33ffeeab token=Token{241ac8fa ActivityRecord{3a306f25 u0 com.android.gallery3d/.app.MovieActivity t8}}} to stack=1 task=8 at 0
D/AndroidRuntime( 8810): Shutting down VM
V/WindowManager(  532): Adding window Window{9176377 u0 Starting com.android.gallery3d} at 3 of 8 (after Window{fbaaa9 u0 com.android.launcher3/com.android.launcher3.Launcher})
I/ActivityManager(  532): Start proc 8830:com.android.gallery3d/u0a29 for activity com.android.gallery3d/.app.MovieActivity
E/Vold    (  186): Failed to find mounted volume for /storage/extsd/Android/data/com.android.gallery3d/cache/
W/Vold    (  186): Returning OperationFailed - no handler for errno 0
W/ContextImpl( 8830): Failed to ensure directory: /storage/extsd/Android/data/com.android.gallery3d/cache
E/Vold    (  186): Failed to find mounted volume for /storage/extsd1/Android/data/com.android.gallery3d/cache/
W/Vold    (  186): Returning OperationFailed - no handler for errno 0
W/ContextImpl( 8830): Failed to ensure directory: /storage/extsd1/Android/data/com.android.gallery3d/cache
E/Vold    (  186): Failed to find mounted volume for /storage/udisk/Android/data/com.android.gallery3d/cache/
W/Vold    (  186): Returning OperationFailed - no handler for errno 0
W/ContextImpl( 8830): Failed to ensure directory: /storage/udisk/Android/data/com.android.gallery3d/cache
E/Vold    (  186): Failed to find mounted volume for /storage/udisk1/Android/data/com.android.gallery3d/cache/
W/Vold    (  186): Returning OperationFailed - no handler for errno 0
W/ContextImpl( 8830): Failed to ensure directory: /storage/udisk1/Android/data/com.android.gallery3d/cache
E/Vold    (  186): Failed to find mounted volume for /storage/sata/Android/data/com.android.gallery3d/cache/
W/Vold    (  186): Returning OperationFailed - no handler for errno 0
W/ContextImpl( 8830): Failed to ensure directory: /storage/sata/Android/data/com.android.gallery3d/cache
I/MediaFocusControl(  532):  AudioFocus  requestAudioFocus() from android.media.AudioManager@2e4f24b9 req=2flags=0x0
D/OpenGLRenderer( 8830): Use EGL_SWAP_BEHAVIOR_PRESERVED: true
I/imx6.gralloc( 8830): open gpu gralloc module!
D/Atlas   ( 8830): Validating map...
V/WindowManager(  532): Adding window Window{f721867 u0 com.android.gallery3d/com.android.gallery3d.app.MovieActivity} at 3 of 9 (before Window{9176377 u0 Starting com.android.gallery3d})
I/OpenGLRenderer( 8830): Initialized EGL, version 1.4
D/OpenGLRenderer( 8830): Enabling debug mode 0
E/BufferQueueProducer(  187): [com.android.gallery3d/com.android.gallery3d.app.MovieActivity] dequeueBuffer: can't dequeue multiple buffers without setting the buffer count
V/WindowManager(  532): Adding window Window{b840ebd u0 SurfaceView} at 3 of 10 (before Window{f721867 u0 com.android.gallery3d/com.android.gallery3d.app.MovieActivity})
I/ActivityManager(  532): Displayed com.android.gallery3d/.app.MovieActivity: +1s396ms
I/MediaFocusControl(  532):  AudioFocus  requestAudioFocus() from android.media.AudioManager@2e4f24b9 req=1flags=0x0
D/MediaPlayer( 8830): Couldn't open file on client side, trying server side
I/art     (  532): Background partial concurrent mark sweep GC freed 7759(595KB) AllocSpace objects, 31(5MB) LOS objects, 33% free, 6MB/10MB, paused 3.032ms total 142.274ms
I/OMXPlayer(  192): Loading content: http://rjones:1234
I/OMXPlayer(  192): LEVEL: 1 FUNCTION: MediaTypeInspect LINE: 1965
I/OMXPlayer(  192): Can't inspect media content type by subfix.
I/OMXPlayer(  192): GetContentTypeByPipeHandle, uri http://rjones:1234
W/InputMethodManager( 8830): Ignoring onBind: cur seq=14, given seq=13
I/OMXPlayer(  192): LEVEL: 1 FUNCTION: GetContentLength LINE: 115
I/OMXPlayer(  192): GetContentLength url_seek AVSEEK_SIZE fail
I/OMXPlayer(  192): server seekable: 0
I/OMXPlayer(  192): file size: 0, buffer size : 0 
I/OMXPlayer(  192): DetectAudioTypeByFrame: server unseekable
I/OMXPlayer(  192): DetectAudioTypeByFrame: server unseekable
I/OMXPlayer(  192): DetectAudioTypeByFrame: server unseekable
I/OMXPlayer(  192): GetContentTypeAndPipeHandle ok, type 0
I/OMXPlayer(  192): LEVEL: 1 FUNCTION: MediaTypeInspect LINE: 1973
I/OMXPlayer(  192): Can't inspect media content type.
I/OMXPlayer(  192): LEVEL: 1 FUNCTION: Load LINE: 823
I/OMXPlayer(  192): Can't inspec content format.
I/OMXPlayer(  192): GMPlayer Stop
I/OMXPlayer(  192): LEVEL: 1 FUNCTION: Stop LINE: 965
I/OMXPlayer(  192): GMPlayer Stop from wrong state 6
E/MediaPlayer( 8830): error (-2147483648, 0)
E/MediaPlayer( 8830): Error (-2147483648,0)
D/VideoView( 8830): Error: -2147483648,0
V/WindowManager(  532): not Base app: Adding window Window{3a60e74 u0 com.android.gallery3d/com.android.gallery3d.app.MovieActivity} at 5 of 10
E/BufferQueueProducer(  187): [com.android.gallery3d/com.android.gallery3d.app.MovieActivity] dequeueBuffer: can't dequeue multiple buffers without setting the buffer count

I've also attempted to use the added cactus player app included with the codecs to do the video stream playing. Logcat snippet:

I/ActivityManager(  538): START u0 {dat=/data/download (2) flg=0x10000000 cmp=com.freescale.cactusplayer/.VideoPlayer} from uid 0 on display 0
V/WindowManager(  538): addAppToken: AppWindowToken{28d1cfdc token=Token{2f099c4f ActivityRecord{4d5d6ae u0 com.freescale.cactusplayer/.VideoPlayer t64}}} to stack=1 task=64 at 3
V/WindowManager(  538): Adding window Window{377afe61 u0 com.freescale.cactusplayer/com.freescale.cactusplayer.VideoPlayer} at 9 of 14 (after Window{af6b624 u0 com.freescale.cactusplayer/com.freescale.cactusplayer.VideoPlayer})
E/BufferQueueProducer(  186): [com.freescale.cactusplayer/com.freescale.cactusplayer.VideoPlayer] dequeueBuffer: can't dequeue multiple buffers without setting the buffer count
V/WindowManager(  538): Adding window Window{db6c647 u0 SurfaceView} at 9 of 15 (before Window{377afe61 u0 com.freescale.cactusplayer/com.freescale.cactusplayer.VideoPlayer})
I/OMXPlayer(  191): file size: 8573588, buffer size : 131072 
I/OMXPlayer(  191): it is a MPEG2 content 
I/OMXPlayer(  191): Loading content: sharedfd://23:0:8573588:0
I/OMXPlayer(  191): LEVEL: 1 FUNCTION: MediaTypeInspect LINE: 1965
I/OMXPlayer(  191): Can't inspect media content type by subfix.
I/OMXPlayer(  191): file size: 8573588, buffer size : 131072 
I/OMXPlayer(  191): it is a MPEG2 content 
I/OMXPlayer(  191): MediaTypeInspectByContent role: parser.mpg2
I/OMXPlayer(  191): Core parser BLN_MAD-MMLAYER_MPG2PARSER_04.04.37  build on Aug 12 2015 17:15:05 
I/OMXPlayer(  191): Mpeg2CreateParser:parser created successfully
I/OMXPlayer(  191): LEVEL: 1 FUNCTION: ExportIndexTable LINE: 175
I/OMXPlayer(  191): Fail to open index file to export index: sharedfd://.idx
I/OMXPlayer(  191): nPreCacheSize = 186382
I/OMXPlayer(  191): GetOneSample() send eos , track 0
I/OMXPlayer(  191): SetupDecoder MP3 decoder Wrapper  build on Apr 10 2014 16:18:39 
I/OMXPlayer(  191): LEVEL: 1 FUNCTION: ProcessDataBuffer LINE: 232
I/OMXPlayer(  191): AudioFilterCheckFrameHeader fail.
I/OMXPlayer(  191): LEVEL: 1 FUNCTION: ProcessDataBuffer LINE: 232
I/OMXPlayer(  191): AudioFilterCheckFrameHeader fail. 

...

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: ProcessDataBuffer LINE: 232

I/OMXPlayer(  191): AudioFilterCheckFrameHeader fail.

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: FilterOneBuffer LINE: 3081

I/OMXPlayer(  191): FilterOneBuffer: vpu dec buf failure: ret=0x1 

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: Load LINE: 864

I/OMXPlayer(  191): StartPipeline faint

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: Load LINE: 877

I/OMXPlayer(  191): Load faint and stop

I/OMXPlayer(  191): GMPlayer Stop

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: StateTransDownWard LINE: 341

I/OMXPlayer(  191): OMX.Freescale.std.video_render.surface.sw-based Invalid state transitioin: 1 -> 4.

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: SysEventHandler LINE: 1804

I/OMXPlayer(  191): OMX.Freescale.std.video_decoder.mpeg2.hw-based report Error 8000100b.

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: FlushInputBuffer LINE: 3622

I/OMXPlayer(  191): FlushInputBuffer: failure state transition, current state=2 

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: StateTransDownWard LINE: 341

I/OMXPlayer(  191): OMX.Freescale.std.video_render.surface.sw-based Invalid state transitioin: 1 -> 2.

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: DeInitFilterComponent LINE: 2530

I/OMXPlayer(  191): invalid state: 2, close vpu manually 

I/OMXPlayer(  191): GMPlayer Stop

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: Stop LINE: 965

I/OMXPlayer(  191): GMPlayer Stop from wrong state 6

I/ActivityManager(  538): Displayed com.freescale.cactusplayer/.VideoPlayer: +1s618ms

I/OMXPlayer(  191): GMPlayer Stop

I/OMXPlayer(  191): LEVEL: 1 FUNCTION: Stop LINE: 965

I/OMXPlayer(  191): GMPlayer Stop from wrong state 6 

Am I missing something here? Is there a better way to test MJPEG support while using these extended codecs?

[rljones · Accepted solution]

Ended up coming back to this and have a solution implemented that is using hardware decoding. I've done this by editing the extended codec source to include a custom parser I made that understands an mjpeg live stream, creates frames, and passes them to the decoder. If anyone needs some help doing this send me an email.

[diegoadrian]

Hello

Unfortunately, I get confirmed internally that we do not support the MJPEG protocol in our app. You will need to create an app that could support that protocol. I apologize for this inconvenient. 

However, I found some related information that could help you. Please see the below web-pages:

Play RTSP streaming in an Android application - Stack Overflow 

Simple Python Motion Jpeg (mjpeg server) from webcam. Using: OpenCV,BaseHTTPServer · GitHub 

Unfortunately, this is as far as I can go.

I hope this information could help you, and I apologize for the inconveniences this could cause you. Have a nice day!

Best Regards,

Diego.