gstreamer hang in vpu_WaitForInt

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

gstreamer hang in vpu_WaitForInt

646 Views
johannobermayr
Contributor I

Hello,

we have a own embedded system with i.MX6 Dualcore with Kernel 3.10.53 SMP
we have a testapplication with libgstreamer-1.0.so.0.203.0
we have a own thread with GstLoop

static int GstLoop(void *arg)
{
 OSmutexLock(&gstLock);
 gstInstance.loop = g_main_loop_new (NULL, TRUE);
 OSmutexUnlock(&gstLock);

 // run loop
 g_main_loop_run (gstInstance.loop);

 gst_element_set_state (gstInstance.pipeline, GST_STATE_NULL/*READY*/);

 gst_object_unref (gstInstance.sink);
 gst_object_unref (gstInstance.transform);
 gst_object_unref (gstInstance.sinkBin);

 /* could be that stopandcleanup() was called from GST context, than object_unref would hang. */
 g_main_loop_unref(gstInstance.loop);
 g_source_remove (gstInstance.bus_watch_id);
 gst_object_unref (gstInstance.pipeline);

 OSmutexLock(&stateLock);
 memset(&gstInstance, 0, sizeof(gstInstance));
 gstInstance.state = E_NOT_LOADED;
 OSmutexUnlock(&stateLock);

 return 0;
}

some seconds after starting the video we send a
 g_main_loop_quit(gstInstance.loop);
 to stop the video.
 
But we see, that the GstLoop will sometimes hang in the function pthread_mutex_lock called from g_mutex_lock with follow stack.
It can be work 2 days without trouble, or we have see, that this problem are 5 times per day.

Nov 29 12:33:39 test-arm user.info LRT: Thread is: gst_Loop:525:21916, pid:21916, objThis:0
Nov 29 12:33:39 test-arm user.info LRT:   Stack: 0xB5E41000-B5E4A000, size:36864, Min:15560
Nov 29 12:33:39 test-arm user.info LRT:   [0x000f06e4] LRT.exe(OSdumpstackContext+0xC4)  SP[0xB5E46FB8], BP[0xB5E472C4]
Nov 29 12:33:39 test-arm user.info LRT:   [0x000f0c28] LRT.exe(OSbacktrace+0x38)  SP[0xB5E46FB8], BP[0xB5E472D4]
Nov 29 12:33:39 test-arm user.info LRT:   [0x0013c8f4] LRT.exe(pthread_mutex_lock+0xB0)  SP[0xB5E46FB8], BP[0xB5E4732C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb636362c] libglib-2.0.so.0.3800.2(g_mutex_lock+0x10)  SP[0xB5E46FB8], BP[0xB5E47334]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb3b8356c] libgstimxvpu.so  SP[0xB5E46FB8], BP[0xB5E4735C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cc3ac] libgstreamer-1.0.so.0.203.0(gst_element_change_state+0x2C)  SP[0xB5E46FB8], BP[0xB5E47394]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cca6c] libgstreamer-1.0.so.0.203.0(gst_element_change_state+0x6EC)  SP[0xB5E46FB8], BP[0xB5E473FC]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cc0b4] libgstreamer-1.0.so.0.203.0(gst_element_set_state+0x94)  SP[0xB5E46FB8], BP[0xB5E47414]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61ad8e8] libgstreamer-1.0.so.0.203.0(gst_bin_recalculate_latency+0xC58)  SP[0xB5E46FB8], BP[0xB5E474D4]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb5ecbd88] libgstplayback.so  SP[0xB5E46FB8], BP[0xB5E47534]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cc3ac] libgstreamer-1.0.so.0.203.0(gst_element_change_state+0x2C)  SP[0xB5E46FB8], BP[0xB5E4756C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cca6c] libgstreamer-1.0.so.0.203.0(gst_element_change_state+0x6EC)  SP[0xB5E46FB8], BP[0xB5E475D4]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cc0b4] libgstreamer-1.0.so.0.203.0(gst_element_set_state+0x94)  SP[0xB5E46FB8], BP[0xB5E475EC]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61ad8e8] libgstreamer-1.0.so.0.203.0(gst_bin_recalculate_latency+0xC58)  SP[0xB5E46FB8], BP[0xB5E476AC]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb5ed9ca4] libgstplayback.so  SP[0xB5E46FB8], BP[0xB5E47734]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cc3ac] libgstreamer-1.0.so.0.203.0(gst_element_change_state+0x2C)  SP[0xB5E46FB8], BP[0xB5E4776C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cca6c] libgstreamer-1.0.so.0.203.0(gst_element_change_state+0x6EC)  SP[0xB5E46FB8], BP[0xB5E477D4]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cc0b4] libgstreamer-1.0.so.0.203.0(gst_element_set_state+0x94)  SP[0xB5E46FB8], BP[0xB5E477EC]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61ad8e8] libgstreamer-1.0.so.0.203.0(gst_bin_recalculate_latency+0xC58)  SP[0xB5E46FB8], BP[0xB5E478AC]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61f0384] libgstreamer-1.0.so.0.203.0(gst_param_spec_fraction+0xE74)  SP[0xB5E46FB8], BP[0xB5E47944]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb5ee24d4] libgstplayback.so  SP[0xB5E46FB8], BP[0xB5E4797C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cc3ac] libgstreamer-1.0.so.0.203.0(gst_element_change_state+0x2C)  SP[0xB5E46FB8], BP[0xB5E479B4]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cc51c] libgstreamer-1.0.so.0.203.0(gst_element_change_state+0x19C)  SP[0xB5E46FB8], BP[0xB5E479EC]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cca6c] libgstreamer-1.0.so.0.203.0(gst_element_change_state+0x6EC)  SP[0xB5E46FB8], BP[0xB5E47A54]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61cc0b4] libgstreamer-1.0.so.0.203.0(gst_element_set_state+0x94)  SP[0xB5E46FB8], BP[0xB5E47A6C]
Nov 29 12:33:39 test-arm user.info LRT:   [0x000faddc] LRT.exe(GstLoop+0x70)  SP[0xB5E46FB8], BP[0xB5E47A7C]
Nov 29 12:33:39 test-arm user.info LRT:   [0x0012499c] LRT.exe(trampoline+0x4C8)  SP[0xB5E46FB8], BP[0xB5E47B84]
Nov 29 12:33:39 test-arm user.info LRT:   [0x00122340] LRT.exe(pthread_startthread_hook+0x3C)  SP[0xB5E46FB8], BP[0xB5E47B94]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb6f360e8] libpthread-2.19.so(__pthread_get_minstack+0x1690)  SP[0xB5E46FB8], BP[0xB5E47CDC]

we have overwritten the pthread_mutex_lock function, and have replace the pthread_mutex_lock with pthread_mutex_timedlock and 150seconds timeout.
after this we get the mutex data owner to see, witch task will have the mutex.
and we get the callstack for this task
Nov 29 12:33:39 test-arm user.info LRT: Callstack for thread:'???':21919
Nov 29 12:33:39 test-arm user.info LRT:   [0xb69a138c] libc-2.19.so(ioctl+0xC)  SP[0xB43F2044], BP[0xB43F2054]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb5f7c77c] libvpu.so.4(vpu_WaitForInt+0x34)  SP[0xB43F2044], BP[0xB43F207C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb3b6e178] libimxvpuapi.so.0(imx_vpu_dec_decode+0x368)  SP[0xB43F2044], BP[0xB43F2154]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb3b8406c] libgstimxvpu.so  SP[0xB43F2044], BP[0xB43F2254]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb6150488] libgstvideo-1.0.so.0.203.0(gst_is_video_overlay_prepare_window_handle_message+0x1438)  SP[0xB43F2044], BP[0xB43F22DC]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61506fc] libgstvideo-1.0.so.0.203.0(gst_is_video_overlay_prepare_window_handle_message+0x16AC)  SP[0xB43F2044], BP[0xB43F2364]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb6152c04] libgstvideo-1.0.so.0.203.0(gst_is_video_overlay_prepare_window_handle_message+0x3BB4)  SP[0xB43F2044], BP[0xB43F240C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61e39e8] libgstreamer-1.0.so.0.203.0(gst_flow_get_name+0x1C0C)  SP[0xB43F2044], BP[0xB43F2494]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb610fbe4] libgstbase-1.0.so.0.203.0(gst_base_transform_set_passthrough+0x24C0)  SP[0xB43F2044], BP[0xB43F24DC]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61e39e8] libgstreamer-1.0.so.0.203.0(gst_flow_get_name+0x1C0C)  SP[0xB43F2044], BP[0xB43F2564]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb60ed3c4] libgstbase-1.0.so.0.203.0(gst_base_parse_push_frame+0x83C)  SP[0xB43F2044], BP[0xB43F2624]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb60f15dc] libgstbase-1.0.so.0.203.0(gst_base_parse_finish_frame+0x6B4)  SP[0xB43F2044], BP[0xB43F26C4]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb3bdab34] libgstvideoparsersbad.so  SP[0xB43F2044], BP[0xB43F27C4]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb60ea864] libgstbase-1.0.so.0.203.0(gst_base_parse_frame_new+0x548)  SP[0xB43F2044], BP[0xB43F2874]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb60ee5e8] libgstbase-1.0.so.0.203.0(gst_base_parse_push_frame+0x1A60)  SP[0xB43F2044], BP[0xB43F294C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb61e39e8] libgstreamer-1.0.so.0.203.0(gst_flow_get_name+0x1C0C)  SP[0xB43F2044], BP[0xB43F29D4]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb5617438] libgstcoreelements.so  SP[0xB43F2044], BP[0xB43F2AAC]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb621a590] libgstreamer-1.0.so.0.203.0(gst_tag_setter_get_tag_merge_mode+0x3FC)  SP[0xB43F2044], BP[0xB43F2B1C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb621b73c] libgstreamer-1.0.so.0.203.0(gst_task_join+0x368)  SP[0xB43F2044], BP[0xB43F2B2C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb6345efc] libglib-2.0.so.0.3800.2(g_thread_pool_new+0x4A8)  SP[0xB43F2044], BP[0xB43F2B6C]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb63452e8] libglib-2.0.so.0.3800.2(g_test_get_filename+0x174)  SP[0xB43F2044], BP[0xB43F2B84]
Nov 29 12:33:39 test-arm user.info LRT:   [0x00122340] LRT.exe(pthread_startthread_hook+0x3C)  SP[0xB43F2044], BP[0xB43F2B94]
Nov 29 12:33:39 test-arm user.info LRT:   [0xb6f360e8] libpthread-2.19.so(__pthread_get_minstack+0x1690)  SP[0xB43F2044], BP[0xB43F2CDC]

so we see that this task will hang in the function vpu_WaitForInt.
But after getting this information (callstack) the task will continue.

Thanks for help
 Johann


 

Labels (3)
0 Kudos
1 Reply

441 Views
igorpadykov
NXP Employee
NXP Employee

Hi Johann

seems community gstreamer-imx plugins used in the case

(like libimxvpuapi, libgstimxvpu) and this could be posted on community mail list

meta-freescale Info Page 

building for android · Issue #21 · Freescale/gstreamer-imx · GitHub 

difference between community gstreamer-imx plugins and
nxp gstreamer plugins (fsl-gst-plugin/ imx-gst1.0-plugin packages) described on
trac.gateworks.com/wiki/Yocto/gstreamer
NXP plugins
https://github.com/Freescale/meta-freescale/tree/master/recipes-multimedia

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

0 Kudos