Phil Endecott

VPU: timestamps, and audio/video sync

Discussion created by Phil Endecott on Nov 3, 2011
Latest reply on Nov 4, 2011 by Kan Hu
Does anyone here know anything about the VPU?  I have basic video playback working, and now I need to get audio-video sync correct. This is from an MPEG transport stream (DVB) and I believe that I need to use the Presentation Time Stamp (PTS) fields from the two PES streams for synchronisation.  It is simple enough to feed the two streams into the two decoders in synchrony. Unfortunately this is insufficient, as the latency from writing to the VPU input buffer to getting a frame out is variable. According to the MPEG spec, the correct way to do this is to feed in the data to both audio and video decoders as it is received, and to fetch video frames and audio data out of the decoders in synchrony.  The difficulty is that I don't know the PTS timestamps for the frames as they come out. Ideally, I think I'd like the VPU to report the corresponding PTS value in the DecOutputInfo struct, however this does not seem to be present.  Has anyone else ever looked at this? (Are any of Freescale's VPU engineers reading this?)  Thanks, Phil.