playbin2 Introduction

►Playbin2 provides a stand-alone everything-in-one abstraction for an audio and/or video player.

•Audio sink and video sink can be specified by application

•Supports subtitle for video files

•Adjustable A/V offset

•Selects audio/video/subtitle streams by input selector

•Audio post-processing by audio convert, audio resample and volume

•

►The pipeline is created by playbin2 dynamically:

•Typefind checks the source type

•Demuxer parses the source data and create pads for all the streams, respectively

•Playbin2 searches for and creates decoders in element factory who have capability to decode specific format of audio/video

•Playbin2 creates and connects audio sink chain and video sink chain

Playbin2: interaction with application

►Set/get properties of playbin2

•“audio-sink”, “video-sink”

•“current-audio”, “current-video”, “current-text”

•“n-audio”, “n-video”, “n-text” (read only)

•“uri”, “suburi”, “av-offset”, “volume”, “mute”

►Events

•Bidirectional: flush start, flush end

•Downstream: eos, new segment, tag

•Upstream: qos, seek

►Queries

•Position, duration, latency, rate, seeking, segment

►Messages

•Eos, error, warning, info, tag, buffering, state changed, duration, latency,

►Signals

•“video-changed”, “audio-changed”, “text-changed”

•“video-tags-changed”, “audio-tags-changed”, “text-tags-changed”

Playback: track selection

►Pipeline for audio/video/subtitle multi-track file

•Demuxer creates src pads for all the tracks, respectively

•Each track has its own decoder and sink pad of input selector

•One sink pad in input selector is activated to play corresponding track

•Audio, video and subtitle have separated input selectors.

•Take audio for example:

►Input selector

•Only push the data from active sink pad to its src pad

•The data from all the sink pads shall be synchronized, so that timestamps can be consecutive when switch to another track. Otherwise, there will be no sound or video for a long time after switching track.

•If demuxer gets samples in file mode (need a large queue before decoder):

A lock is set for each inactive tracks to synchronize tracks. Otherwise, the inactive tracks would be decoded in full speed (usually much more faster than active track).

The lock compares the received timestamp with the last timestamp of active track. If larger, waits until the next timestamp (buffer) of acitived track arrives; if smaller, drop this buffer and get the next buffer.

•If demuxer gets samples in track mode and the queue is small:

All tracks is synchronized by demuxer