Sidebar
k23:k23.9:k23.9.7:start
Table of Contents
23.9.7 MediaPipeline (gb.media)
The class implements a GStreamer pipeline. A pipeline is a special container that provides its sub-controls with a global clock and a message bus. This class is also the base class for the MediaPlayer control, which implements a fully functional media player. The class works like a read/write array and can be created.
A MediaPipeline is based on a GStreamer element of type “Pipeline”. How to create a new MediaPipeline:
Dim hMediaPipeline As MediaPipeline hMediaPipeline = New MediaPipeline ( [ Parent As MediaContainer, Type As String, Polling As Integer ] ) [ As "EventName" ]
23.9.7.1 Properties
The MediaPipeline class has these properties:
| Property | Data type | Description |
|---|---|---|
| Children | .MediaContainer.Children | Returns a virtual collection of all MediaContainer child objects. |
| Duration, Length | Float | Returns the playback duration in seconds. |
| Inputs | String[] | Returns the names of all inputs. |
| Name | String | Returns or sets the name of the MediaControl. |
| Outputs | String[] | Returns the names of all outputs. |
| Parent | MediaContainer | Returns the parent element of the control. |
| Position | Float | Returns or sets the current (time) position within the processed data in seconds. |
| Speed | Float | Since version 3.13. Returns or sets the speed of media playback. 1.0 is the normal speed. A speed greater than 1 means faster playback, a speed greater than 0 and less than 1 means slower playback. If the speed is negative, playback is in reverse. A speed that is too low (between -1E-6 and +1E-6) results in an error message. |
| State | Integer | Returns or defines the state of a MediaControl. It can be described using one of the following constants: Media.Null (1), Media.Ready (2), Media.Paused (3) or Media.Playing (4). |
| Type | String | Returns the type of MediaControl. |
Table 23.9.7.1.1 : Properties of the MediaPipeline class
23.9.7.2 Methods
The MediaPipeline class has these methods:
| Method | Return type | Description |
|---|---|---|
| AddInput ( Child As MediaControl [ , Name As String ] ) | - | Selects an input of a child control that serves as input for the container. `Child` is a child control of the container. `Name` is the name of the input. If not specified, the first available input of the child control is used. |
| AddOutput ( Child As MediaControl [ , Name As String ] ) | - | Selects an output of a child control to act as an output for the container. `Child` is a child control of the container. `Name` is the name of the output. If nothing is specified, the first available output of the child control is used. |
| Close( ) | - | Closes the pipeline, i.e. sets its state to Ready and the state of all subordinate elements to zero. |
| Forward ( [ Frames As Integer ] ) | - | Since version 3.13. Forwards a certain number of frames. `Frames` is the number of frames to be moved, by default one. |
| GetLastImage ( ) | Image | Returns the last image displayed by a video output control. |
| GetLink ( Name As String ) | MediaLink | Provides the description of an input or output by name. |
| LinkLaterTo ( Target As MediaControl ) | - | Links the current control to the target control as soon as a new output appears on it. Some controls have outputs that can appear spontaneously, i.e. when the data arrives. In this case, the LinkTo method fails because the output is not yet present at the time of the call. You must therefore use LinkLaterTo. |
| LinkTo ( Target As MediaControl [ , Output As String, Input As String ] ) | - | Links an output of the current MediaControl with the input of the target MediaControl. `Target` is the target control, `Output` is the name of the output of the current control and `Input` is the name of the input of the target control. If output and input are not specified, GStreamer tries to find the best output and input that matches the link request, according to the type of source control and the type of target control. Some controls have outputs that may appear “on the fly”, i.e. when the data arrives. In this case, the LinkTo method will fail because the output is not present at the time of the call. You must use LinkLaterTo instead. |
| Pause( ) | - | Pause the pipeline, i.e. its status and the status of all subordinate pipelines is set to “paused”. |
| Play ( [ Async As Boolean ] ) | - | Play is started, i.e. the status of the pipeline and all its subordinate elements is set to “Playing”. |
| Seek ( Position As Float [ , Flags As Integer ] ) | - | Since version 3.13. Moves the stream to a specific (time) position, taking into account the restrictions defined by the specified flags. `Position` is the stream position in seconds and `Flags` describes the search options. |
| Stop( ) | - | Stops the pipeline, i.e. sets its state and the state of all its subordinate controls to `Ready`. |
| SetWindow ( Control As Control [ , X As Integer, Y As Integer, Width As Integer, Height As Integer ] ) | - | This method instructs the MediaControl to draw its output within a specific GUI control. `Control` is the control in which to draw. `X, Y, Width, Height` specify a target rectangle within the control. If it is not specified, the entire surface of the control is used. Only controls that implement the GStreamer-X-OVERLAY interface support this method. The target control must have its own window. To ensure this, use a DrawingArea with the property Cached. |
Table 23.9.7.2.1 : Methods of the MediaPipeline class
23.9.7.3 Events
The MediaPipeline class has these events:
| Event | Description |
|---|---|
| AboutToFinish ( ) | This event is triggered when the current media playback is about to end. It is triggered approximately two seconds before the end of the media. |
| Buffering ( ) | This event is triggered when data is buffered. |
| Duration ( ) | This event is triggered when the duration of a pipeline has changed. Read the Duration property again to get the new duration. |
| End ( ) | This event is triggered when (during playback) the end of the data stream has been reached. |
| Event ( Message As MediaMessage ) | This event is triggered when the media object triggers an internal event. `Message` is the internal event. |
| Message ( Source As MediaControl, Type As Integer, Message As String ) | This event is triggered when a subordinate control sends a message. `Source` is the child control that sends the message. `Type` is the message type and `Message` is the content of the message. The type of a message can be one of the following constants: Media.Error, Media.Info or Media.Warning. |
| Position ( ) | Since version 3.13. This event is triggered when the stream position - which represents a time value - changes. |
| Progress ( ) | This event is triggered while the medium is playing - until it is stopped. |
| Start ( ) | This event is triggered when a new medium is played. |
| State ( ) | This event is triggered when the status of the control element has changed. |
| Tag ( TagList As MediaTagList ) | This event is triggered when some tags are found. `TagList` is the list of tags found. |
Table 23.9.7.3.1 : Events of the MediaPipeline class
23.9.7.4 Audio player project
The following project implements an audio player for ogg files. The following GStreamer command is the basis for the adaptation in Gambas:
$ gst-launch-1.0 filesrc location=output_o.ogg ! oggdemux ! vorbisdec ! audioconvert ! autoaudiosink
Figure 23.9.7.4.1: Pipeline
The source code is specified in full and briefly commented:
' Gambas class file Private mpPipeline As MediaPipeline Private mcSource As MediaControl Private mcOggDemux As MediaControl Private mcVorbisDecoder As MediaControl Private mcAudioConvert As MediaControl Private mcAutoAudioSink As MediaControl Public Sub Form_Open() FMain.Resizable = False End Public Sub CreatePipeline() '-- Generate Pipeline mpPipeline = New MediaPipeline As "hPipeline" '-- Generate MediaControls mcSource = New MediaControl(mpPipeline, "filesrc") mcSource["location"] = User.Home &/ "output_o.ogg" mcOggDemux = New MediaControl(mpPipeline, "oggdemux") mcVorbisDecoder = New MediaControl(mpPipeline, "vorbisdec") mcAudioConvert = New MediaControl(mpPipeline, "audioconvert") mcAutoAudioSink = New MediaControl(mpPipeline, "autoaudiosink") '-- Link MediaControls mcSource.LinkTo(mcOggDemux) mcOggDemux.LinkLaterTo(mcVorbisDecoder) mcVorbisDecoder.LinkTo(mcAudioConvert) mcAudioConvert.LinkTo(mcAutoAudioSink) End Public Sub btnFilePlay_Click() CreatePipeline() '-- Start/Play Pipeline mpPipeline.Play() End Public Sub Form_Close() If mpPipeline And If mpPipeline.State = Media.Playing Then mpPipeline.Stop() mpPipeline.Close() Endif End
Comment
- Firstly, the required media controls and the pipeline mpPipeline are defined as MediaControls.
- When the program is started, the pipeline is created and initialised in the CreatePipeline() procedure.
- Playback of the audio file output_o.ogg is triggered by calling mpPipeline.Play() by pressing the btnFilePlay_Click() button.
- When the programme is ended, the pipeline is stopped and closed.
You can extend the audio player with the following extensions:
- Insert a dialogue for selecting audio files with the filter for ogg files.
- If you use a different pipeline with the universal decoder 'decodebin', you can also play other audio formats such as mp3.
Alternative pipeline:
$ gst-launch-1.0 filesrc location=audiofile.ogg ! decodebin ! audioconvert ! audioresample ! autoaudiosink $ gst-launch-1.0 filesrc location=audiofile.mp3 ! decodebin ! audioconvert ! audioresample ! autoaudiosink
23.9.7.5 Notes on linking MediaControls
When linking individual MediaControls - as in the following lines:
'-- Link MediaControls mcSource.LinkTo(mcOggDemux) mcOggDemux.LinkLaterTo(mcVorbisDecoder) mcVorbisDecoder.LinkTo(mcAudioConvert) mcAudioConvert.LinkTo(mcAutoAudioSink)
the question arose as to whether there are sufficient criteria for which links the LinkTo(…) or LinkLaterTo(…) methods are used.
LinkLaterTo() is only required for elements in which the source pad is specified as “sometimes available”. “Sometime pads” are also called “dynamic pads”: https://gstreamer.freedesktop.org/documentation/application-development/basics/pads.html?gi-language=c. These are pads that do not have a permanent connection.
In view of all the media projects tested so far, this only applies to 2 elements, although there could of course be more. Based on our current knowledge, only the following elements have sometimes-src pads:
(1) uridecodebin (2) alle Demuxer
In addition, instead of “src pads”, it might be better to speak of “source pads” (i.e. outputs), because unlike a filter, demuxers such as those for videos have sometimes-source pads with names such as “video…” and “audio…” instead of “src:”
Conclusion
- In all media projects tested so far, only one LinkLaterTo() has ever been used in a pipeline. This no longer applies to parallel pipelines in a MediaContainer.
- Demuxers always require a LinkLaterTo, otherwise an error message “Unable to link controls” will appear if you try to connect a demuxer output with a LinkTo() instead of a LinkLaterTo(). If you always use a LinkLaterTo() instead of a LinkTo(), no error message is triggered. However, the PipeLine will then no longer work.
- For applications with higher requirements, you should use GStreamer shell commands.
The following source code excerpt from the above audio player project is correct and works as expected:
'-- Link MediaControls mcSource.LinkTo(mcOggDemux) mcOggDemux.LinkLaterTo(mcVorbisDecoder) mcVorbisDecoder.LinkTo(mcAudioConvert) mcAudioConvert.LinkTo(mcAutoAudioSink)
23.9.7.6 Excursus
For your own projects, you can use the following tested GStreamer shell commands as exercises to implement them in Gambas projects with the help of the media classes.
23.9.7.6.1 P1 - MP3 files
Listen
$ gst-launch-1.0 \ uridecodebin uri="http://icecast.ndr.de/ndr/ndrinfo/schleswigholstein/mp3/128/stream.mp3" ! audioconvert ! autoaudiosink
Save
$ gst-launch-1.0 \ uridecodebin uri="http://icecast.ndr.de/ndr/ndrinfo/schleswigholstein/mp3/128/stream.mp3" ! audioconvert ! lamemp3enc target=bitrate bitrate=128 cbr=true \ ! filesink location=$HOME"/output_m.mp3"
Listen and save
$ gst-launch-1.0 \ uridecodebin uri="http://icecast.ndr.de/ndr/ndrinfo/schleswigholstein/mp3/128/stream.mp3" ! audioconvert ! tee name=radio ! queue \ ! autoaudiosink radio. ! queue \ ! lamemp3enc target=bitrate bitrate=128 cbr=true ! filesink location=$HOME"/output_m.mp3"
Play
$ gst-launch-1.0 \ filesrc location=mp.mp3 ! decodebin ! audioconvert ! audioresample ! autoaudiosink
23.9.7.6.2 P2 - OGG files
Listen
$ gst-launch-1.0 \ uridecodebin uri="http://icecast.ndr.de/ndr/ndrinfo/schleswigholstein/mp3/128/stream.mp3" \ ! audioconvert ! autoaudiosink
Save
$ gst-launch-1.0 \ uridecodebin uri="http://icecast.ndr.de/ndr/ndrinfo/schleswigholstein/mp3/128/stream.mp3" \ ! audioconvert ! audioresample ! vorbisenc ! oggmux ! \ filesink location=$HOME"/output_o2.ogg"
Listen and save
$ gst-launch-1.0 \ uridecodebin uri="http://icecast.ndr.de/ndr/ndrinfo/schleswigholstein/mp3/128/stream.mp3" \ ! tee name=radio ! queue ! audioconvert ! autoaudiosink radio. \ ! queue ! audioconvert ! audioresample ! vorbisenc \ ! oggmux ! filesink location=$HOME"/output_o.ogg"
Play
$ gst-launch-1.0 \ filesrc location=output_o.ogg ! oggdemux ! vorbisdec ! audioconvert ! audioresample ! autoaudiosink
23.9.7.6.3 P3 - Display video files
$ gst-launch-1.0 playbin uri=https://upload.wikimedia.org/wikipedia/commons/4/41/Big_Buck_Bunny_medium.ogv $ gst-launch-1.0 playbin uri=https://gstreamer.freedesktop.org/data/media/sintel_trailer-480p.webm
Download
Three Gambas projects
Project_1: audio player
Project_2: web radio player
Project_3: Web radio player with recording function
Download
k23/k23.9/k23.9.7/start.txt · Last modified: 26.03.2024 by emma
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial 3.0 Unported
