Tutorials > How to combine the video mixer and the automatic fader

Inside the previous tutorial How to use the video mixer we have seen how two video players can mix the output of their respective video streams in one single window and inside the tutorial How to use the Automatic Fader we have seen how two players can fade two audio streams on the same speakers of a specific sound card: now we will see how these two features could be combined together in order to create a video mixer that will support the fading of both audio and video streams at the same time.

The easiest approach for implementing this feature is probably linking the volume level of the player fading-out, on the top of the Z-order (meaning that it will by default cover the video stream of the player fading-in), with the alpha channel transparency of its video stream: while the volume on this player is fading down, we raise the transparency level of the related video stream so the video stream of the player fading-in, which is at the bottom of the Z-Order, appears through until the video stream of the player fading-out totally disappears when its volume level reaches level 0.

Let's see in detail how we could proceed; first of all we need to initialize the control by instancing two players and initialize and instruct the automatic fader in order to manage one single playlist whose items will be played by the two players: for video clips, both players are set to work with the AUDIO_RENDERER_MODE_CUSTOM_2 audio rendering mode, allowing a better synchronization between audio and video stream.

Visual Basic.NET

' init the control

audioDjStudio1.InitSoundSystem (2, 0, 0, 0, 0)

' initialize the fader which will use this single playlist split for player 1 and player 2

audioDjStudio1.Fader.PlayListUseSingle(Player_1)

audioDjStudio1.Fader.Init(enumFadeTypes.FADE_SINGLE_PLAYLIST, Player_1, Player_2)

' set the audio rendering mode

audioDjStudio1.DisplayVideoPlayer.AudioRendererModeSet(Player_1, enumAudioRendererModes.AUDIO_RENDERER_MODE_CUSTOM_2)

audioDjStudio1.DisplayVideoPlayer.AudioRendererModeSet(Player_2, enumAudioRendererModes.AUDIO_RENDERER_MODE_CUSTOM_2)

Visual C#

// init the control

audioDjStudio1.InitSoundSystem (2, 0, 0, 0, 0);

// initialize the fader which will use this single playlist split for player 1 and player 2

audioDjStudio1.Fader.PlayListUseSingle(Player_1);

audioDjStudio1.Fader.Init(enumFadeTypes.FADE_SINGLE_PLAYLIST, Player_1, Player_2);

// set the audio rendering mode

audioDjStudio1.DisplayVideoPlayer.AudioRendererModeSet(Player_1, enumAudioRendererModes.AUDIO_RENDERER_MODE_CUSTOM_2);

audioDjStudio1.DisplayVideoPlayer.AudioRendererModeSet(Player_2, enumAudioRendererModes.AUDIO_RENDERER_MODE_CUSTOM_2);

For Visual Basic 6 it's recommended adding the code above into the Form_Load handler of the container form while for Visual C++ it's recommended adding the code below into the OnInitDialog (WM_INITDIALOG Windows message) handler of the container dialog box.

Now we can continue by creating the video mixer into a secondary form, whose client area will be totally covered by the video mixer surface, and attach the two players:

Visual Basic.NET

' create the video mixer object into a secondary form (FormMixer) and get back its unique identifier

m_nMixerUniqueID = audioDjStudio1.DisplayVideoMixer.Create(m_frmMixer.Handle, _

0, 0, m_frmMixer.ClientSize.Width, m_frmMixer.ClientSize.Height, Color.Black)

' attach the 2 instanced players to the VideoMixer object

audioDjStudio1.DisplayVideoPlayer.AttachToVideoMixer(Player_1, m_nMixerUniqueID)

audioDjStudio1.DisplayVideoPlayer.AttachToVideoMixer(Player_2, m_nMixerUniqueID)

' initialize a variable that will store the index of the player currently fading-out

m_nPlayerFadingOut = -1

Visual C#

// create the video mixer object and get back its unique identifier

m_nMixerUniqueID = audioDjStudio1.DisplayVideoMixer.Create(m_frmMixer.Handle, 0, 0,

m_frmMixer.ClientSize.Width, m_frmMixer.ClientSize.Height, Color.Black);

// attach the 2 instanced players to the VideoMixer object

audioDjStudio1.DisplayVideoPlayer.AttachToVideoMixer(Player_1, m_nMixerUniqueID);

audioDjStudio1.DisplayVideoPlayer.AttachToVideoMixer(Player_2, m_nMixerUniqueID);

// initialize a variable that will store the index of the player currently fading-out

m_nPlayerFadingOut = -1;

we have also initialized a variable, named m_nPlayerFadingOut, that will store the index of the player currently fading-out; this will be useful later to determine the player that will need its alpha channel transparency to be modified. You may notice the absence of the video preview window for the involved video players: the presence of the preview windows requires an amount of CPU which is probably not justified for this specific situation where all should be automated so omitting its creation may be a good idea.

Now we could proceed with loading a playlist but we still need to decide if we want to work with standard playlist formats, like M3U, PLS and WPL, which don't contain any automation information (so we will totally rely upon settings of the automatic fader) or if we prefer working with the PDJ playlist format which allows creating very customized automations. The difference here is quite important because, differently from audio clips whose duration can be determined on the fly, with video clips we cannot automatically obtain the duration because building filters graph through DirectShow is a lengthy operation that would require a huge amount of time when dealing with long playlists. As a general rule, if the playlist is in PDJ format and contains fading automation, we can load the playlist using the automation mode while in all other cases we should load it in speed or full mode

Visual Basic.NET

' check the format to load

If bFormatIsPDJ = True Then

audioDjStudio1.PlayListLoad(Player_1, strPathname, enumPlayListModes.PLAYLIST_AUTOMATION_MODE)

else

' dealing with video clips whose duration is unknown until effective loading, it's better to

' disable the automatic check that verifies if loaded files are long enough to fit the fader settings

' this is only a sample application without strong error checking so use this setting carefully :-)

audioDjStudio1.Fader.CheckItemsDurationOnStart = False

audioDjStudio1.PlayListLoad(Player_1, strPathname, enumPlayListModes.PLAYLIST_SPEED_MODE)

end if

' start fading

audioDjStudio1.Fader.StartManualFading

Visual C#

// check the format to load

if (bFormatsIsPDJ)

audioDjStudio1.PlayListLoad(Player_1, strPathname, enumPlayListModes.PLAYLIST_AUTOMATION_MODE);

else

{

// dealing with video clips whose duration is unknown until effective loading, it's better to

// disable the automatic check that verifies if loaded files are long enough to fit the fader settings

// this is only a sample application without strong error checking so use this setting carefully :-)

audioDjStudio1.Fader.SetCheckItemsDurationOnStart (FALSE);

audioDjStudio1.PlayListLoad(Player_1, strPathname, enumPlayListModes.PLAYLIST_SPEED_MODE);

}

// start fading

audioDjStudio1.Fader.StartManualFading();

As seen above, once loaded the automatic fader can be started through the Fader.StartManualFading method; at this point the multimedia engine automatically checks if the loaded playlist contains at least two songs whose duration is long enough to apply the fader's timings and settings: if this check should fail, the call to the Fader.StartManualFading method would return an error code.

When dealing with playlists in PDJ format, containing volume fading automation, this wouldn't be an issue because the playlist is tailored to allow a total automation but when dealing with other playlist formats, which don't contain any kind of information about embedded video clips, the first fading would obviously fail: for this reason if we should be sure that all video clips are compatible with fader's settings, it could be useful disabling this automatic check by setting the Fader.CheckItemsDurationOnStart property to "False".

The automatic fader has now been started so it immediately begins loading into the involved players the sequence of playlist's video clips: it must be remarked that, when dealing with video clips, due to the filters graph building by DirectShow, the loading could be a bit more "heavy" respect to audio clips so you will have to keep count of some small delay during the loading.

The automatic fader already knows how to load and start video clips and also knows how to fade the respective audio stream but it still doesn't know how to manage the fading of the video streams into the stream mixer's window so, for achieving this purpose, we still need to add some coding.

As mentioned before, a possible solution would be linking the volume level of the player fading-out, on the top of the Z-order (meaning that it will by default cover the video stream of the player fading-in), to the alpha channel transparency of its video stream: while the volume on this player is fading down, we raise the transparency level of the related video stream so the video stream of the player fading-in, which is at the bottom of the Z-Order, appears through until the video stream of the player fading-out totally disappears when its volume level reaches level 0. The best approach is obviously synchronizing with events generated by the automatic fader as seen below:

Visual Basic.NET

Private Sub audioDjStudio1_SoundLoaded(ByVal sender As Object, ByVal e As AudioDjStudio.SoundLoadedEventArgs) Handles audioDjStudio1.SoundLoaded

' this is the next player to fade-in so put it at the bottom of the Z-Order

audioDjStudio1.DisplayVideoMixer.PlayerZOrderSet (m_nMixerUniqueID, e.nPlayerIndex, 100)

... do other stuffs

End Sub

Private Sub audioDjStudio1_FadeOutStarted(ByVal sender As Object, ByVal e As AudioDjStudio.PlayerEventArgs) Handles audioDjStudio1.FadeOutStarted

' keep count of the player fading out because we will apply transparency on its video stream

m_nPlayerFadingOut = e.nPlayerIndex

End Sub

Private Sub audioDjStudio1_FadeOutCompleted(ByVal sender As Object, ByVal e As AudioDjStudio.PlayerEventArgs) Handles audioDjStudio1.FadeOutCompleted

' the player is no more fading

m_nPlayerFadingOut = -1

End Sub

Private Sub audioDjStudio1_FadeInCompleted(ByVal sender As Object, ByVal e As AudioDjStudio.PlayerEventArgs) Handles audioDjStudio1.FadeInCompleted

' fade in has been completed so put it at the top of the Z-Order

audioDjStudio1.DisplayVideoMixer.PlayerZOrderSet (m_nMixerUniqueID, e.nPlayerIndex, 0)

End Sub

Private Sub audioDjStudio1_FadingVolumeChanged(ByVal sender As Object, ByVal e As AudioDjStudio.FadingEventArgs) Handles audioDjStudio1.FadingVolumeChanged

' check if the player is fading-out

If audioDjStudio1.GetPlayerStatus(e.nPlayerIndex) = enumPlayerStatus.SOUND_PLAYING AndAlso m_nPlayerFadingOut = e.nPlayerIndex Then

' raise the transparent factor on the player fading out

audioDjStudio1.DisplayVideoMixer.PlayerAlphaSet (m_nMixerUniqueID, e.nPlayerIndex, 100 - e.fCurrVolume)

End If

End Sub

Visual C#

private void audioDjStudio1_SoundLoaded(object sender, AudioDjStudio.SoundLoadedEventArgs e)

{

// this is the next player to fade-in so put it at the bottom of the Z-Order

audioDjStudio1.DisplayVideoMixer.PlayerZOrderSet(m_nMixerUniqueID, e.nPlayerIndex, 100);

... do other stuffs

}

private void audioDjStudio1_FadeOutStarted(object sender, AudioDjStudio.PlayerEventArgs e)

{

// keep count of the player fading out because we will apply transparency on its video stream

m_nPlayerFadingOut = e.nPlayerIndex;

}

private void audioDjStudio1_FadeOutCompleted(object sender, AudioDjStudio.PlayerEventArgs e)

{

// the player is no more fading

m_nPlayerFadingOut = -1;

}

private void audioDjStudio1_FadeInCompleted(object sender, AudioDjStudio.PlayerEventArgs e)

{

// fade in has been completed so put the player at the top of the Z-Order

audioDjStudio1.DisplayVideoMixer.PlayerZOrderSet(m_nMixerUniqueID, e.nPlayerIndex, 0);

}

private void audioDjStudio1_FadingVolumeChanged(object sender, AudioDjStudio.FadingEventArgs e)

{

// check if the player is fading-out

if (audioDjStudio1.GetPlayerStatus(e.nPlayerIndex) == enumPlayerStatus.SOUND_PLAYING && m_nPlayerFadingOut == e.nPlayerIndex)

// raise the transparent factor on the player fading out

audioDjStudio1.DisplayVideoMixer.PlayerAlphaSet(m_nMixerUniqueID, e.nPlayerIndex, (int)(100.0 - e.fCurrVolume));

}

This is one of the possible approaches but you could obviously implement a totally different way to manage the Z-Order and the alpha channel transparency. As a final note, please, keep count that code snippets above don't contain strong error checking: in a production environment it's recommended to add some check on values returned by various methods calls.

A sample of use of the VideoMixer object in conjunction with the Fader object in Visual Basic.NET and Visual C# can be found inside the following sample installed with the product's setup package:

- VideoMixerWithFader