# Special blocks

# Introduction

Special blocks are blocks that do not fit into any other category.

# Null Renderer

The null renderer block sends the data to null. This block may be required if your block has outputs you do not want to use.

# Block info

Name: NullRendererBlock.

Pin direction	Media type	Pins count
Input	Any	1

# The sample pipeline

The sample pipeline is shown below. It reads a file and sends the video data to the video samples grabber, where you can grab each video frame after decoding. The Null renderer block is used to end the pipeline.

graph LR;
    UniversalSourceBlock-->VideoSampleGrabberBlock;
    VideoSampleGrabberBlock-->NullRendererBlock;

# Sample code

private void Start()
{
  // create the pipeline
  var pipeline = new MediaBlocksPipeline();

  // create universal source block
  var filename = "test.mp4";
  var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri(filename)));

  // create video sample grabber block and add the event handler
  var sampleGrabber = new VideoSampleGrabberBlock();
  sampleGrabber.OnVideoFrameBuffer += sampleGrabber_OnVideoFrameBuffer;

  // create null renderer block
  var nullRenderer = new NullRendererBlock();

  // connect blocks
  pipeline.Connect(fileSource.VideoOutput, sampleGrabber.Input);        
  pipeline.Connect(sampleGrabber.Output, nullRenderer.Input);   

  // start the pipeline
  await pipeline.StartAsync();
}

private void sampleGrabber_OnVideoFrameBuffer(object sender, VideoFrameXBufferEventArgs e)
{
    // received new video frame
}

# Platforms

Windows, macOS, Linux, iOS, Android.

# Tee

The tee block splits the video or audio data stream into multiple streams that completely copy the original stream.

# Block info

Name: TeeBlock.

Pin direction	Media type	Pins count
Input	Any	1
Output	Same as input	2 or more

# The sample pipeline

graph LR;
    UniversalSourceBlock-->TeeBlock;
    TeeBlock-->VideoRendererBlock;
    TeeBlock-->H264EncoderBlock;
    H264EncoderBlock-->MP4SinkBlock;

# Sample code

var pipeline = new MediaBlocksPipeline();

var filename = "test.mp4";
var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri(filename)));

var videoTee = new TeeBlock(2);
var h264Encoder = new H264EncoderBlock(new OpenH264EncoderSettings());
var mp4Muxer = new MP4SinkBlock(new MP4SinkSettings(@"output.mp4"));
var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);

pipeline.Connect(fileSource.VideoOutput, videoTee.Input);
pipeline.Connect(videoTee.Outputs[0], videoRenderer.Input);
pipeline.Connect(videoTee.Outputs[1], h264Encoder.Input);
pipeline.Connect(h264Encoder.Output, mp4Muxer.CreateNewInput(MediaBlockPadMediaType.Video));

await pipeline.StartAsync();

# Sample applications

Simple Capture Demo

# Platforms

Windows, macOS, Linux, iOS, Android.

# Super MediaBlock

The Super MediaBlock allows you to combine multiple blocks into a single block.

# Block info

Name: SuperMediaBlock.

Pin direction	Media type	Pins count
Input	Any	1
Output	Any	1

# The sample pipeline

graph LR;
    VirtualVideoSourceBlock-->SuperMediaBlock;
    SuperMediaBlock-->NullRendererBlock;

Inside the SuperMediaBlock:

graph LR;
    FishEyeBlock-->ColorEffectsBlock;

Final pipeline:

graph LR;
    VirtualVideoSourceBlock-->FishEyeBlock;
    subgraph SuperMediaBlock
    FishEyeBlock-->ColorEffectsBlock;
    end
    ColorEffectsBlock-->NullRendererBlock;

# Sample code

var pipeline = new MediaBlocksPipeline();

var videoViewBlock = new VideoRendererBlock(pipeline, VideoView1);

var videoSource = new VirtualVideoSourceBlock(new VirtualVideoSourceSettings());

var colorEffectsBlock = new ColorEffectsBlock(VisioForge.Core.Types.X.VideoEffects.ColorEffectsPreset.Sepia);
var fishEyeBlock = new FishEyeBlock();

var superBlock = new SuperMediaBlock();
superBlock.Blocks.Add(fishEyeBlock);
superBlock.Blocks.Add(colorEffectsBlock);
superBlock.Configure(pipeline);

pipeline.Connect(videoSource.Output, superBlock.Input);
pipeline.Connect(superBlock.Output, videoViewBlock.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# AESCipher

The AESCipher enum defines the types of AES ciphers that can be used. (Source: VisioForge.Core/Types/X/Special/AESCipher.cs)

# Enum Values

AES_128: AES 128-bit cipher key using CBC method.
AES_256: AES 256-bit cipher key using CBC method.

# Platforms

Windows, macOS, Linux, iOS, Android.

# EncryptorDecryptorSettings

The EncryptorDecryptorSettings class holds settings for encryption and decryption operations. (Source: VisioForge.Core/Types/X/Special/EncryptorDecryptorSettings.cs)

# Properties

Cipher (AESCipher): Gets or sets the AES cipher type. Defaults to AES_128.
Key (string): Gets or sets the encryption key.
IV (string): Gets or sets the initialization vector (16 bytes as hex).
SerializeIV (bool): Gets or sets a value indicating whether to serialize the IV.

# Constructor

EncryptorDecryptorSettings(string key, string iv): Initializes a new instance with the given key and initialization vector.

# Platforms

Windows, macOS, Linux, iOS, Android.

# CustomMediaBlockPad

The CustomMediaBlockPad class defines information for a pad within a CustomMediaBlock. (Source: VisioForge.Core/Types/X/Special/CustomMediaBlockPad.cs)

# Properties

Direction (MediaBlockPadDirection): Gets or sets the pad direction (input/output).
MediaType (MediaBlockPadMediaType): Gets or sets the media type of the pad (e.g., Audio, Video).
CustomCaps (Gst.Caps): Gets or sets custom GStreamer capabilities for an output pad.

# Constructor

CustomMediaBlockPad(MediaBlockPadDirection direction, MediaBlockPadMediaType mediaType): Initializes a new instance with the specified direction and media type.

# Platforms

Windows, macOS, Linux, iOS, Android.

# CustomMediaBlockSettings

The CustomMediaBlockSettings class provides settings for configuring a custom media block, potentially wrapping GStreamer elements. (Source: VisioForge.Core/Types/X/Special/CustomMediaBlockSettings.cs)

# Properties

ElementName (string): Gets the name of the GStreamer element or Media Blocks SDK element. To create a custom GStreamer Bin, include square brackets, e.g., "[ videotestsrc ! videoconvert ]".
UsePadAddedEvent (bool): Gets or sets a value indicating whether to use the pad-added event for dynamically created GStreamer pads.
ElementParams (Dictionary<string, object>): Gets the parameters for the element.
Pads (List<CustomMediaBlockPad>): Gets the list of CustomMediaBlockPad definitions for the block.
ListProperties (bool): Gets or sets a value indicating whether to list element properties to the Debug window after creation. Defaults to false.

# Constructor

CustomMediaBlockSettings(string elementName): Initializes a new instance with the specified element name.

# Platforms

Windows, macOS, Linux, iOS, Android.

# InputSelectorSyncMode

The InputSelectorSyncMode enum defines how an input-selector (used by SourceSwitchSettings) synchronizes buffers when in sync-streams mode. (Source: VisioForge.Core/Types/X/Special/SourceSwitchSettings.cs)

# Enum Values

ActiveSegment (0): Sync using the current active segment.
Clock (1): Sync using the clock.

# Platforms

Windows, macOS, Linux, iOS, Android.

# SourceSwitchSettings

The SourceSwitchSettings class configures a block that can switch between multiple input sources. (Source: VisioForge.Core/Types/X/Special/SourceSwitchSettings.cs)

The summary "Represents the currently active sink pad" from the code comment might be slightly misleading or incomplete for the class name SourceSwitchSettings. The properties suggest it's for configuring a source switcher.

# Properties

PadsCount (int): Gets or sets the number of input pads. Defaults to 2.
DefaultActivePad (int): Gets or sets the initially active sink pad.
CacheBuffers (bool): Gets or sets whether the active pad caches buffers to avoid missing frames when reactivated. Defaults to false.
DropBackwards (bool): Gets or sets whether to drop buffers that go backwards relative to the last output buffer pre-switch. Defaults to false.
SyncMode (InputSelectorSyncMode): Gets or sets how the input-selector syncs buffers in sync-streams mode. Defaults to InputSelectorSyncMode.ActiveSegment.
SyncStreams (bool): Gets or sets if all inactive streams will be synced to the running time of the active stream or to the current clock. Defaults to true.
CustomName (string): Gets or sets a custom name for logging purposes. Defaults to "SourceSwitch".

# Constructor

SourceSwitchSettings(int padsCount = 2): Initializes a new instance, optionally specifying the number of pads.

# Platforms

Windows, macOS, Linux, iOS, Android.

# Queue Block

The Queue block provides buffering between pipeline elements to smooth out processing variations and enable asynchronous data flow.

# Block info

Name: QueueBlock.

Pin direction	Media type	Pins count
Input	any	1
Output	any	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var queue = new QueueBlock();
pipeline.Connect(fileSource.VideoOutput, queue.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(queue.Output, videoRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# MultiQueue Block

The MultiQueue block provides synchronized buffering for multiple streams, essential for maintaining A/V sync in complex pipelines.

# Block info

Name: MultiQueueBlock.

Pin direction	Media type	Pins count
Input	any	multiple
Output	any	multiple

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var multiqueue = new MultiQueueBlock();
var videoInput = multiqueue.CreateNewInput(MediaBlockPadMediaType.Video);
var audioInput = multiqueue.CreateNewInput(MediaBlockPadMediaType.Audio);

pipeline.Connect(fileSource.VideoOutput, videoInput);
pipeline.Connect(fileSource.AudioOutput, audioInput);

// Connect outputs to encoders/renderers
await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Source Switch

The SourceSwitch block allows dynamic switching between multiple input sources without interrupting the pipeline.

# Block info

Name: SourceSwitchBlock.

Pin direction	Media type	Pins count
Input	any	multiple
Output	any	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var source1 = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("video1.mp4")));
var source2 = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("video2.mp4")));

var switchSettings = new SourceSwitchSettings(2) { DefaultActivePad = 0 };
var sourceSwitch = new SourceSwitchBlock(switchSettings);

pipeline.Connect(source1.VideoOutput, sourceSwitch.Input);
pipeline.Connect(source2.VideoOutput, sourceSwitch.CreateNewInput(MediaBlockPadMediaType.Video));

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(sourceSwitch.Output, videoRenderer.Input);

await pipeline.StartAsync();

// Switch to second source
sourceSwitch.SetActivePad(1);

# Platforms

Windows, macOS, Linux, iOS, Android.

# Universal Decoder

The UniversalDecoder block automatically detects and decodes various compressed audio and video formats.

# Block info

Name: UniversalDecoderBlock.

Pin direction	Media type	Pins count
Input	compressed media	1
Output	uncompressed media	multiple

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var decoder = new UniversalDecoderBlock();
pipeline.Connect(fileSource.VideoOutput, decoder.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(decoder.VideoOutput, videoRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Universal Demux Decoder

The UniversalDemuxDecoder block combines demuxing and decoding in a single block for simplified pipeline construction.

# Block info

Name: UniversalDemuxDecoderBlock.

Pin direction	Media type	Pins count
Input	container format	1
Output video	uncompressed video	1
Output audio	uncompressed audio	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var demuxDecoder = new UniversalDemuxDecoderBlock("test.mp4");

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(demuxDecoder.VideoOutput, videoRenderer.Input);

var audioRenderer = new AudioRendererBlock();
pipeline.Connect(demuxDecoder.AudioOutput, audioRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Barcode Detector

The BarcodeDetector block detects and decodes various barcode formats in video streams.

# Block info

Name: BarcodeDetectorBlock.

Pin direction	Media type	Pins count
Input	uncompressed video	1
Output	uncompressed video	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var videoSource = new SystemVideoSourceBlock(videoSettings);

var barcodeDetector = new BarcodeDetectorBlock();
barcodeDetector.OnBarcodeDetected += (sender, args) =>
{
    Console.WriteLine($"Barcode detected: {args.Type} - {args.Data}");
};
pipeline.Connect(videoSource.Output, barcodeDetector.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(barcodeDetector.Output, videoRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# DataMatrix Decoder

The DataMatrixDecoder block detects and decodes DataMatrix 2D barcodes in video streams.

# Block info

Name: DataMatrixDecoderBlock.

Pin direction	Media type	Pins count
Input	uncompressed video	1
Output	uncompressed video	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var videoSource = new SystemVideoSourceBlock(videoSettings);

var dataMatrixDecoder = new DataMatrixDecoderBlock();
dataMatrixDecoder.OnDataMatrixDetected += (sender, args) =>
{
    Console.WriteLine($"DataMatrix detected: {args.Data}");
};
pipeline.Connect(videoSource.Output, dataMatrixDecoder.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(dataMatrixDecoder.Output, videoRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Frame Doubler

The FrameDoubler block duplicates video frames to increase the effective frame rate.

# Block info

Name: FrameDoublerBlock.

Pin direction	Media type	Pins count
Input	uncompressed video	1
Output	uncompressed video	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var frameDoubler = new FrameDoublerBlock();
pipeline.Connect(fileSource.VideoOutput, frameDoubler.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(frameDoubler.Output, videoRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Video Enhancement

The VideoEnhancement block applies AI-based video enhancement and upscaling.

# Block info

Name: VideoEnhancementBlock.

Pin direction	Media type	Pins count
Input	uncompressed video	1
Output	uncompressed video	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var enhancementSettings = new VideoEnhancementSettings
{
    UpscaleFactor = 2,
    DenoiseStrength = 0.5
};
var enhancement = new VideoEnhancementBlock(enhancementSettings);
pipeline.Connect(fileSource.VideoOutput, enhancement.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(enhancement.Output, videoRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux (requires appropriate AI models).

# Decode Bin

The DecodeBin block automatically selects and manages appropriate decoders for media streams.

# Block info

Name: DecodeBinBlock.

Pin direction	Media type	Pins count
Input	compressed media	1
Output	uncompressed media	dynamic

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var decodeBin = new DecodeBinBlock();
pipeline.Connect(fileSource.VideoOutput, decodeBin.Input);

// DecodeBin will create output pads dynamically as streams are discovered
await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Parse Bin

The ParseBin block automatically parses media streams and exposes elementary streams.

# Block info

Name: ParseBinBlock.

Pin direction	Media type	Pins count
Input	compressed media	1
Output	parsed streams	dynamic

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var parseBin = new ParseBinBlock();
pipeline.Connect(fileSource.VideoOutput, parseBin.Input);

// ParseBin will create output pads for each discovered stream
await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Custom Transform

The CustomTransform block allows integration of custom transformation logic into the pipeline.

# Block info

Name: CustomTransformBlock.

Pin direction	Media type	Pins count
Input	any	1
Output	any	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var customSettings = new CustomTransformSettings
{
    ElementName = "videoscale", // GStreamer element name
    Properties = new Dictionary<string, object>
    {
        { "method", 0 }
    }
};
var customTransform = new CustomTransformBlock(customSettings);
pipeline.Connect(fileSource.VideoOutput, customTransform.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(customTransform.Output, videoRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Data Sample Grabber

The DataSampleGrabber block intercepts and provides access to data buffers flowing through the pipeline.

# Block info

Name: DataSampleGrabberBlock.

Pin direction	Media type	Pins count
Input	any	1
Output	any	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var dataSG = new DataSampleGrabberBlock();
dataSG.OnDataBuffer += (sender, args) =>
{
    // Process buffer data
    var bufferSize = args.BufferSize;
    var bufferData = args.BufferData;
};
pipeline.Connect(fileSource.VideoOutput, dataSG.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(dataSG.Output, videoRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Debug Timestamp

The DebugTimestamp block logs detailed timestamp information for debugging synchronization issues.

# Block info

Name: DebugTimestampBlock.

Pin direction	Media type	Pins count
Input	any	1
Output	any	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync(new Uri("test.mp4")));

var debugTimestamp = new DebugTimestampBlock();
pipeline.Connect(fileSource.VideoOutput, debugTimestamp.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(debugTimestamp.Output, videoRenderer.Input);

await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.

# Data Processor

The DataProcessor block provides custom data processing capabilities for non-standard data formats.

# Block info

Name: DataProcessorBlock.

Pin direction	Media type	Pins count
Input	any	1
Output	any	1

# Sample code

var pipeline = new MediaBlocksPipeline();

var dataSource = new CustomDataSource();

var dataProcessor = new DataProcessorBlock();
pipeline.Connect(dataSource.Output, dataProcessor.Input);

// Process and forward data
await pipeline.StartAsync();

# Platforms

Windows, macOS, Linux, iOS, Android.