Apple Platform Blocks - VisioForge Media Blocks SDK .Net¶

Media Blocks SDK .Net

This section covers MediaBlocks specifically optimized for Apple platforms (iOS, macOS, tvOS).

Available Blocks¶

Audio Sources¶

• SystemAudioSourceBlock: macOS and iOS audio capture through Core Audio, configured with OSXAudioSourceSettings on macOS and Mac Catalyst, IOSAudioSourceSettings on iOS
• See Audio Sources Documentation

Audio Sinks¶

• OSXAudioSinkBlock: macOS audio playback

• See Audio Rendering Documentation

• IOSAudioSinkBlock: iOS audio playback

• See Audio Rendering Documentation

Video Sources¶

• IOSVideoSourceBlock: iOS camera capture
• See Video Sources Documentation

Video Encoders¶

• AppleProResEncoderBlock: Apple ProRes professional video codec
• See ProRes Encoder Documentation

Video Processing¶

• MetalVideoCompositorBlock: GPU-accelerated multi-input video compositor using Apple Metal
• MetalConvertScaleBlock: GPU-accelerated format conversion and scaling using Apple Metal
• MetalDeinterlaceBlock: GPU-accelerated deinterlacing using Apple Metal
• MetalOverlayBlock: GPU-accelerated image overlay using Apple Metal
• MetalTransformBlock: GPU-accelerated flip, rotate, and crop using Apple Metal
• MetalVideoFilterBlock: GPU-accelerated brightness, contrast, saturation, hue, gamma, sharpness, sepia, invert, film grain, and vignette using Apple Metal

Metal Video Compositor¶

Metal Video Compositor Block¶

The MetalVideoCompositorBlock composites multiple video streams in real time using the Apple Metal GPU framework. Each input stream has configurable position, size, z-order, alpha, and blend operator. The block produces a single BGRA video output.

Block info¶

Name: MetalVideoCompositorBlock.

Settings¶

The block takes a MetalVideoCompositorSettings instance:

Each input stream is a MetalVideoMixerStream:

The sample pipeline¶

graph LR;
    VideoSource1-->MetalVideoCompositorBlock;
    VideoSource2-->MetalVideoCompositorBlock;
    MetalVideoCompositorBlock-->VideoRendererBlock;

Sample code¶

var pipeline = new MediaBlocksPipeline();

// Configure compositor: 1920x1080 @ 30fps
var settings = new MetalVideoCompositorSettings(1920, 1080, VideoFrameRate.FPS_30);

// First stream: left half of screen
settings.AddStream(new MetalVideoMixerStream(
    rectangle: new Rect(0, 0, 960, 1080),
    zorder: 0));

// Second stream: right half of screen
// Rect ctor is (left, top, right, bottom). For the right half of a 1920x1080
// canvas use right=1920 and bottom=1080 — the previous form (960, 0, 960, 1080)
// has left==right and produces a zero-width box.
settings.AddStream(new MetalVideoMixerStream(
    rectangle: new Rect(960, 0, 1920, 1080),
    zorder: 1));

var compositor = new MetalVideoCompositorBlock(settings);

// Render composited output
var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(compositor.Output, videoRenderer.Input);

await pipeline.StartAsync();

// Real-time: fade out stream 0 over 2 seconds
compositor.StartFadeOut(settings.Streams[0].ID, TimeSpan.FromSeconds(2));

Availability¶

bool available = MetalVideoCompositorBlock.IsAvailable();

Returns true if the vfmetalcompositor GStreamer plugin is available on the current system.

Platforms¶

macOS, iOS. On iOS the Metal elements are part of the device runtime only - they are not available in the iOS Simulator, where IsAvailable() returns false.

Metal Convert/Scale¶

Metal Convert/Scale Block¶

The MetalConvertScaleBlock performs pixel-format conversion and scaling in a single GPU pass using the Apple Metal framework. It can optionally add letterbox/pillarbox borders to preserve the source aspect ratio.

Block info¶

Name: MetalConvertScaleBlock.

Settings¶

The block takes a MetalConvertScaleSettings instance:

The sample pipeline¶

graph LR;
    VideoSource-->MetalConvertScaleBlock;
    MetalConvertScaleBlock-->VideoRendererBlock;

Sample code¶

var pipeline = new MediaBlocksPipeline();

var videoSource = new IOSVideoSourceBlock(videoSettings);

// Bilinear scaling with letterbox borders
var settings = new MetalConvertScaleSettings
{
    Method = MetalScalingMethod.Bilinear,
    AddBorders = true,
    BorderColor = 0xFF000000
};
var convertScale = new MetalConvertScaleBlock(settings);
pipeline.Connect(videoSource.Output, convertScale.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(convertScale.Output, videoRenderer.Input);

await pipeline.StartAsync();

Availability¶

bool available = MetalConvertScaleBlock.IsAvailable();

Returns true if the vfmetalconvertscale GStreamer plugin is available on the current system.

Platforms¶

macOS, iOS. On iOS the Metal elements are part of the device runtime only - they are not available in the iOS Simulator, where IsAvailable() returns false.

Metal Deinterlace¶

Metal Deinterlace Block¶

The MetalDeinterlaceBlock removes interlacing artifacts on the GPU using the Apple Metal framework. It supports bob, weave, linear, and greedy-H (motion-adaptive) algorithms.

Block info¶

Name: MetalDeinterlaceBlock.

Settings¶

The block takes a MetalDeinterlaceSettings instance:

The sample pipeline¶

graph LR;
    VideoSource-->MetalDeinterlaceBlock;
    MetalDeinterlaceBlock-->VideoRendererBlock;

Sample code¶

var pipeline = new MediaBlocksPipeline();

var videoSource = new IOSVideoSourceBlock(videoSettings);

// Motion-adaptive greedy-H deinterlacing
var settings = new MetalDeinterlaceSettings
{
    Method = MetalDeinterlaceMethod.GreedyH,
    FieldLayout = MetalDeinterlaceFieldLayout.Auto,
    MotionThreshold = 0.1
};
var deinterlace = new MetalDeinterlaceBlock(settings);
pipeline.Connect(videoSource.Output, deinterlace.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(deinterlace.Output, videoRenderer.Input);

await pipeline.StartAsync();

Availability¶

bool available = MetalDeinterlaceBlock.IsAvailable();

Returns true if the vfmetaldeinterlace GStreamer plugin is available on the current system.

Platforms¶

macOS, iOS. On iOS the Metal elements are part of the device runtime only - they are not available in the iOS Simulator, where IsAvailable() returns false.

Metal Overlay¶

Metal Overlay Block¶

The MetalOverlayBlock composites a PNG or JPEG image onto video frames on the GPU using the Apple Metal framework. The overlay can be positioned in absolute pixels or as a fraction of the frame size, with adjustable opacity.

Block info¶

Name: MetalOverlayBlock.

Settings¶

The block takes a MetalOverlaySettings instance:

The sample pipeline¶

graph LR;
    VideoSource-->MetalOverlayBlock;
    MetalOverlayBlock-->VideoRendererBlock;

Sample code¶

var pipeline = new MediaBlocksPipeline();

var videoSource = new IOSVideoSourceBlock(videoSettings);

// Logo at top-left with 80% opacity
var settings = new MetalOverlaySettings
{
    Location = "logo.png",
    X = 20,
    Y = 20,
    Alpha = 0.8
};
var overlay = new MetalOverlayBlock(settings);
pipeline.Connect(videoSource.Output, overlay.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(overlay.Output, videoRenderer.Input);

await pipeline.StartAsync();

Availability¶

bool available = MetalOverlayBlock.IsAvailable();

Returns true if the vfmetaloverlay GStreamer plugin is available on the current system.

Platforms¶

macOS, iOS. On iOS the Metal elements are part of the device runtime only - they are not available in the iOS Simulator, where IsAvailable() returns false.

Metal Transform¶

Metal Transform Block¶

The MetalTransformBlock applies flip, rotate, and crop operations on the GPU using the Apple Metal framework.

Block info¶

Name: MetalTransformBlock.

Settings¶

The block takes a MetalTransformSettings instance:

The sample pipeline¶

graph LR;
    VideoSource-->MetalTransformBlock;
    MetalTransformBlock-->VideoRendererBlock;

Sample code¶

var pipeline = new MediaBlocksPipeline();

var videoSource = new IOSVideoSourceBlock(videoSettings);

// Rotate 90 degrees clockwise and crop 10px from each side
var settings = new MetalTransformSettings
{
    Method = MetalTransformMethod.Clockwise,
    CropLeft = 10,
    CropRight = 10
};
var transform = new MetalTransformBlock(settings);
pipeline.Connect(videoSource.Output, transform.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(transform.Output, videoRenderer.Input);

await pipeline.StartAsync();

Availability¶

bool available = MetalTransformBlock.IsAvailable();

Returns true if the vfmetaltransform GStreamer plugin is available on the current system.

Platforms¶

macOS, iOS. On iOS the Metal elements are part of the device runtime only - they are not available in the iOS Simulator, where IsAvailable() returns false.

Metal Video Filter¶

Metal Video Filter Block¶

The MetalVideoFilterBlock applies color and image adjustments on the GPU using the Apple Metal framework. Brightness, contrast, saturation, hue, gamma, sharpness, sepia, invert, film grain, vignette, chroma key and a 3D LUT are all applied in a single GPU pass.

Block info¶

Name: MetalVideoFilterBlock.

Settings¶

The block takes a MetalVideoFilterSettings instance:

Call Settings.Update() after changing a property to apply it to a running pipeline.

The sample pipeline¶

graph LR;
    VideoSource-->MetalVideoFilterBlock;
    MetalVideoFilterBlock-->VideoRendererBlock;

Sample code¶

var pipeline = new MediaBlocksPipeline();

var videoSource = new IOSVideoSourceBlock(videoSettings);

// Warmer, slightly punchier picture with a soft vignette
var settings = new MetalVideoFilterSettings
{
    Brightness = 0.05,
    Contrast = 1.15,
    Saturation = 1.2,
    Vignette = 0.3
};
var filter = new MetalVideoFilterBlock(settings);
pipeline.Connect(videoSource.Output, filter.Input);

var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(filter.Output, videoRenderer.Input);

await pipeline.StartAsync();

Availability¶

bool available = MetalVideoFilterBlock.IsAvailable();

Returns true if the vfmetalvideofilter GStreamer plugin is available on the current system.

Platforms¶

macOS, iOS. On iOS the Metal elements are part of the device runtime only - they are not available in the iOS Simulator, where IsAvailable() returns false.

Platform Requirements¶

• iOS: iOS 12.0 or later
• macOS: macOS 10.13 or later
• tvOS: tvOS 12.0 or later

Features¶

• Native integration with Apple frameworks (AVFoundation, Core Audio, Core Video)
• Hardware-accelerated processing on Apple Silicon and Intel Macs
• Optimized for low power consumption on mobile devices
• Support for high-quality ProRes encoding
• Integration with iOS camera and microphone permissions

Sample Code¶

iOS Camera Capture¶

var pipeline = new MediaBlocksPipeline();

// iOS video source
var videoSource = new IOSVideoSourceBlock(videoSettings);

// Process and display
var videoRenderer = new VideoRendererBlock(pipeline, VideoView1);
pipeline.Connect(videoSource.Output, videoRenderer.Input);

await pipeline.StartAsync();

macOS Audio Capture and Playback¶

var pipeline = new MediaBlocksPipeline();

// macOS audio source
var audioSource = new SystemAudioSourceBlock(audioSettings);

// macOS audio sink
var audioSink = new OSXAudioSinkBlock();
pipeline.Connect(audioSource.Output, audioSink.Input);

await pipeline.StartAsync();

ProRes Encoding¶

var pipeline = new MediaBlocksPipeline();

var fileSource = new UniversalSourceBlock(await UniversalSourceSettings.CreateAsync("input.mp4"));

// Apple ProRes encoder
// AppleProResEncoderSettings exposes Quality (double 0.0-1.0), Bitrate, MaxKeyframeInterval,
// MaxKeyFrameIntervalDuration, AllowFrameReordering, PreserveAlpha, Realtime — not a named profile enum.
var proresSettings = new AppleProResEncoderSettings
{
    Quality = 0.8
};
var proresEncoder = new AppleProResEncoderBlock(proresSettings);
pipeline.Connect(fileSource.VideoOutput, proresEncoder.Input);

// Output to MOV file
var movSink = new MOVSinkBlock(new MOVSinkSettings("output.mov"));
pipeline.Connect(proresEncoder.Output, movSink.CreateNewInput(MediaBlockPadMediaType.Video));

await pipeline.StartAsync();

Platforms¶

iOS, macOS, tvOS.

Related Documentation¶

• Sources - All source blocks including Apple-specific
• VideoEncoders - Video encoding including ProRes
• AudioRendering - Audio playback