OpenVJ

Real-time video compositing and clip triggering for live performance.

One binary. No runtime plugins. The GPU compositor runs on wgpu, media on GStreamer, audio on CPAL, the interface on egui.

Early development. The engine works today: composition model, effect chain, audio mixer, clip playback, output routing. ROADMAP.md tracks the rest.

Features

• Composition model. A composition is cut into scenes. A scene holds groups, groups hold layers, layers hold clips in columns. Trigger one clip or a whole column at once. Opacity, blend mode, bypass and solo exist at every level. A slot can be an explicit clear, so a column can blank the layers above it. Each scene has a palette of named colours; recolour a whole set of generators in one edit.
• Scenes. Tabs above the grid, one per part of the show. Each carries its own columns, groups, layers, clips, palette, master effect chain, master opacity and auto-advance programme. The composition keeps only what the whole show shares: name, resolution, tempo. A tab is where the operator works, not what the audience sees. The output shows the scene that is playing. It changes only when a clip or column is triggered. Prep the next scene in another tab while the current one is live, then fire one of its columns to cut across. Only the on-air scene is composited and heard; other tabs keep their playing clips, so returning finds them untouched. Add, rename, duplicate, drag to reorder, or remove a scene, but never the last one.
• Video. All layers and groups composited on the GPU in one frame graph, with 20 blend modes. Clips play video files, images, image sequences, generated sources or capture devices, scaled to fit, fill or stretch.
• Live input. Cameras and capture cards as video clips. Microphones, line inputs and desktop audio as sound clips. All placed in the grid from the slot menu.
• Effects. Chains on clips, layers, groups and the master. Built from 30+ WGSL passes across colour, blur, stylize, distort and keying, each with typed parameters.
• Animation. Any parameter or object property can follow live audio (peak, RMS, or a frequency band you pick on the spectrum) or a cycle from an oscillator, envelope or hand-placed points, clocked in seconds, beats or clip time. Each binding reports what its amount is worth, in the value's own units and in pixels where it is a distance. No guessing from a bare fraction.
• Audio. Clip audio decoded and mixed in sync with video. Per-clip gain, per-layer gain, pan, mute, solo, a master fader and metering. Microphones, line inputs and the machine's own output are clips too, so live sound triggers from the grid. Each clip routes its audio to the output, the animators, both or neither. Play a video with its soundtrack muted, or capture a room mic purely to drive the visuals. Cueing a clip can feed the analyser without feeding the speakers, so audio-reactive amounts are set against the real track in silence.
• Interface. Scene tabs over a clip grid with thumbnails. Cue-to-preview lines up what comes next. Drag and drop from the file manager, on-canvas clip placement, and a parameter inspector. Tool panels dock to any edge or tear out into their own windows. Every keyboard shortcut is rebindable.
• Output. Any number of outputs, each with its own source, monitor, resolution and fullscreen or windowed mode. Monitors are re-enumerated on demand.
• External control. A JSON REST and WebSocket API over TCP or a Unix socket, exposing everything the interface can do. A batch endpoint applies a run of edits as one undo step. OSC on the address tree hardware controllers and show-control software already speak. OSCQuery with mDNS discovery. Pixel-mapped output over Art-Net, sACN, DDP and Open Pixel Control. Lighting consoles can browse the grid and pull a live feed over CITP/MSEX.
• Headless. openvj --serve runs the whole engine with no window, driven over the network. Snapshot and recording endpoints let an operator or script see the output.

Download

Builds are attached to their release on git.raccoon.sk/martin/openvj.

Platform
Windows	openvj-1.0.0-windows-x86_64-setup.exe	installer, or a portable .zip
Linux	—	build from source for now
macOS	—	build from source for now

The Windows build carries its own media runtime, so there is nothing to install first. See THIRD-PARTY-NOTICES for what it contains and its terms. Verify a download against the .sha256 beside it.

Building

Requires Rust 1.85+, GStreamer 1.20+ with the base, good and libav plugin sets, and a GPU with Vulkan, Metal, DX12 or OpenGL 3.3.

# Debian / Ubuntu
sudo apt install build-essential pkg-config libgstreamer1.0-dev \
    libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-good \
    gstreamer1.0-plugins-bad gstreamer1.0-libav libasound2-dev

# Fedora
sudo dnf install gcc pkgconf-pkg-config gstreamer1-devel \
    gstreamer1-plugins-base-devel gstreamer1-plugins-good \
    gstreamer1-plugins-bad-free gstreamer1-libav alsa-lib-devel

# Arch
sudo pacman -S base-devel pkgconf gstreamer gst-plugins-base gst-plugins-good \
    gst-plugins-bad gst-libav alsa-lib

# macOS
brew install gstreamer gst-plugins-base gst-plugins-good gst-plugins-bad gst-libav

On Windows, install the GStreamer MSVC runtime and development packages from https://gstreamer.freedesktop.org/download/. Put %GSTREAMER_1_0_ROOT_MSVC_X86_64%\bin on PATH.

Then:

cargo run --release                          # start the application
cargo run --release -- path/to/project.ovj   # open a project
cargo run --release -- --list-outputs        # print monitors, audio and capture devices

Settings

File → Settings…, or Ctrl+,. Four tabs. All settings are per machine, not per project, so a show file opens the same way wherever it is taken:

• Interface — draw scale, analyser style, and whether panels return where they were left.
• Control — the ports below, edited behind an Apply button. Nothing reaches the network until asked.
• Keyboard — every shortcut, rebindable. Click a key to remove it, + to add one. A key already in use is moved, not shared, and the window says where it came from.
• About — version, build number, commit, compiler, GPU, driver, and the GStreamer install this machine has. One button copies the lot for a bug report.

Settings live beside the project files, not in them: preferences.ron for the interface and keys, layout.ron for the panels, control.ron for the ports. That is %APPDATA%\OpenVJ on Windows, ~/Library/Application Support/OpenVJ on macOS, $XDG_CONFIG_HOME/openvj elsewhere. The About tab names the directory.

External control

Nothing listens until asked. Enable a surface in the Control tab, with a flag for one run, or in control.ron to make it permanent:

openvj --api                       # REST + WebSocket on 127.0.0.1:7700
openvj --api 0.0.0.0:7700 --osc    # reachable on the network, plus OSC and OSCQuery
openvj --serve --api --socket /run/openvj.sock   # headless, no window
openvj --write-settings            # write the resolved settings to control.ron

openvjctl drives a running instance from a shell or a cue script:

openvjctl status                                   # what is listening, and how busy it is
openvjctl scene                                    # the scenes, and which one is on the output
openvjctl scene 2                                  # open another tab; nothing is triggered
openvjctl grid                                     # the clip grid as a table
openvjctl trigger column 3                         # fire a column
openvjctl set /composition/layers/1/video/opacity 0.5
openvjctl snapshot frame.png --max-edge 640        # what is on the output right now
openvjctl record --frames 120 --dir /tmp/take      # a run of frames, as PNGs
openvjctl watch                                    # follow the composition as it changes

The OSC and OSCQuery address tree mirrors the namespace the rest of the VJ world already uses: /composition/layers/1/clips/3/connect, /composition/layers/1/video/opacity. An existing controller layout works unchanged. Indices are one-based. Layer 1 is the bottom layer. Continuous values are normalised 0.0..=1.0 over each parameter's range. A connect acts on the press and ignores the release. GET /api/address lists every address with its range and current value.

Scenes fit that namespace without disturbing it. What the whole show shares stays put: composition name, output size, tempo, audio buses, undo. Every leaf a scene owns resolves against the open tab, so a controller layout drawn up before scenes existed follows the tabs with nothing to rewire.

• /scene/... and /composition/selectedscene/... are shorter spellings of the open tab.
• /composition/scenes/<n>/... names one scene wherever the output is, with the whole subtree beneath it: a name, a select that opens the tab and triggers nothing, a selected that reads 1 for the open tab, and a connected that reads 1 for the scene on the output.

selected and connected differ while the next scene is prepared during the current one. That is why a controller gets both: one pad lights where the operator works, the other lights what the audience sees.

Controls that mean "the set as it stands" act only on the open tab and are refused elsewhere: disconnectall, cue/clear, the column connects, and advance/pause. A column connect carries a column but no scene, so accepting it elsewhere would fire the open tab's column under another tab's name. Firing a column cuts the output to the open tab, so a controller selects the scene, then fires: one deliberate act, then another. A clip connect is the exception. It is addressed by its own layer, and a layer belongs to exactly one scene, so it reaches into whichever scene owns that layer and cuts the output there.

Art-Net™ Designed by and Copyright Artistic Licence.

Not implemented, deliberately:

• NDI — its SDK licence requires terms this project cannot grant.
• Spout (Windows) and Syphon (macOS) — permissively licensed and worth having, but cannot be built or verified on a Linux workstation. They wait for a runner on those platforms.

Repository layout

Crate	Contents
openvj	Binary: engine loop, wiring, headless server, CLI
openvj-core	Composition model, transport, effects, animation, actions, project file
openvj-media	GStreamer decoding, frame and audio queues, thumbnails
openvj-render	wgpu compositor, blend modes, WGSL effect passes, output textures
openvj-audio	CPAL output, output and analysis buses, metering, analysis
openvj-ui	egui interface: clip grid, mixer strips, inspector, output settings
openvj-control	Action queue, state snapshots, frame taps, the shared address tree
openvj-http	REST and WebSocket API over TCP and Unix sockets
openvj-osc	OSC input and feedback, OSCQuery, mDNS discovery
openvj-net	Art-Net, sACN, DDP, Open Pixel Control, CITP/MSEX
openvj-cli	openvjctl, the command line client

Projects are plain-text RON (.ovj) and reference media by path, relative to the project file where possible, so a project directory can be copied between machines. A project written before scenes existed opens with its grid as its one scene, under the composition's own name, so nothing already programmed has to be rebuilt.

Contributing

See CONTRIBUTING.md.

Licence

Dual licensed under MIT or Apache-2.0, at your option.