Open Source,Remade with Rust

FFmpeg is the swiss-army knife of media — decode, encode, transcode, mux, probe, repackage just about any audio or video format ever shipped. It is also one of the largest C codebases in continuous production use, with two decades of CVEs in bitstream parsers, container demuxers, and codec arithmetic. Every browser that ships hardware video, every streaming service, every social media app depends on it — and inherits that surface.

ffmpeg (the remade_ffmpeg_rs repo) is a ground-up clean-room rebuild in safe Rust — not a wrapper around the C library, an entirely new codebase using the same wire protocols and file formats. The CLI is drop-in compatible: ffmpeg -i in.mp4 -c:v libaom -c:a libopus out.webm works the same way against upstream FFmpeg flags, plus an ffprobe binary for inspection. Image codecs (AVIF, PNG, JPEG, WebP, GIF, JPEG XL), audio (Opus, AAC, FLAC, Vorbis, PCM), video (AV1, H.264, VP9), and containers (MP4, WebM, AVI, WAV, OGG) all flow through one safe-Rust pipeline.

Apache-2.0 throughout the core library, CLI, and server — no GPL or LGPL dependencies, embeddable in closed-source products without the copyleft tax that upstream FFmpeg's GPL build carries. Many codecs already ship faster and higher-quality than upstream FFmpeg on the paths we've tuned; the project is pre-1.0 and honest about the full-parity work still ahead.

AI media processing today means gluing together half a dozen model runtimes — Whisper.cpp for speech-to-text, one of a dozen SDKs for text-to-speech, Tesseract for OCR, a fresh vision-language model every quarter for image and video understanding — each with its own build system, its own memory-safety story, and its own operational sharp edges. Every application that wants to touch audio, video, or image content ends up shipping a small archipelago of C and Python.

FFAI consolidates the four core AI media tasks into one Rust library and one CLI. ASR (automatic speech recognition) and TTS (text-to-speech) via the Mercury engine; OCR through Carmenta; VLM (vision-language model for image captioning and video understanding) via Argus. Common backbone is the Candle tensor framework; weights are downloaded from manifests rather than vendored, with per-model licensing surfaced transparently so you know what you're shipping. Streaming-capable pipelines throughout.

Library-first design — the ffai CLI is a thin surface over reusable crates, so anything the binary can do your app can call directly. Early benchmarks land at 15% faster than whisper.cpp with 7% better quality on the ASR path. Optional GPU acceleration behind feature flags; core still compiles and runs on CPU-only targets.

OpenAI Whisper became the reference for open-source speech recognition the moment it shipped — journalists, podcasters, enterprise transcription pipelines, and every offline voice interface reach for it first. The catch: every open-source Rust integration today either binds to whisper.cpp over FFI or shells out to the Python reference implementation, meaning a C or Python runtime sits on the hot path for one of the most latency-sensitive workloads an app can run.

Mercury is a pure-Rust reimplementation of Whisper on the Candle tensor framework. Custom mel-spectrogram pipeline, custom tokenization, custom decoding, hand-written AVX2 kernels — no Python runtime, no C/C++ dependencies by default. tiny.en and base.en models today; greedy decoding with full logit-filter grammar. Against whisper.cpp on LibriSpeech: 0.19 percentage points behind on clean speech, marginally ahead on noisy speech, roughly 1.12× slower on end-to-end throughput today — while running 1.38× faster than whisper.cpp's unfused encoder.

Dual-licensed MIT or Apache-2.0 for the code (model weights carry their own licenses). Mercury is the ASR engine that powers FFAI above and ships as a standalone crate so any Rust project can pull it in without the rest of the toolkit. Experimental today — beam search, streaming APIs, and larger models are next.

AV1 is the codec every streaming service and social platform is migrating to — better compression than H.264/H.265 at royalty-free terms — but the reference open-source implementations (rav1e for encode, dav1d for decode) leave real performance on the table on general workloads. Encoding AV1 at scale still means either paying for hardware acceleration or waiting.

rusty-av1-toolkit ships a performance-tuned pair: an rav1e encoder that runs ~1.10× faster with byte-identical output by default, plus a safe-Rust port of dav1d for decode. An optional --racecar flag unlocks about 1.69× encode speed by allowing bitstream changes on the hot paths where byte-identity is not the constraint. Turn it off, get stock rav1e output; turn it on, get the fast lane.

Fork of xiph/rav1e and the memorysafety-org rav1d, kept in sync with upstream. BSD-2-Clause on the toolkit itself; the AV1 patent grant is separately governed by the Alliance for Open Media Patent License. Drops in wherever rav1e or dav1d already do.

libopus has been the reference implementation of the Opus audio codec since 2012 — every voice call, every low-latency music stream, every WebRTC session touches it. It is also a battle-tested C codebase, which means every Rust project that wants Opus today reaches through FFI into a foreign memory-safety story, and every embedded target has to build the C toolchain to link it.

rusty-opus is a pure-Rust implementation of RFC 6716 (with RFC 8251 updates) — SILK for speech, CELT for music and low latency, Hybrid for the middle. Conformance-verified encoder and decoder. Zero C in the dependency tree, no FFI. The only unsafe code lives inside SIMD kernels: hand-written AVX2/FMA for x86-64 and NEON for ARM64, each gated behind runtime CPU-feature checks with scalar fallbacks.

Performance sits at or above libopus on CELT single-thread, and lands roughly 3× libopus on speech thanks to frame-parallel encoding. BSD-3-Clause licensed, derived from opus-rs and maintained as MATA Network's performance fork. Already integrated into the ffmpeg rebuild above for the Opus paths.

x264 and Cisco's openh264 are the two libraries every browser, every video conference, and every streaming app reaches for when it needs H.264. Both are massive C/C++ codebases, and between them they account for a long catalogue of CVEs in bitstream parsing, buffer handling, and motion-compensation arithmetic. Every app that links them inherits that surface.

rusty_h264 is a clean-room ground-up rebuild in pure Rust. The codec core is marked #![forbid(unsafe_code)] — zero unsafe blocks, no C in the dependency tree, no FFI. Bit-exact decoding against ffmpeg across the entire QP range (0–51) for both intra and inter frames, with rate control and motion compensation in the encoder. Constrained Baseline Profile today, with room to grow.

BSD-2-Clause licensed — permissive enough to embed in any product, no copyleft strings. Built specifically to be the codec layer for memory-safe video pipelines: the Comet host, the Starfire client, browser-side decoders, anything that should not ship a 25-year-old C buffer parser into 2026 production.

OAuth, OIDC, Auth0, Clerk, Firebase Auth — every off-the-shelf "Sign in with X" button on the web routes through a third-party provider you have to register with, configure client IDs for, and pay a per-monthly-active-user fee to once you cross their free tier. The provider sees every login, every site, every time.

Sovereign ID flips the model. The user's identity is a keypair on their own device; sign-in produces a token they cryptographically sign, and your backend verifies it entirely locally — no fetches to MATA, no JWKS endpoint, no DID resolver to call. Four npm packages cover the surface: browser SDK, backend verifier, statement verifier, and React bindings. MIT-licensed, zero external dependencies.

Live integration documented in the Freedom Guide. Powers the Sign in with Sovereign ID button you can drop onto any website with one component and a backend verifier call.

Most Rust web stacks today verify identity tokens by talking to an external authority — Auth0's JWKS endpoint, Clerk's verification API, AWS Cognito's token introspection. Every login is an extra round-trip out to a third party, and every outage on their side is an outage you inherit.

mID is the Rust counterpart to the Sovereign ID JavaScript verifier. A user's identity is a keypair they hold on their own device; sign-in produces a token that mID verifies entirely in-process — no I/O, no JWKS, no DID resolver. Works natively on every Rust target and via WebAssembly where native won't run. Dual-licensed Apache-2.0 OR MIT.

Pairs with the sovereign-id JavaScript SDK on the client — both speak the same wire format. Drop mID anywhere you would have called Auth0: Axum middleware, Actix extractor, an edge function, a desktop app. The verification stays in-process, the user stays in control of the keypair.

Cloud databases — Firebase, Supabase, DynamoDB, Postgres-on-RDS — assume always-on infrastructure, a single source of truth that lives in a data center, and a network round-trip for every read and write. The model breaks the moment a user goes offline, the network partitions, or the data needs to live near someone instead of in a US-East-1 rack.

SpaceDB is the opposite: a local-first database that stores encrypted app data across machines near the users who own it, with automatic CRDT convergence when replicas reconnect and offline-first operation by construction. Per-field consistency tiers (convergent, causal+, strong) so a single schema can mix eventual and strong consistency where each fits. Capability-based access control with expiry and budgets, built-in vector search that keeps the corpus local, deterministic compute-to-data with attestation, and honest freshness reporting.

Pluggable transport, storage, and crypto seams — operators implement whatever fits their environment, and no SpaceDB crate depends on any proprietary code. Dual-licensed Apache-2.0 OR MIT, free to embed in applications and run on personal devices indefinitely.

Moonlight is the open-source implementation of NVIDIA's GameStream protocol — the client that lets you stream a PC game session to a phone, tablet, Steam Deck, set-top box, or any other device on your network. It is excellent software, and like most game-streaming clients, it is written in C and C++ across a decade of accumulated networking, decoder, and input-handling code.

Starfire is the same idea, rebuilt from the ground up in Rust. The protocol surface stays compatible so existing GameStream hosts and Sunshine servers keep working, but the entire client — packet parser, video decoder bindings, input pipeline, network state machine — is memory-safe by construction. The classes of bug that historically ship as CVEs in game-streaming clients (buffer overflows in RTP parsing, integer overflows in frame-size handling, use-after-free in the decoder lifecycle) never compile in Starfire.

Built for the same hardware Moonlight runs on — desktop, mobile, embedded — with the same low-latency target. Active development; star the repo to follow along.

Sunshine is the open-source streaming host that powers most self-hosted GameStream setups today — capture the desktop, encode it with whatever hardware is available, deliver frames to a Moonlight client elsewhere on the network. It is GPLv3, written largely in C++, and carries the memory-safety surface that comes with that lineage.

Comet is the same role, rebuilt from scratch in Rust. Hardware-encode-first across NVENC, AMF, QuickSync, and VideoToolbox; a zero-copy capture-to-encode pipeline that keeps frames on the GPU; the same GameStream wire protocol so existing Moonlight clients keep working, plus a Comet-native mode that pairs with Starfire for the lowest end-to-end latency on the stack.

Measured against Sunshine + Moonlight on identical hardware: 2.1× better host latency, 3.8× better decode latency, and a sub-millisecond capture path. Shipping target is Windows desktop and gaming workloads first, with macOS host support tracking VideoToolbox.

Comet is MATA's proprietary host implementation. The Starfire client and shared protocol crates remain open source; the host stays in-house.