ParaBun

parabun:gpu

Metal on macOS, CUDA on Linux and Windows, CPU fallback on hosts without a GPU. A matrix passed to gpu.hold() stays resident across matVec calls, so only the input vector crosses the host↔device boundary per call. Pure Float32Array → Float32Array functions are runtime-compiled to PTX (via NVRTC) or MSL (via newLibraryWithSource:) when the body fits a supported shape: arithmetic, ternary, Math.*.

import gpu from "parabun:gpu";

const M = 1024, K = 768;
const mat = gpu.alloc(M * K, "f32");
for (let i = 0; i < mat.length; i++) mat[i] = Math.random();

const held = gpu.hold(mat);                          // uploaded once
const queries = [
  new Float32Array(K).fill(0.1),
  new Float32Array(K).fill(0.2),
];
for (const q of queries) {
  const scores = gpu.matVec(held, q, M, K);        // no copy
  console.log("top score:", Math.max(...scores));
}
gpu.release(held);

Beyond matVec / simdMap, parabun:gpu ships conv2D, scan, reduce, argMin / argMax, histogram, variance, and median / quantile. CUDA device kernels back all of these (bitonic-sort path for quantile, two-stage Hillis-Steele for scan, atomic-privatized for histogram); CPU paths run when CUDA is unavailable. Metal mirror is the next batch.

parabun:image

A Sharp-class image module baked into the runtime — JPEG / PNG / WebP decode and encode (libjpeg-turbo, libpng, libwebp + libsharpyuv vendored statically), bilinear and Lanczos resize, separable Gaussian blur, unsharp-mask sharpen, Sobel edge-detect, 90 / 180 / 270 rotate, flip, crop, brightness / contrast / saturation adjust, threshold, invert, grayscale, per-channel histogram, and Porter-Duff source-over alpha compositing. No npm install sharp, no Node-ABI-versioned binary distribution.

import image from "parabun:image";

const bytes = await Bun.file("photo.jpg").bytes();
const img = image.decode(bytes);
const small = image.resize(img, { width: 800, height: 600, kernel: "lanczos" });
const sharp = image.sharpen(small, { amount: 1.5 });
const webp = image.encode(sharp, { format: "webp", quality: 85 });
await Bun.write("photo.webp", webp);

parabun:audio

A from-scratch audio toolkit: WAV / MP3 decode, Opus encode and decode (libopus 1.6.1), rnnoise-based denoiser, FFT, RBJ Audio EQ Cookbook biquads (lowpass / highpass / bandpass / notch), resample, STFT spectrogram, mel spectrogram (Whisper-mode included for STT pipelines), voice-activity detection, AGC, peak / RMS / windowed envelope, mix, normalize, interleave / deinterleave, and PCM type conversion. Heavy codecs (libopus, minimp3, rnnoise) ship statically.

OS audio I/O is wired on Linux: audio.devices() enumerates ALSA capture and playback devices, audio.capture({ device, sampleRate, channels }) returns a stream whose .frames() async-iterator yields Float32Array PCM straight from snd_pcm_readi, and audio.play({ ... }).write(samples) pushes PCM through snd_pcm_writei. The capture stream exposes reactive peakLevel and active Signals — RMS rate-limited to 10 Hz so a level meter is one effect() away. CoreAudio + WASAPI mount on the same surface in follow-ups.

import audio from "parabun:audio";
import rtp from "para:rtp";

await using mic = await audio.capture({ sampleRate: 48000, channels: 1 });
const enc = new audio.OpusEncoder({ sampleRate: 48000, channels: 1, application: "voip" });
const den = new audio.Denoiser();
const agc = new audio.Gain({ targetLevel: 0.1 });

const ssrc = 0x12345678;
let sequence = 0, timestamp = 0;

for await (const frame of mic.frames()) {
  den.process(frame.samples);                                       // suppress noise (in place)
  agc.process(frame.samples);                                       // normalize loudness
  const opus = enc.encode(frame.samples);
  const packet = rtp.pack({ payloadType: 111, sequence: sequence++, timestamp, ssrc, payload: opus });
  // send `packet` over your transport (UDP, WebRTC, …)
  timestamp += frame.samples.length;
}

parabun:camera

V4L2 capture on Linux. camera.devices() reads /sys/class/video4linux/ and runs VIDIOC_QUERYCAP on each to filter to actual capture devices. camera.formats(path) enumerates the supported (format, width, height, fps) tuples. camera.open(...) mmaps the kernel ring buffer and starts streaming, and cam.frames() is an async iterator of frames. AVFoundation (macOS) and Media Foundation (Windows) backends are planned on the same JS surface.

parabun:video

Scaffold only — the JS surface is in place (video.probe, video.decode, video.encode, video.decodeAll, with codec / container / acceleration options) but the native side hasn't been wired yet. The plan is libavcodec on desktop, V4L2 M2M on Pi 5, NVDEC/NVENC on Jetson, all behind the same JS API.

parabun:gpio

Linux gpiochip uAPI v2 over /dev/gpiochip* — the modern chardev interface, not legacy sysfs. gpio.open(chip, line, opts) returns a Line with read() / write() / async-iterable edge events. Validated end-to-end on a Raspberry Pi 5. Pairs with para:signals — line.value is a reactive Signal that updates on read and on edge events.

parabun:i2c

i2c-dev wrapper with full SMBus quick / byte / word / block transactions plus a raw readWriteRaw() for chips that need 16-bit register addressing or arbitrary frame layouts. i2c.scan(bus) probes addresses, i2c.open(bus, addr) hands back a typed device handle. Works on any Linux board with /dev/i2c-*.

parabun:spi

spidev wrapper with both half-duplex (read / write) and full-duplex (transfer) plus multi-segment (transferSegments) for mode / speed / CS-hold changes mid-burst. Per-device mode / bitsPerWord / maxSpeedHz / lsbFirst are sticky and apply to every transfer until changed.

parabun:llm

An in-tree native inference stack covering three model classes: Llama / Qwen2 chat + completion (LLM), BERT-family sentence embedders (Encoder), and Whisper STT (WhisperModel). Weights mmap off disk; residual stream and KV cache live on-device. Per-token traffic across PCIe is a 4-byte argmax. Q4_K and Q6_K matVec kernels use a 1-warp-per-row, 4-warps-per-block layout; QKV and Gate+Up projections are byte-concatenated at load time and dispatched as one matVec per layer.

import llm from "parabun:llm";

using m = await llm.LLM.load("./Llama-3.2-1B-Instruct-Q4_K_M.gguf");

for await (const piece of m.chat([
  { role: "system", content: "You are helpful and concise." },
  { role: "user", content: "What is the capital of France?" },
])) {
  process.stdout.write(piece);
}

Numbers are within run-to-run noise of ollama on this model and hardware. Chat templates for Llama-3, ChatML, and Mistral-Instruct are detected from the GGUF's tokenizer.chat_template. Only the CUDA backend is wired in this module today; Metal kernels are pending.

llm.serve({ engine, modelId, port }) exposes any model (or anything else implementing .chat() / .generate() / .embed()) over an OpenAI-compatible HTTP API. Routes: GET /v1/models, POST /v1/chat/completions (sync and SSE streaming), POST /v1/completions, POST /v1/embeddings. Optional bearer auth and a FIFO concurrency gate (default 1). Default port is 11434, matching ollama's, so OpenAI clients that auto-discover a local ollama work unchanged.

WhisperModel loads whisper.cpp ggml-*.bin files (F32 / F16 / Q4_0 / Q5_0 / Q5_1 / Q8_0) and runs encoder-decoder STT — KV cache, chunked long-audio, beam search, language detection across all 99 Whisper languages. CUDA-accelerated end-to-end (encoder im2col conv + matmuls + per-head batched attention; decoder per-token matVecs + LM head). On an RTX 4070 Ti, an 11 s JFK clip transcribes in 1.6 s with tiny.en — about 6.9× real-time.

Both LLM and WhisperModel instances expose reactive para:signals Signals: m.busy (refcounted, flips while a chat / generate / embed / transcribe call is in flight) and m.device ("cuda" | "metal" | "cpu", stable for the life of the instance). One effect() reads them straight into UI state — no manual notify.

parabun:vision

vision.frames(stream, { decodeMjpg? }) takes a frame iterator from parabun:camera (or any source yielding the same shape) and yields packed-RGBA8 frames. yuyv, nv12, and rgb24 are converted inline; mjpeg requires the caller to pass image.decode from parabun:image (cross-builtin imports between bun: modules aren't supported, so dependencies are passed in at the call site). vision.detectMotion adds a downsampled-luma frame-diff estimator with temporal smoothing, plus opt-in per-frame region segmentation: pass { regions: { minPixels } } and each MotionFrame carries a regions array of bounding boxes (4-connected, two-pass union-find on the mask, scaled back to source-frame coordinates) so callers can route on where motion is, not just whether.

vision.recognize(frame, { engine: "tesseract", language: "eng" }) drives Tesseract through libtesseract.so.5 FFI (system-installed — apt install libtesseract-dev tesseract-ocr-eng / brew install tesseract), iterates word-level results, and returns one Detection per word with confidence in [0, 1] and bbox in source-frame pixels. The "easyocr" engine is reserved.

vision.onnx(modelPath) returns an ONNX session bound to a system-installed libonnxruntime — the C ABI doesn't export flat symbols, so the FFI dlopens OrtGetApiBase(), walks the resulting OrtApi struct of ~150 function pointers at version-pinned offsets, and rebinds the ones we need via linkSymbols. session.run({ name: { data, shape } }) takes Float32Array tensors, runs inference, and returns a Map of output tensors. Install: brew install onnxruntime / apt install libonnxruntime-dev, or download a release tarball and point PARABUN_ONNX_LIB at the .so.

vision.detect(frame, { engine: "yolo", model }) drives YOLOv8/YOLOv11 ONNX models on top of vision.onnx: bilinear-letterbox preprocess to 640×640 CHW float32 in [0, 1], one inference pass, channel-major decode of the [1, 4+nc, anchors] output, greedy IoU NMS (same-class), and coordinate unmapping back to source-frame pixels. Returns Detection[] with COCO labels by default; pass opts.classes for custom-trained models. Session is LRU-cached per model path so repeat calls skip the dominant ONNX load cost. SSD and RT-DETR engines are reserved.

vision.track() instantiates a stateful tracker that turns per-frame Detection[] into stable Track[] across the stream — each contiguous occurrence of the same object keeps the same monotonic id so callers can draw consistent labels, accumulate trajectories, and route effects per-object instead of per-frame. SORT-style greedy IoU matching with same-class gating; tracks coast for maxFramesMissed frames to ride out brief occlusions without yo-yoing ids; trajectoryLength retains the last N bboxes per track when drawing trails.

parabun:speech

speech.listen(stream, { sampleRate }) takes an audio chunk iterator (parabun:audio's capture stream, a file reader, anything yielding { samples }) and yields one utterance per detected speech burst. The classifier is RMS-against-an-adaptive-noise-floor, with pre-roll to catch word onsets, hangover to seal on silence, and a minimum-length filter to drop clicks and breath sounds.

speech.transcribe(utt, { engine: "whisper", model }) dispatches to the WhisperModel in parabun:llm, with a per-process model cache so the weights aren't reloaded between calls. speech.speak(text, { engine: "piper", model }) drives the Piper voice synthesizer (subprocess in v1, libpiper FFI v2 tracked) and returns f32 mono PCM at the voice's native sample rate, ready to hand straight to audio.play().write(). The listen stream also exposes reactive active / noiseFloor / lastUtterance signals.

parabun:assistant

The 3-line case. Composes parabun:audio (mic + speaker), parabun:speech (VAD + STT + TTS), and parabun:llm (Llama / Qwen2 inference) into a complete on-device voice loop: await using bot = await assistant.create({ llm, stt, tts, system });, then await bot.run(). Mic captures, VAD gates, Whisper transcribes, the LLM generates, Piper synthesizes, ALSA plays — fully local, no cloud round-trip. bot.turns() exposes the loop as an async iterator; bot.ask(text) skips STT for text-only turns; bot.say(text) pushes a proactive utterance.

Reactive surface: bot.state ("idle" | "listening" | "thinking" | "speaking"), bot.history, bot.lastTurn, and bot.interrupted are all para:signals Signals — wire them straight into UI without polling. Persistent memory is one option away: pass memory: "/path/to/memory.sqlite" and the conversation transcript replays into history on every create. Power users keep their seat — bot.llm exposes the underlying model, so anything reachable directly via parabun:llm / parabun:speech / parabun:audio is reachable through bot too.