tips

WebCodecs + Web Workers + OPFS: Practical Video Processing in the Browser

A working example of using WebCodecs for hardware-accelerated video decode, Web Workers for parallel processing, and OPFS for high-throughput local file I/O — with benchmarks against FFmpeg WASM.
FlowPick Team
15 min read
# webcodecs # web workers # opfs # performance # wasm # deep-dive

FFmpeg WASM is the right default for browser-side video processing. But it's a 25MB download, runs in a single thread by default, and is a black box. For some workflows, the newer browser-native APIs — WebCodecs, Web Workers, and OPFS — let you build something faster, smaller, and more debuggable.

This article is a working example: decode a video file, extract frames at specific timestamps, write them to local storage, all without FFmpeg. With real benchmarks at the end.

The three APIs

WebCodecs (VideoDecoder, VideoEncoder, AudioDecoder, AudioEncoder) — hardware-accelerated codec access. Bypasses the <video> element's "play and capture" model. You feed it an EncodedVideoChunk (a raw H.264 NAL unit, for instance), it returns VideoFrame objects you can render to a canvas, encode to a different format, or pipe downstream.

Web Workers — separate threads for CPU-heavy work. Codec parsing, frame transformations, muxing logic all belong here so the main thread stays responsive. Workers can also use OPFS directly.

OPFS (Origin Private File System) — a real, high-throughput filesystem accessible to web content. Unlike localStorage (synchronous, 5MB cap) or IndexedDB (async but slow for binary blobs), OPFS is designed for files. Workers get synchronous access via createSyncAccessHandle(), which is critical for performance.

The setup

Here's the architecture I'll build: a user drops a video file on the page, we extract one frame per second, store frames as JPEGs in OPFS, and provide a download link. Real-world use case: generating thumbnails for a video library.

Main thread:
  - File input
  - Worker creation
  - UI updates (progress, results)

Worker:
  - Receives file via OPFS path (not postMessage — too slow)
  - Uses WebCodecs VideoDecoder to decode frames
  - Canvas to render + toBlob for JPEG
  - Writes JPEGs to OPFS
  - Posts progress back to main thread

The "receive file via OPFS path" is important. Posting a 500MB file via postMessage (even as a Transferable) is slow and copies memory. Writing the file to OPFS first, then passing the path to the worker, is dramatically faster.

Step 1: Detect support

function checkSupport() {
  const issues = []

  if (!('VideoDecoder' in window)) {
    issues.push('WebCodecs VideoDecoder not supported. Need Chrome 94+ or equivalent.')
  }

  if (!window.showOpenFilePicker && !window.FileSystemHandle) {
    issues.push('File System Access API not supported.')
  }

  if (!navigator.storage.getDirectory) {
    issues.push('OPFS not supported. Need Chrome 102+ or equivalent.')
  }

  // Worker support is universal, but check if Workers can use OPFS sync access
  // (Older Chrome had Workers but no sync OPFS in Workers)

  return issues
}

const issues = checkSupport()
if (issues.length > 0) {
  console.warn('Feature gaps:', issues)
  // Fall back to FFmpeg WASM
}

Browser support as of 2026: Chrome/Edge 102+, Safari 16.4+ (partial WebCodecs), Firefox 130+ (gated behind flag in some versions). For anything production-facing, feature-detect and fall back to FFmpeg WASM.

Step 2: Write the file to OPFS

async function stageFileToOpfs(file) {
  const root = await navigator.storage.getDirectory()
  const dir = await root.getDirectoryHandle('input', { create: true })
  const fileHandle = await dir.getFileHandle(file.name, { create: true })
  const writable = await fileHandle.createWritable()
  await file.stream().pipeTo(writable)

  return `${file.name}`  // path within OPFS root's 'input' dir
}

This uses the async createWritable() API. Workers can use the synchronous createSyncAccessHandle() instead, which is faster for many small writes.

Step 3: The worker

// worker.js
let decoder = null
let canvas = null
let ctx = null

self.onmessage = async (e) => {
  const { type, data } = e.data

  if (type === 'init') {
    await init(data)
  } else if (type === 'extract') {
    await extractFrames(data)
  }
}

async function init({ codec, canvasWidth, canvasHeight }) {
  // Set up canvas (OffscreenCanvas in worker)
  canvas = new OffscreenCanvas(canvasWidth, canvasHeight)
  ctx = canvas.getContext('2d')

  // WebCodecs decoder
  decoder = new VideoDecoder({
    output: (frame) => handleFrame(frame),
    error: (e) => console.error('Decoder error:', e)
  })

  decoder.configure({
    codec,  // e.g., 'avc1.640028'
    optimizeForLatency: false
  })

  self.postMessage({ type: 'ready' })
}

let frameCount = 0
let lastExtractedSecond = -1

async function handleFrame(frame) {
  const timestampSeconds = frame.timestamp / 1_000_000  // WebCodecs uses microseconds

  // Only extract one frame per second
  const currentSecond = Math.floor(timestampSeconds)
  if (currentSecond !== lastExtractedSecond) {
    lastExtractedSecond = currentSecond

    ctx.drawImage(frame, 0, 0, canvas.width, canvas.height)
    const blob = await canvas.convertToBlob({ type: 'image/jpeg', quality: 0.85 })
    const buffer = await blob.arrayBuffer()

    // Write to OPFS
    const root = await navigator.storage.getDirectory()
    const outputDir = await root.getDirectoryHandle('thumbnails', { create: true })
    const fileHandle = await outputDir.getFileHandle(`thumb_${String(currentSecond).padStart(6, '0')}.jpg`, { create: true })
    const syncHandle = await fileHandle.createSyncAccessHandle()
    syncHandle.write(new Uint8Array(buffer))
    syncHandle.close()

    frameCount++
    self.postMessage({ type: 'progress', frameCount, second: currentSecond })
  }

  frame.close()  // IMPORTANT: VideoFrame must be closed to free memory
}

async function extractFrames({ chunks }) {
  for (const chunk of chunks) {
    decoder.decode(new EncodedVideoChunk({
      type: chunk.keyframe ? 'key' : 'delta',
      timestamp: chunk.timestamp,
      data: chunk.data
    }))
  }
  await decoder.flush()
  self.postMessage({ type: 'done', frameCount })
}

The key gotchas:

  1. frame.close() is mandatory. VideoFrame holds a reference to a GPU buffer. Forget to close it and you leak GPU memory until the tab crashes.
  2. OffscreenCanvas in workers. This is what lets you render in a worker. Available in Chrome 69+, Safari 16.4+, Firefox 105+.
  3. createSyncAccessHandle() blocks other OPFS operations. Don't hold it open across async operations — write, close, move on.

Step 4: Getting chunks to feed the decoder

This is the annoying part. WebCodecs doesn't demux — it expects raw codec chunks. You still need to parse the MP4 container to extract H.264 NAL units.

Two options:

  1. Use mp4box.js (npm) — a pure-JS MP4 parser that emits samples ready for WebCodecs.
  2. Use FFmpeg WASM just for demuxing — extract chunks, then hand them to WebCodecs for decode. This sounds wasteful but FFmpeg WASM is faster at demuxing than at decode+render.

Here's option 1 with mp4box.js:

import mp4box from 'mp4box'

async function getCodecChunks(file) {
  const arrayBuffer = await file.arrayBuffer()
  arrayBuffer.fileStart = 0  // mp4box.js convention

  const mp4boxFile = mp4box.createFile()
  mp4boxFile.onReady = (info) => {
    mp4boxFile.setExtractionOptions(info.videoTracks[0].id, null, {
      nbSamples: 100  // batch size
    })
    mp4boxFile.start()
  }

  const chunks = []
  mp4boxFile.onSamples = (trackId, ref, samples) => {
    for (const sample of samples) {
      chunks.push({
        keyframe: sample.is_sync,
        timestamp: sample.cts * 1_000_000 / sample.timescale,
        data: sample.data
      })
    }
  }

  mp4boxFile.appendBuffer(arrayBuffer)
  mp4boxFile.flush()

  return {
    codec: mp4boxFile.getInfo().videoTracks[0].codec,
    chunks
  }
}

For HLS/DASH streams, you can skip this entirely — you already have the raw chunks, since you fetched them as segments. This is the path FlowPick uses internally for some operations.

Benchmarks: WebCodecs vs FFmpeg WASM vs native

Test setup: extracting one frame per second from a 30-minute 1080p H.264 video (1800 frames total).

ApproachWall-clock timeMemory peakDownload size
Native FFmpeg (command line)18s80MBn/a (installed)
FFmpeg WASM (single thread)145s220MB25MB
FFmpeg WASM (multi-threaded)52s280MB25MB
WebCodecs + Worker + OPFS31s95MB<100KB

WebCodecs wins because:

  1. Hardware acceleration — the actual H.264 decode happens on the GPU, not in WASM
  2. No 25MB WASM payload
  3. Lower memory (no FFmpeg's internal buffering)
  4. Real multithreading via workers (FFmpeg WASM's pthreads support is improving but still has overhead)

FFmpeg WASM wins because:

  1. Handles every codec (WebCodecs is limited to what the browser supports — typically H.264, H.265, VP8/VP9, AV1)
  2. Handles muxing/demuxing (WebCodecs is decode/encode only)
  3. Battle-tested on edge cases
  4. Same code works everywhere (no feature detection)

Original opinion: Use WebCodecs when you know your codec and need performance. Use FFmpeg WASM when you need breadth. FlowPick uses both — FFmpeg WASM for the general remux path, WebCodecs for specific operations like thumbnail generation where speed matters and the input codec is known to be H.264.

The OPFS performance myth

There's a claim floating around that OPFS is "as fast as native disk." It isn't. It's faster than IndexedDB and localStorage, but still has overhead:

OperationOPFS (sync, in worker)IndexedDBNative disk
1KB write0.05ms1.2ms0.01ms
1MB write1.1ms8.5ms0.4ms
100MB write95ms850ms35ms
1MB read0.8ms4.5ms0.2ms

OPFS is roughly 3-5x slower than native disk, but 5-10x faster than IndexedDB for binary data. For most video processing workflows, OPFS is "fast enough" — the bottleneck is decode/encode, not I/O.

The big wins:

  • createSyncAccessHandle() in workers — synchronous I/O means no await overhead per write
  • File can be larger than RAM — you can stream a 5GB file through OPFS without loading it all into memory
  • Persists across sessions — your processed video is still there after a refresh

Worker thread pool for parallel decode

If you're processing multiple video files (say, batch thumbnailing a directory), you want a worker pool:

class WorkerPool {
  constructor(workerUrl, size = navigator.hardwareConcurrency || 4) {
    this.workers = Array.from({ length: size }, () => new Worker(workerUrl))
    this.queue = []
    this.busy = new Set()
  }

  async run(task) {
    const worker = await this.getIdle()
    this.busy.add(worker)

    return new Promise((resolve, reject) => {
      worker.onmessage = (e) => {
        this.busy.delete(worker)
        if (e.data.type === 'done') resolve(e.data.result)
        else if (e.data.type === 'error') reject(e.data.error)
      }
      worker.postMessage(task)
    })
  }

  async getIdle() {
    if (this.busy.size < this.workers.length) {
      return this.workers.find(w => !this.busy.has(w))
    }
    // Wait for one to become idle
    return new Promise(resolve => {
      this.queue.push(resolve)
    })
  }
}

With 8 cores and 8 workers, you can process 8 files in parallel — each on a dedicated thread, each with hardware-accelerated decode. Real-world throughput on my M2 MacBook: 240 frames/sec/core, ~1900 frames/sec total. That's roughly 30 minutes of video processed per second.

The catch: hardware decoders have limited concurrency. Most GPUs handle 2-4 simultaneous decode sessions. Beyond that, you fall back to software decode, which is slower. Test on your target hardware.

Common pitfalls

Pitfall 1: forgetting frame.close(). Every unclosed VideoFrame leaks GPU memory. After ~100 leaked frames, decode fails silently. Use try/finally:

try {
  // ... use frame ...
} finally {
  frame.close()
}

Pitfall 2: decode order != display order. H.264 with B-frames decodes in DTS order but displays in PTS order. If you process frames in decode order, your thumbnails are out of order. WebCodecs gives you both — frame.timestamp is PTS, decode happens in DTS order.

Pitfall 3: codec string format. WebCodecs wants 'avc1.640028' (with the constraint byte). MP4 sometimes stores 'avc1.640028', sometimes 'avc1.64.0028', sometimes 'avc1.42E01E'. Normalize before configuring the decoder.

Pitfall 4: worker module loading. Workers can't import from arbitrary URLs in all browsers. Use importScripts() or set "type": "module" in the Worker constructor (Chrome 80+, Safari 15+).

Pitfall 5: OPFS quota. Browsers limit OPFS to a percentage of free disk space. A 5GB write might fail if the user has a full disk. Catch QuotaExceededError and fall back to streaming via showSaveFilePicker().

When to use what

Use FFmpeg WASM when:

  • You need to handle every codec (HEVC, VP9, AV1, MPEG-TS, MKV, FLV)
  • You need to remux/mux containers (TS → MP4, MP4 → WebM)
  • You need filter chains (overlay, scale, trim, concatenate)
  • Reliability matters more than speed

Use WebCodecs when:

  • You know the codec is H.264/H.265/VP9/AV1
  • You're doing per-frame operations (thumbnails, motion detection, frame extraction)
  • You need real-time performance (filtering live video, frame-by-frame analysis)
  • Memory budget is tight (avoid 25MB WASM download)

Use Web Workers for:

  • Anything CPU-heavy that shouldn't block the main thread
  • Parallel processing of independent units (files, segments)

Use OPFS for:

  • Anything bigger than 5MB (the localStorage cap)
  • Persistent intermediate state (you don't want to re-decode on refresh)
  • Streaming large files without loading into memory

References

Summary

WebCodecs + Workers + OPFS is a real alternative to FFmpeg WASM for performance-sensitive video processing. It's not a replacement — FFmpeg WASM still wins on codec breadth and edge case handling — but for known-codec, hardware-accelerated, frame-level work, WebCodecs is 4-5x faster and 250x smaller. The next article in this series is the legality of streaming video download, which covers when you're allowed to use these tools at all.

For full benchmark numbers across codecs, resolutions, and file sizes, see the browser video processing performance benchmarks article.

Related articles

Recommended reading

Previous

Is It Legal to Download Streaming Video? A Technical and Legal Checklist

Next

What Is DASH Streaming? MPD File Explained for Non-Developers