← Graph

Orchestrating video transcoding in ruby

talk 34 connections

wroclove.rb 2019 talk by Michal Matyas (his 4th wroclove.rb, first time speaking). Originally titled 'Transcoding in Ruby: a story'; renamed because the actual transcoding was done by ffmpeg, not Ruby. Based on conversation logs rather than the actual code (speaker no longer has access). Narrates three-phase evolution of a Rails media platform's transcoding pipeline: (1) MVP with CarrierWave + carrierwave-video + carrierwave_backgrounder and ActiveRecord callbacks — ended up as spaghetti with a CarrierWave bug preventing reprocessing of a single version, and inability to transcode versions based on other versions; (2) one-DB-model-per-file rewrite that still used CarrierWave — suffered race conditions from multiple uploaders setting status in parallel and got stuck in permanent 'processing' state; (3) full rewrite with a single processing worker using streamio-ffmpeg with custom args, priority-based transcoding (lowest quality first, like YouTube), Sidekiq with separate queues for processing (CPU-bound) and uploads to S3 (network-bound). Explains why AWS Lambda (15-minute max execution), Zencoder, and Amazon Elastic Transcoder were rejected (cost). Covers the MP4 container format (ISO base media / QuickTime origin; ftyp, mdat, metadata atoms) and delivers 9 ffmpeg transcoding tips. Q&A asks about ffmpeg vulnerabilities.

type
talk
talk Orchestrating video transcoding in ruby
about
FFmpeg tool
The pipeline uses ffmpeg as the transcoding engine; most tips are ffmpeg-specific.
talk Orchestrating video transcoding in ruby
about
Final iteration uses streamio-ffmpeg with custom args, mainly for progress tracking.
talk Orchestrating video transcoding in ruby
about
The MVP and second iteration used CarrierWave and hit its limits.
talk Orchestrating video transcoding in ruby
about
Used in the MVP for video processing via ffmpeg.
talk Orchestrating video transcoding in ruby
about
Used in the MVP to run transcoding in background workers.
talk Orchestrating video transcoding in ruby
about
Sidekiq tool
Final pipeline uses Sidekiq with distinct queues for processing and S3 uploads.
talk Orchestrating video transcoding in ruby
about
Amazon S3 tool
Final rewrite uploads all derivatives to S3 on a dedicated upload queue.
talk Orchestrating video transcoding in ruby
about
AWS Lambda tool
Considered and rejected due to 15-minute execution limit.
talk Orchestrating video transcoding in ruby
about
Zencoder tool
Considered and rejected due to cost for a full media platform.
talk Orchestrating video transcoding in ruby
about
Considered and rejected due to cost.
talk Orchestrating video transcoding in ruby
about
Docker tool
Not used because at the time Docker wasn't production-ready/popular.
talk Orchestrating video transcoding in ruby
about
MP4 concept
Deep dive into MP4 as a container format and its metadata structure.
talk Orchestrating video transcoding in ruby
about
WebM concept
Initially transcoded alongside MP4; dropped as browser MP4 support improved.
talk Orchestrating video transcoding in ruby
about
H.264 concept
Tip on choosing H.264 profiles for compatibility vs quality.
talk Orchestrating video transcoding in ruby
about
YUV 4:2:0 concept
Always convert to YUV 4:2:0 to avoid browser playback bugs.
talk Orchestrating video transcoding in ruby
about
ISO base media / MP4 originate from QuickTime; hand-camera QuickTime input triggered a pixel-format bug.
talk Orchestrating video transcoding in ruby
about
ffprobe tool
Mentioned when describing duration-approximation pitfalls during output validation.
asked_at
Orchestrating video transcoding in ruby talk
Audience Q&A after the talk.
person Michal Matyas
authored
Orchestrating video transcoding in ruby talk
Speaker and author of the wroclove.rb 2019 talk.
from_talk
Orchestrating video transcoding in ruby talk
Lesson drawn from the MVP's callback-driven design.
from_talk
Orchestrating video transcoding in ruby talk
Lesson from hitting CarrierWave's limits on version-based transcoding.
from_talk
Orchestrating video transcoding in ruby talk
Main architectural lesson from the final rewrite.
from_talk
Orchestrating video transcoding in ruby talk
Explicit 'lesson number one' in the talk.
from_talk
Orchestrating video transcoding in ruby talk
Tip #1 on transcoding.
from_talk
Orchestrating video transcoding in ruby talk
Tip #2: move MP4 metadata to the front of the file.
from_talk
Orchestrating video transcoding in ruby talk
Tip #3: pass custom args instead of using streamio-ffmpeg's DSL.
from_talk
Orchestrating video transcoding in ruby talk
Tip #4: no universal solution; URL transcoding can be 50× slower.
from_talk
Orchestrating video transcoding in ruby talk
Tip #5: use presets; also mentions parallel-chunk transcoding.
from_talk
Orchestrating video transcoding in ruby talk
Tip #6 on profiles.
from_talk
Orchestrating video transcoding in ruby talk
Tip #7 on pixel format compatibility.
from_talk
Orchestrating video transcoding in ruby talk
Tip #8 on collecting metadata for diagnostics.
from_talk
Orchestrating video transcoding in ruby talk
Tip #9 on validating ffmpeg output.
from_talk
Orchestrating video transcoding in ruby talk
Recommendation drawn from the Q&A on ffmpeg vulnerabilities.
talk Orchestrating video transcoding in ruby
presented_at
Delivered at wroclove.rb 2019.

Provenance

Created
2026-04-17 16:17 seed
Read by
28 extractions