Image Optimization Pipeline
Live infrastructureBetaWatch what happens behind the scenes when you optimize an image: presigned upload, async queue, background worker, and real-time notification back to your browser.
Concepts explained
Initiates the request, uploads the file directly to storage using a presigned URL, and listens for real-time updates via SSE. The browser does more work than you might expect: it never just waits.
Popular choices: Chrome, Firefox, Safari, Edge
Your web server. It handles HTTP requests, enforces rate limits and validation, generates presigned URLs, and enqueues jobs. It does not touch the file itself. The file goes directly between the browser and storage.
Popular choices: Node.js, Go, Python (FastAPI), Ruby on Rails, Java Spring
A place to store files at any scale. You access them via HTTP using a unique key. The browser uploads directly here without going through your API, saving bandwidth, memory, and money.
Popular choices: AWS S3, Cloudflare R2, Google Cloud Storage, Backblaze B2
A holding area for background tasks. Instead of doing slow work inside an HTTP request, you drop a job in the queue and respond instantly. A worker picks it up when ready. If the worker crashes, the job stays and gets retried automatically.
Popular choices: Redis (BullMQ), AWS SQS, RabbitMQ, Google Cloud Pub/Sub
A separate process that runs independently from your web server. It picks up jobs from the queue and does the heavy work: resizing images, sending emails, generating PDFs. You can run many workers in parallel without touching the API.
Popular choices: Any runtime works: Node.js, Python, Go, Ruby, Rust
Stores structured metadata about each run: status, timing, file sizes, and output keys. The actual files live in object storage. The database lets you query and display results instantly without reading raw files.
Popular choices: PostgreSQL, MySQL, Supabase, PlanetScale, Neon