Vision, speech, embeddings and images
Multimodal Inference
How vision-language models, embedding and reranking services, speech recognition, text-to-speech, real-time voice pipelines and diffusion models run, what they cost and how to serve each.
1 tracks · 5 lessons · about 1 hours · 1 free lessons · 0 animated diagrams
What you will be able to do
- ✓Estimate cost and latency for image, audio and embedding workloads
- ✓Build a streaming voice pipeline and budget its latency
- ✓Serve embeddings and rerankers at scale
- ✓Understand diffusion sampling steps and their speed levers
Who it is for
Engineers adding images, audio and retrieval to LLM products.
Prerequisites
Local LLM Inference basics.
Curriculum
Vision, Audio and Embeddings
IntermediateVision-language models, speech recognition and synthesis, embeddings and rerankers at scale, diffusion models and real-time voice pipelines.
- 1How vision-language models work, and what images costFree15 min
- 2Speech recognition and text-to-speech: how they work and how to serve themPro17 min
- 3Embeddings and rerankers at scalePro16 min
- 4Image and video diffusion: how generation works and what it costsPro16 min
- 5Real-time voice pipelines: the latency budget, turn-taking and streamingPro17 min