Vision, Audio and Embeddings
Vision-language models, speech recognition and synthesis, embeddings and rerankers at scale, diffusion models and real-time voice pipelines.
0 of 5 complete
- ▶ Up next15 min+50 XPHow vision-language models work, and what images costThe architecture of a vision-language model (image encoder, projector, language model), how images become tokens with…
- Pro17 min+50 XPSpeech recognition and text-to-speech: how they work and how to serve themAutomatic speech recognition (log-mel features, encoder-decoder and CTC models, chunking, timestamps, real-time…
- Pro16 min+50 XPEmbeddings and rerankers at scaleHow embedding and reranking models work, bi-encoders versus cross-encoders, a runnable similarity search with int8…
- Pro16 min+50 XPImage and video diffusion: how generation works and what it costsDiffusion as iterative denoising, latent diffusion and why it is 48x cheaper per step, samplers and step counts…
- Pro17 min+50 XPReal-time voice pipelines: the latency budget, turn-taking and streamingArchitect a voice assistant: the cascaded ASR to LLM to TTS pipeline versus speech-to-speech models, a worked 1,030…