Part 10 · Foundation models and agents
Large pretrained policies, language-model agents that call robot skills, and learned world models.
Learn first
Section titled “Learn first”This part assumes the following. If any of it is new, learn it on AI Engineering from Scratch first:
- Embodied VLAs: RT-2, OpenVLA, π0, GR00T
- The Agent Loop: Observe, Think, Act
- Tool Use and Function Calling
- MCP Fundamentals: Stateless Requests and JSON-RPC
- Building an MCP Server: Stateless Python and TypeScript
- World Models & Video Diffusion
Lessons
Section titled “Lessons”| # | Lesson | Run type | Time | Status |
|---|---|---|---|---|
| 41 | Vision-language-action models | ART | ~60 min | ✅ done |
| 42 | LLM agents with robot tools | CPU | ~75 min | ✅ done |
| 43 | World models: learned dynamics | CPU | ~60 min | ✅ done |