Design Scalable AI Systems

This intermediate course teaches you how to design scalable, reliable AI systems that work in real-world production environments. You’ll learn how to build end-to-end architectures that meet throughput, latency, and fault-tolerance goals, and you’ll move from conceptual design to detailed component diagrams and interface specifications. Using industry patterns adopted by modern ML teams, you’ll practice estimating QPS, defining autoscaling rules for the inference layer, structuring data flow between the feature store and model API, and instrumenting your system with a monitoring stack. By the end of the course, you will have created a complete architecture document—including diagrams and interface definitions—that engineering teams can use to implement a scalable AI product

• Design end-to-end AI architectures meeting throughput, latency, and fault-tolerance goals
• Produce complete architecture documents with component diagrams and interface specifications
• Apply industry-standard machine learning system design patterns
• Manage data flow between feature stores and model APIs efficiently
• Estimate and regulate query per second (QPS) for system stability

• Basic familiarity with AI concepts and terminology
• Preparedness to engage in applied exercises or case studies