Evaluating and Debugging Generative AI

Machine learning and AI projects require managing diverse data sources, vast data volumes, model and parameter development, and conducting numerous test and evaluation experiments. Overseeing and tracking these aspects of a program can quickly become an overwhelming task. This course will introduce you to Machine Learning Operations tools that manage this workload. You will learn to use the Weights & Biases platform which makes it easy to track your experiments, run and version your data, and collaborate with your team

• Evaluate programs using LLMs and generative image models with platform-independent tools
• Instrument training notebooks and add tracking, versioning, and logging
• Implement monitoring and tracing for LLMs in complex interactions over time

• Basic familiarity with the software or workflow used
• Ability to follow step-by-step instructions in English