Evaluating Large Language Model Outputs

This course addresses evaluating Large Language Models (LLMs), starting with foundational evaluation methods, exploring advanced techniques with Vertex AI's tools like Automatic Metrics and AutoSxS, and forecasting the evolution of generative AI evaluation. This course is ideal for AI Product Managers looking to optimize LLM applications, Data Scientists interested in advanced AI model evaluation techniques, AI Ethicists and Policy Makers focused on responsible AI deployment, and Academic Researchers studying the impact of generative AI across various domains. A basic understanding of artificial intelligence, machine learning concepts, and familiarity with natural language processing (NLP) is recommended. Prior experience with Google Cloud Vertex AI is beneficial but not required. It covers practical applications, integrating human judgment with automatic methods, and prepares learners for future trends in AI evaluation across various media, including text, images, and audio. This comprehensive approach ensures you are equipped to assess LLMs effectively, enhancing business strategies and innovation

• Identify fundamentals of Large Language Models and current evaluation methods
• Apply hands-on knowledge of Vertex AI's Automatic Metrics and AutoSxS
• Evaluate emerging trends in generative AI evaluation including human assessment

• No deep prior experience required, basic computer and internet skills helpful
• Ability to read course instructions in English and complete short activities