Recommender Systems: Evaluation and Metrics

In this course you will learn how to evaluate recommender systems. You will gain familiarity with several families of metrics, including ones to measure prediction accuracy, rank accuracy, decision-support, and other factors such as diversity, product coverage, and serendipity. You will learn how different metrics relate to different user goals and business goals. You will also learn how to rigorously conduct offline evaluations (i.e., how to prepare and sample data, and how to aggregate results). And you will learn about online (experimental) evaluation. At the completion of this course you will have the tools you need to compare different recommender system alternatives for a wide variety of uses

• Understand how to evaluate recommender systems effectively
• Apply various metric families including prediction and rank accuracy
• Analyze diversity, product coverage, and serendipity factors
• Relate metrics to user and business goals
• Conduct rigorous offline and online evaluations
• Compare different recommender system alternatives

• Basic computer and internet skills
• Ability to read English instructions and complete short practice exercises