Source Systems, Data Ingestion, and Pipelines

In this course, you will explore various types of source systems, learn how they generate and update data, and troubleshoot common issues you might encounter when trying to connect to these systems in the real world. You’ll dive into the details of common ingestion patterns and implement batch and streaming pipelines. You’ll automate and orchestrate your data pipelines using infrastructure as code and pipelines as code tools. You’ll also explore AWS and open source tools for monitoring your data

• Gather stakeholder needs and translate to system requirements
• Implement batch and streaming ingestion on AWS
• Integrate security, data management, and orchestration in data systems

• Basic understanding of data workflows and cloud computing