Question
When should you use Step Functions vs MWAA?
Click to reveal answer
Answer
Step Functions for simple linear/branching workflows; MWAA (Airflow) for complex DAG-based scheduling with many dependencies.
Click to flip back
All Pipeline Orchestration Flashcards
Q: When should you use Step Functions vs MWAA?
A: Step Functions for simple linear/branching workflows; MWAA (Airflow) for complex DAG-based scheduling with many dependencies.
Q: What is a Glue workflow?
A: An orchestration of multiple Glue crawlers and ETL jobs that run as a coordinated pipeline.
Q: What is a Step Functions Map state?
A: A state that processes each element of an array in parallel or sequentially.
Q: What format does MWAA use for pipeline definitions?
A: Python-based DAGs (Directed Acyclic Graphs) using Apache Airflow syntax.
Q: How do you monitor Glue job failures?
A: CloudWatch Logs, Glue job run status metrics, and CloudWatch alarms on failure events.
Q: What is the benefit of Glue Data Quality?
A: Validates data against rules during ETL, catching issues before they reach downstream consumers.