Apache Airflow interview questions

27 questions from the Apache Airflow: Orchestrate Data Pipelines course, each with the correct answer and an explanation. Try to answer before you open the answer.

Airflow Basics

1. What is Airflow best at?

  1. Real-time stream processing
  2. Processing huge datasets inside its own workers
  3. Scheduling and monitoring batch workflows of dependent tasks
  4. Serving web pages
Show answer

Answer: Scheduling and monitoring batch workflows of dependent tasks

Airflow orchestrates; the heavy lifting should happen in systems built for it.

Lesson: What Airflow Is and Is Not

2. Which component decides which tasks are ready to run?

  1. The scheduler
  2. The browser
  3. The metadata cache of the OS
  4. The Python interpreter alone
Show answer

Answer: The scheduler

The scheduler evaluates schedules and dependencies and queues tasks for the executor.

Lesson: Architecture: Scheduler, Executor, Workers, Database

3. Why install Airflow with a constraints file?

  1. It is required to open the UI
  2. It makes Airflow free
  3. It removes the scheduler
  4. It pins compatible dependency versions and avoids conflicts
Show answer

Answer: It pins compatible dependency versions and avoids conflicts

Constraints give a tested set of package versions for each release.

Lesson: Installing and Running Locally

4. What is a task instance?

  1. The DAG file itself
  2. One task within one specific DAG run, with its own state
  3. The metadata database
  4. A worker machine
Show answer

Answer: One task within one specific DAG run, with its own state

A task instance is the runtime record of a task for a particular run.

Lesson: Core Concepts: DAG, Task, Run, Instance

Writing DAGs

5. What do provider packages add to Airflow?

  1. A new database
  2. Operators, hooks and sensors for external systems
  3. Faster CPUs
  4. Browser themes
Show answer

Answer: Operators, hooks and sensors for external systems

Providers package the integrations for clouds, databases and tools.

Lesson: Operators and Providers

6. In TaskFlow, how does data pass from one @task to the next?

  1. It cannot pass data
  2. Through a global variable
  3. Through email
  4. As the return value, stored as an XCom
Show answer

Answer: As the return value, stored as an XCom

Return values become XComs and Airflow wires the dependency automatically.

Lesson: The TaskFlow API

7. What does `extract >> [clean_a, clean_b] >> load` mean?

  1. Both clean tasks run after extract, and load waits for both
  2. load runs first
  3. The tasks run one after another in list order only
  4. It is a syntax error
Show answer

Answer: Both clean tasks run after extract, and load waits for both

A list on the right of >> fans out; a list on the left fans in.

Lesson: Setting Dependencies

8. What does `catchup=False` do?

  1. Deletes old runs
  2. Disables the DAG
  3. Creates only the latest run instead of one per missed interval
  4. Makes the DAG run twice
Show answer

Answer: Creates only the latest run instead of one per missed interval

Without catchup, Airflow does not backfill every interval since start_date.

Lesson: Scheduling: Cron, Intervals and Catchup

Passing Data and Templating

9. What should you pass through XCom for a 2 GB dataset?

  1. Nothing, XCom is forbidden
  2. The whole dataset
  3. Only its storage location
  4. A screenshot
Show answer

Answer: Only its storage location

XCom is for small metadata; large data belongs in external storage.

Lesson: XComs and Their Limits

10. What does `{{ ds }}` give you?

  1. The logical date of the run as YYYY-MM-DD
  2. The current wall-clock time
  3. The DAG file name
  4. A random number
Show answer

Answer: The logical date of the run as YYYY-MM-DD

It is tied to the run, so re-runs process the same date every time.

Lesson: Jinja Templating and Macros

11. Where should a database password for a DAG live?

  1. In a public repository
  2. Hard-coded in the DAG file
  3. In a task name
  4. In a Connection or a secrets backend
Show answer

Answer: In a Connection or a secrets backend

Connections and secrets backends keep credentials out of source control.

Lesson: Connections, Variables and Secrets

Control Flow

12. What happens to the paths not chosen by a branch task?

  1. They run anyway
  2. They are skipped
  3. They fail
  4. They are deleted from the DAG
Show answer

Answer: They are skipped

Unchosen branches get the skipped state for that run.

Lesson: Branching

13. Which rule runs a failure alert only when an upstream task failed?

  1. all_success
  2. one_failed
  3. always_skip
  4. none_failed
Show answer

Answer: one_failed

one_failed triggers as soon as at least one upstream task has failed.

Lesson: Trigger Rules

14. Why set a timeout on a sensor?

  1. It deletes the DAG
  2. Timeouts make sensors faster
  3. Sensors require it syntactically
  4. So it fails instead of waiting forever if the condition never becomes true
Show answer

Answer: So it fails instead of waiting forever if the condition never becomes true

Bounded waiting prevents silent, indefinite hangs.

Lesson: Sensors and Deferrable Operators

15. When is dynamic task mapping useful?

  1. When the number of parallel tasks is only known at run time
  2. When you want to edit the DAG file while it runs
  3. When there are no tasks
  4. Never
Show answer

Answer: When the number of parallel tasks is only known at run time

Mapping generates task instances from data produced during the run.

Lesson: Dynamic Task Mapping

Designing Reliable Pipelines

16. Why should a task be idempotent?

  1. Because Airflow forbids writing files
  2. To make it slower
  3. So retries and re-runs do not duplicate or corrupt data
  4. To avoid using dates
Show answer

Answer: So retries and re-runs do not duplicate or corrupt data

Safe repetition is what lets orchestrators retry and backfill confidently.

Lesson: Idempotent, Partitioned Tasks

17. What makes backfills safe?

  1. Appending without keys
  2. Using datetime.now()
  3. Idempotent tasks that use the run's dates
  4. Running everything at once with no limits
Show answer

Answer: Idempotent tasks that use the run's dates

Deterministic, date-based tasks can be replayed without side effects.

Lesson: Backfills and Reprocessing

18. Where should heavy data transformation usually run?

  1. In systems built for it, such as Spark or the warehouse
  2. Inside the Airflow scheduler
  3. Inside XCom
  4. In the browser
Show answer

Answer: In systems built for it, such as Spark or the warehouse

Airflow coordinates work; specialised engines process the data.

Lesson: Keep Airflow Lightweight

Reliability and Operations

19. When is retrying a task safe?

  1. Only for tasks without dates
  2. Never
  3. Only when it appends rows without keys
  4. When the task is idempotent
Show answer

Answer: When the task is idempotent

Repeating an idempotent task cannot duplicate or corrupt data.

Lesson: Retries, Timeouts and Callbacks

20. What does a pool do?

  1. Stores XComs
  2. Limits how many tasks can use a shared resource at once
  3. Encrypts connections
  4. Renames DAGs
Show answer

Answer: Limits how many tasks can use a shared resource at once

Pools cap concurrent use of a limited resource across DAGs.

Lesson: Concurrency, Pools and Priorities

21. Why check data quality inside the pipeline?

  1. Quality checks make DAGs fail more often by design only
  2. A successful DAG run can still produce wrong or missing data
  3. Airflow cannot run SQL
  4. It replaces monitoring
Show answer

Answer: A successful DAG run can still produce wrong or missing data

Task success only means the code ran; explicit checks confirm the data meets expectations.

Lesson: Monitoring, Logs and SLAs

Scaling and Deployment

22. Which executor launches a new pod for each task?

  1. No executor does this
  2. LocalExecutor
  3. CeleryExecutor
  4. KubernetesExecutor
Show answer

Answer: KubernetesExecutor

KubernetesExecutor creates a pod per task for isolation and per-task resources.

Lesson: Executors and Deployment Options

23. What does a DagBag import test catch?

  1. Syntax errors, bad imports and cycles in DAG files
  2. Network outages
  3. Slow queries in the warehouse
  4. Wrong business logic automatically
Show answer

Answer: Syntax errors, bad imports and cycles in DAG files

It loads the DAG files the way the scheduler would and reports anything that fails to parse.

Lesson: Deploying DAGs with Git and CI

Putting It Together

24. Why is a backfill of any past week safe in this design?

  1. Data is never stored
  2. Airflow forbids backfills
  3. Tasks are idempotent and use the run's date
  4. Because there are no retries
Show answer

Answer: Tasks are idempotent and use the run's date

Deterministic, date-partitioned writes mean re-running a date yields the same result.

Lesson: Case Study: A Daily Sales Pipeline

25. A daily DAG with logical date 5 October covers which period?

  1. Only the minute it starts
  2. 6 Oct to 7 Oct
  3. 5 Oct 00:00 to 6 Oct 00:00, running after it ends
  4. The whole month
Show answer

Answer: 5 Oct 00:00 to 6 Oct 00:00, running after it ends

The data interval is the day starting at the logical date; the run starts once the interval is over.

Lesson: Revision: Cheat Sheet and Self-Check

26. Everything after a branch is skipped, including the join task. What is the likely fix?

  1. Set the join task's trigger_rule to none_failed_min_one_success
  2. Delete the branch
  3. Increase retries
  4. Use datetime.now()
Show answer

Answer: Set the join task's trigger_rule to none_failed_min_one_success

The default all_success rule skips a join when one upstream path was skipped.

Lesson: Revision: Cheat Sheet and Self-Check

27. Which approach to passing a 5 GB DataFrame between tasks is correct?

  1. Email it
  2. Return the DataFrame so it is stored in the metadata DB
  3. Print it in the logs
  4. Write it to object storage and pass the path through XCom
Show answer

Answer: Write it to object storage and pass the path through XCom

XComs are for small values; large data belongs in external storage.

Lesson: Revision: Cheat Sheet and Self-Check

← Back to the course