How to answer “Walk me through a data pipeline you built”
a guide for data engineers.
Almost every data engineering interview asks it in some form. Most answers list the tools. Strong answers explain the decisions. Here is what interviewers are listening for, a structure you can reuse and a worked example.
The question behind the question
The interviewer already knows what Airflow and Spark are. They want to know whether you understand why the pipeline looks the way it does.
Design judgement
Why batch and not streaming? Why this storage format, this partitioning, this schedule? Every choice had an alternative, and they want to hear you weigh it.
Scale and constraints
How much data, how often, how fresh it had to be and what it cost. Numbers make the design make sense.
Life after launch
What broke, how you knew, and what you changed. A pipeline that never failed is one nobody looked at closely.
Purpose, shape, decision, failure, result
Five parts, in about three minutes. Start with who used the data, so every technical detail has a reason.
- 01
Purpose
What to sayWho consumed the data and what they did with it. A dashboard, a model, a finance close, a customer feature.
What they listen forThat you built for a user, not for the tools.
- 02
Shape
What to saySources, volume, frequency and freshness. One sentence of architecture: where data came from, where it landed, what ran it.
What they listen forThat you can explain the system simply, with real scale.
- 03
Key decision
What to sayThe one or two choices that mattered most, what you considered and why you picked what you did. This is the heart of the answer.
What they listen forTrade-offs, in your own words, with a reason tied to the purpose.
- 04
Failure
What to sayWhat went wrong once it was live, how you found out and what you changed so it would not happen silently again.
What they listen forOwnership and honesty. Whether you improve systems, not just repair them.
- 05
Result
What to sayWhat changed for the people who used it, with a number you can defend. Then, briefly, what you would do differently now.
What they listen forImpact, and the self-awareness to see the design's limits.
What it sounds like
An illustrative answer. It is not from a real candidate, and the numbers are made up.
The finance team closed the month from an orders table that took two days to rebuild by hand. I built the pipeline that replaced it.
About 40 million order events a day from three services, landing in S3, loaded into the warehouse every hour, orchestrated in Airflow.
We looked at streaming, but finance only needed hourly freshness and the team had no streaming experience. I chose hourly incremental merges keyed on order id, which kept cost low and made reruns safe.
In the second month an upstream service started sending late events, so some days were short. I added a three-day lookback to the merge and a row-count check against the source that fails the run.
Month-end close went from two days of manual work to a table that was ready the morning after. If I built it again, I would agree an event contract with the upstream teams first.
Notice what it does: it starts with a user, gives the scale in one breath, spends its time on a decision with a rejected alternative, admits a failure and ends with a change someone could feel.
Pick one worth telling
You may have built many pipelines. Pick the one where you can explain the most decisions, not the one with the most tools.
- A pipeline that replaced manual work someone depended on
- A migration, such as on-premise to cloud or a warehouse change
- A batch job you moved to incremental or streaming, or decided not to
- A pipeline you inherited and fixed, with a clear before and after
- A cost or performance problem you found and solved
Mistakes to avoid
The same few errors turn good work into a forgettable answer.
- Reciting the tool list. Name a tool only when you explain why it was chosen.
- Describing the target design, not what you actually built and ran.
- Saying “we” throughout. Make your own part and your own decisions clear.
- Skipping scale. Without volume, frequency and freshness, no design choice makes sense.
- Pretending nothing broke. Interviewers trust an answer more when it includes a failure.
- Sharing confidential details. Anonymise the company and the data, but keep the numbers and the reasoning.
Get the details before you forget them
The decisions and the numbers fade first. Find them while you still have access.
- The design doc or ticket where the approach was agreed, and the options you turned down
- Orchestrator run history for volumes, durations and failure counts
- Warehouse query history and the cloud bill, for cost before and after
- The pull requests for the first version and for each fix after launch
This is also where resumes fall short. Most Data Engineer resumes we read say what was built, and very few say what the engineer decided or owned. See where Data Engineer resumes lose out, or read how to answer the production incident question.
Common questions
- How technical should I get?
- Start simple and let the interviewer pull you deeper. Give one sentence of architecture, then the decision. If they ask about partitioning or idempotency, that is your cue to go into detail.
- What if I only maintained a pipeline someone else designed?
- Talk about what you changed and why. Fixing a failure, cutting cost or adding a check are real decisions. Say clearly which parts were inherited.
- Should I draw it?
- If there is a whiteboard or a shared screen, yes. A box for each source, the storage, the transformation and the consumer is enough. Keep talking about decisions while you draw.
- What follow-up questions should I expect?
- How did you handle late or duplicate data? What happens if a run fails halfway? How did you test it? What did it cost? What would you change now? Prepare one honest sentence for each.
Write the decisions down while you still remember why.
graph asks you about the pipelines on your resume the way an interviewer would, and builds your interview prep from your own answers.