How to answer “Tell me about a model you put into production”
for data scientists and ML engineers.
Most answers spend three minutes on the algorithm. The interviewer is waiting to hear what the model changed, how you knew it worked and what happened after launch. Here is how to give them that.
The question behind the question
Training a model is the part most candidates can describe. Getting one to change a decision, and keeping it working, is the part interviewers are hiring for.
The decision it changed
Who acted differently because of the model? Pricing, fraud review, a ranking, a forecast someone planned with. A model with no decision behind it is a notebook.
How you measured it
The offline metric you chose and why, and the business measure you checked after launch. They want to see you knew the difference.
Life after launch
Drift, retraining, monitoring, a rollback. What happened in month three tells them more than the AUC.
Problem, baseline, choice, launch, impact
Five parts, in about three minutes. Keep the model itself to one part, and spend the rest on why and what happened.
- 01
Problem
What to sayThe decision the model supports and who makes it. What it cost to get that decision wrong.
What they listen forThat you started from the business problem, not the dataset.
- 02
Baseline
What to sayWhat happened before the model: a rule, a heuristic, a person's judgement. Give its numbers if you have them.
What they listen forWhether you knew what “better” had to beat.
- 03
Choice
What to sayThe model and features you chose, what you tried and dropped, and the metric you optimised and why it fit the problem.
What they listen forTrade-offs: accuracy against latency, interpretability, data you could get in production.
- 04
Launch
What to sayHow it went live: batch or real time, how you tested it against the baseline, what you monitored, and one thing that went wrong.
What they listen forOwnership past the notebook, and honesty about problems.
- 05
Impact
What to sayWhat changed in the business measure, how you know it was the model, and what you would do differently now.
What they listen forA defensible result, and the judgement to see the model's limits.
What it sounds like
An illustrative answer. It is not from a real candidate, and the numbers are made up.
Our support team reviewed every refund request by hand, and the queue was taking four days. Slow refunds were the top complaint in customer surveys.
A rule auto-approved refunds under a fixed amount. It cleared about a third of the queue, but fraud losses on those approvals were rising.
I trained a gradient-boosted model on two years of reviewed requests. I tried a deep model, but it was no better and the reviewers needed reasons, so I kept the simpler one and showed the top factors with each score. I optimised for precision at a fixed approval rate, because a wrong approval cost more than a slow one.
It ran in shadow mode for three weeks next to the rule, then on half the traffic. A month in, a new product line skewed the scores, so I added a drift check on input features and a monthly retrain.
It auto-approved about twice as many requests as the rule, with lower fraud losses on approvals in the test, and the queue dropped to under a day. Next time I would agree the monitoring with the support team before launch, not after.
Notice what it does: it opens with a decision and its cost, names the baseline, explains a choice against a rejected alternative, admits a problem after launch and ends with a measured change.
Pick one worth telling
Pick the model with the clearest decision behind it, not the most impressive architecture.
- A model that replaced a rule or a manual review
- A forecast that someone planned stock, staff or budget with
- A ranking or recommendation you tested against the old one
- A model you had to retrain, roll back or retire, and why
- An LLM or GenAI feature you took from prototype to real users, with how you evaluated it
Mistakes to avoid
The same few errors turn good work into a forgettable answer.
- Spending most of the time on the algorithm and hyperparameters.
- Quoting only an offline metric. Say what changed in the business, or why you could not measure it.
- Calling a model “in production” when it ran once in a notebook. Be precise about how it was used.
- Claiming the whole result. Name your part next to engineering, product and the people who acted on it.
- Skipping what went wrong. Drift, bad labels and a rollback are strong material, not weaknesses.
- Sharing confidential details. Anonymise the company and the data, but keep the scale and the reasoning.
Get the details before you forget them
The baseline and the post-launch numbers are the first things you lose. Find them while you still have access.
- The experiment or A/B test write-up, with the baseline numbers
- Your experiment tracker runs, including the models you dropped
- Monitoring dashboards and any drift or retraining alerts
- The launch doc or ticket, and messages from the team that used the model
Resumes show the same gap: they describe the model that was trained more often than the decision it changed. See where Data Scientist resumes lose out and where AI and ML Engineer resumes lose out.
Common questions
- What if my model never made it to production?
- Say so, and talk about what you learned and what it would have taken. Interviewers respect honesty more than a stretched claim. If a stakeholder used the output, even in a spreadsheet, describe that use exactly.
- I cannot share the real numbers. What do I say?
- Use relative numbers or ranges: “roughly halved the queue”, “a few points better than the rule”. Keep the baseline and the method, which matter more than the exact figure.
- Does this work for LLM and GenAI projects?
- Yes. Replace the metric with how you evaluated outputs, such as a test set, human review or user feedback, and talk about cost, latency and failure cases as your trade-offs.
- What follow-up questions should I expect?
- Why that metric? How did you handle class imbalance or label quality? How did you know it was the model and not something else? What would make you retire it? Prepare one honest sentence for each.
Write the baseline down before it disappears.
graph asks you about the models on your resume the way an interviewer would, and builds your interview prep from your own answers.