What to ask when interviewing an AI consultancy.
Most AI consultancies sell strategy slides. Few ship code. Three questions separate the builders from the deck-makers — and what good answers look like.
- Ask to see a system they shipped, in production, that someone is actually using.
- Ask who runs the application after launch — a year out, three years out.
- Ask who's actually doing the work — the partners or the offshore team.
"Show me a system you shipped, in production, today."
Real builders can answer this in five minutes with concrete details: the workflow, the architecture, the metrics, what they'd do differently. Strategy-only firms answer with case study PDFs that summarize a project nobody can quite point to in production.
If they can't show you a system — not a prototype, not a slide, an actual production deployment with users — you're not hiring builders. Decide whether that's what you want.
Bad answer: "Here's a slide deck from our most recent engagement. The system is deployed at a global Fortune 100." (No name. No metric. No live demo.)
Good answer: "This is the application. It's used by 200 analysts every day. Here's how it handles a real query end-to-end. We measure citation accuracy at 96%, p95 latency at 4.2 seconds. Last week's incident was a model-provider outage; here's what we changed."
"Who runs it after launch?"
Most consultancies build it, declare victory, and disappear. Six months later your operations team is troubleshooting a misbehaving application alone, and the team that wrote it is on a different engagement.
Ask who supports the application a year out, three years out. Who handles model upgrades, security patches, integration changes, and incident response. The right answer isn't "we'll figure it out at the end." It's a specific commitment, a specific team, written into the contract.
Our answer: we host and run the application as a managed service. The team that built it is the team that operates it — on the same cadence, with the same people, for as long as you want us there.
Bad answer: "We'll do a knowledge transfer at the end of the engagement and your team will run it from there." (Translation: in 12 months you own a system you didn't build, with documentation that won't survive your next reorg.)
Good answer: "We host and operate the system. The same engineers who built it ship every change, respond to every incident, and meet with your team monthly. If you ever want to bring it in-house, we'll plan the transition."
"Who's actually doing the work?"
The senior partner shows up to the pitch. The senior partner shows up to the kickoff. After that, the senior partner disappears and a junior team in another country builds the application without ever talking to your operating team. This is the dominant pattern in big-firm consulting and it's how most engagements quietly drift.
Ask explicitly: who is on the engagement every week? What are their names? How senior are they? Does that change after the contract is signed?
Bad answer: "We'll assign a delivery team after kickoff. You'll have a senior engagement partner overseeing it." (Translation: senior partner ghost; offshore juniors do the build.)
Good answer: "Here are the four people on the engagement, by name. Three are principals or senior engineers, one is an applied-AI specialist. They're on every call. They write the code. They'll still be the team in year three."
BizzSoftware designs, builds, secures, and runs the internal applications your teams work in every day — with AI features built in. About us →