We are an agency, so read this knowing that. We also tell people to hire a freelancer when that is the right call, and it often is. The real question is not agency or freelancer. It is how far your AI system has to go: a demo, an internal tool, or something your customers rely on.
The real question
Not who builds it, but how far it has to go.
01
Demo
Prototype or proof of concept
A failure teaches you something
02
Internal tool
Your team uses it
A wrong answer is an inconvenience
03
Customers rely on it
Faces customers, touches real systems
A wrong answer is an incident
A freelancer
One skilled person can own all of it
The gap
An agency or a team
Several specialists, and support after launch
The short answer
Hire a freelancer for a prototype, a proof of concept or a contained internal tool, where a single skilled person can own everything and a failure is cheap. Hire an agency (or build a team) when the agent will face customers, touch real systems such as orders, payments or patient data, or needs ongoing evaluation, monitoring and support after launch. Most AI projects that stall do so in the gap between those two.
What building an AI agent actually involves
A working demo is perhaps a fifth of the job. A production agent also needs:
- Data and retrieval: cleaning documents, chunking, embeddings, search that finds the right passage, and keeping it fresh.
- Tool integration: connecting to your APIs (CRM, order system, calendar) with permissions so the agent can only do what it should.
- Evaluation: test sets of real questions, automated scoring, and a way to know a change made things better, not worse.
- Guardrails: handling prompt injection, off-topic requests, personal data and unsafe answers.
- Operations: logging, tracing, cost limits, latency budgets, alerts, and a fallback to a human.
- Product and UI: the chat or app the user actually touches, on web or mobile.
One excellent freelancer can do several of these. It is rare for one person to do all of them, at the same time, with nothing slipping.
What you are actually hiring for
The working demo is about a fifth of the job.
01
Data and retrieval
02
Tool integration
03
Evaluation
04
Guardrails
05
Operations
06
Product and UI
One excellent freelancer covers several of these. Covering all six at once, with nothing slipping, is what a team is for.
Side by side
| Freelancer | Agency | |
|---|---|---|
| Best for | Prototypes, POCs, internal tools | Customer-facing or business-critical systems |
| Cost | Lower day rate | Higher rate, more people |
| Speed to a demo | Often faster | Similar |
| Speed to production | Depends on one person's range | Parallel work across data, AI, app and ops |
| Breadth | One person's skills | Several specialists |
| Continuity | Risk if they become unavailable | A team that stays on |
| Accountability | Personal | Contract, process, handover |
When a freelancer is the right choice
- You want to test whether an idea works before investing.
- The tool is internal, and a wrong answer is an inconvenience, not a customer incident.
- You have an engineer in-house who can review the work and take it over.
- The scope is narrow: one data source, one workflow.
When an agency is the right choice
- The agent talks to your customers or acts on their accounts.
- It must connect to several systems, or to regulated data.
- You need the AI and the product around it: web, mobile, data pipeline.
- You need someone accountable for quality after launch: evals, monitoring, model updates.
- Your prototype already works in the demo and fails in real use. That is usually an evaluation and retrieval problem, which takes a team to fix (we wrote about it in Why AI chatbots break in production).
A two-minute test
One yes is usually enough to need a team.
- 01
Will it talk to your customers, or act on their accounts?
Yes team - 02
Does it connect to several systems, or to regulated data?
Yes team - 03
Do you need the product around it too: web, mobile, a data pipeline?
Yes team - 04
Does someone need to own quality after launch: evals, monitoring, model updates?
Yes team - 05
Does a prototype already work in the demo and fail in real use?
Yes team
Questions to ask either one
- How will you measure whether the agent is good? Can I see the test set?
- What happens when it doesn't know the answer?
- How do you stop it doing something it shouldn't (prompt injection, wrong tool call)?
- What will it cost per conversation at our volume, and how is that capped?
- Who maintains it after launch, and what does that cost?
- Who owns the code, prompts and data?
If the answers are vague, the build will be too, whoever does it.
Reading the answers
A vague answer now is a vague build later.
“How will you measure whether the agent is good?”
We'll test it thoroughly before launch.
Here is the test set: a few hundred real questions. It runs on every change, and a release that scores worse is blocked.
“What happens when it doesn't know the answer?”
It's very accurate, so that's rare.
It says it could not find that, shows what it searched, and offers a person. We test that path like any other.
“What will it cost per conversation at our volume?”
It depends on usage.
Here is our estimate per conversation at your volume, and the daily cap that stops a bad night becoming a bad month.
How we work
Kalpstack is a small team that builds AI agents, chatbots and RAG systems together with the web, mobile and data work around them. On a first call we tell you what we would build, what it would cost, and which parts you should not build at all, including when a freelancer or an off-the-shelf tool would serve you better. Book a call.