AI Automation Agencies and Chatbots: What to Buy, What to Build, and What to Avoid

Every AI automation agency now sells a chatbot. The gap between the demo and something that survives a Monday morning support queue is where most projects fail. This guide is for buyers evaluating agencies to build a customer-facing or internal AI chatbot: what the work actually involves, what it costs, what to insist on in scoping, and how to tell a serious build partner from a Zapier reseller with a system prompt.

The market has matured fast. In 2023, most agency chatbot work was a wrapper around GPT-3.5 with a hardcoded knowledge base. By 2026, competent builds use retrieval-augmented generation (RAG) with hybrid search, evaluation harnesses, refusal patterns, and observability. The bar has risen. So has the price of getting it wrong.

What an AI automation agency actually does with chatbots

A chatbot project inside an automation agency is rarely just a chatbot. It sits inside a broader workflow: the bot answers a question, then creates a ticket in Zendesk, updates a HubSpot contact, escalates to a human via Slack, or triggers a follow-up email through n8n. The chatbot is the interface; the automation is the value.

A typical mid-market engagement covers six things:

• Knowledge ingestion. Pulling content from your help centre, product docs, Notion, Confluence, PDFs, and structured databases into a vector store (commonly Postgres with pgvector, Pinecone, or Weaviate).
• Retrieval design. Hybrid search combining semantic (embeddings) and lexical (BM25) retrieval, with re-ranking. Pure vector search rarely holds up in production.
• Prompt and refusal logic. How the bot answers, when it declines, when it hands off, how it cites sources. This is where most public bots embarrass themselves.
• Integration. Deployment into Intercom, Zendesk, Slack, Microsoft Teams, WhatsApp Business, your website, or a custom app. Auth, session handling, conversation history.
• Evaluation. An automated test suite of representative queries, run on every prompt or model change, scored for accuracy, hallucination rate, and tone.
• Observability and iteration. Logging every conversation, flagging low-confidence answers, feeding failures back into the knowledge base weekly.

An agency that skips any of these six is selling you a demo. The demo will work in the pitch and fail in week three of production.

RAG vs fine-tuning: what your agency should recommend

Almost every mid-market chatbot should start with RAG, not fine-tuning. The reasoning is practical: RAG lets you update the knowledge base without retraining, cite sources for auditability (essential under GDPR and for FCA-regulated firms), and swap the underlying model as OpenAI, Anthropic, and Google keep leapfrogging each other. Fine-tuning locks you into a model version and a training snapshot.

Fine-tuning earns its place in three cases: when you need a very specific tone or format that prompting cannot reliably produce, when latency matters enough that a smaller fine-tuned model beats a large general one, or when you have genuinely proprietary reasoning patterns (not just proprietary facts - those go in RAG). For a customer support bot, an internal HR assistant, or a sales enablement tool, RAG is almost always the right starting point.

If an agency's first instinct is to fine-tune, ask why. The honest answer is usually one of: they don't know how to build a good RAG pipeline, they're padding the invoice, or they have a specific reason they haven't explained. Push for the reason.

What a serious chatbot build costs in the UK

Pricing across the UK agency market varies more than any other AI service line, because "chatbot" covers everything from a £2k Voiceflow template to a £150k multi-agent system with custom retrieval. Realistic bands for mid-market work in 2026:

• £8k-£20k: A single-purpose bot on a defined knowledge base (help centre, product docs), deployed to one channel, with basic evaluation. Suitable for internal tools or lower-stakes customer FAQs. Typical timeline: 4-6 weeks.
• £25k-£60k: Multi-source knowledge ingestion, hybrid retrieval, integration with CRM or ticketing, refusal patterns, evaluation harness, monitoring dashboard. Suitable for customer support augmentation or a sales assistant. Typical timeline: 8-14 weeks.
• £75k-£200k: Multi-agent workflows (a router bot that delegates to specialist bots), custom tooling, deep CRM/ERP integration, human-in-the-loop review flows, high-compliance sectors. Typical timeline: 14-24 weeks.

Retainers for ongoing operation typically run £2k-£8k per month depending on volume and change velocity. This covers monitoring, weekly knowledge-base refreshes, prompt iteration based on failure logs, and model migrations when a better option ships. According to McKinsey's 2024 State of AI survey, organisations that treat GenAI deployments as "set and forget" see accuracy degrade materially within six months as source content drifts. Ongoing operation is not optional.

Anything under £5k for a real customer-facing bot is either a template product, offshored to a team you won't meet, or scoped so tightly it will disappoint. That's not a snob's point - it's arithmetic. A competent RAG build takes 80-150 hours of senior engineer time at minimum.

GDPR, the ICO, and what regulated buyers must ask

Any UK chatbot handling personal data falls under UK GDPR and the Data Protection Act 2018. The ICO published specific guidance on AI and data protection that any agency you hire should be able to discuss fluently. The questions that matter:

• Where is data processed? If the bot uses OpenAI's API, prompts are processed in the US. This is legal under the UK-US Data Bridge but must be documented in your DPIA and privacy notice.
• What is logged, for how long, and who can see it? Conversation logs often contain personal data. Retention policies should be explicit.
• How are subject access requests handled? If a user asks what the bot knows about them, you need a process.
• What's the model's training data policy? Both OpenAI and Anthropic offer enterprise agreements where your data is not used for training. Confirm this in writing.
• Have you completed a DPIA? For any customer-facing bot handling personal data, a Data Protection Impact Assessment is effectively mandatory.

FCA-regulated firms have additional obligations under the FCA's approach to AI, particularly around consumer duty and outcome fairness. If your agency cannot walk you through these, they are not ready for regulated work.

Evaluation: the single biggest tell of a serious agency

Ask any agency this question in the first meeting: "How will we know the bot is working, before and after launch?" The answer separates the credible from the rest.

A serious answer includes a golden dataset of 100-500 representative queries with expected answers, automated scoring using LLM-as-judge techniques (typically GPT-4 or Claude grading responses against rubrics), tracking metrics like retrieval precision, answer faithfulness, and refusal appropriateness, regression testing on every prompt or model change, and production monitoring with conversation-level flags for low confidence, off-topic drift, or user frustration signals.

A weak answer is "we'll test it thoroughly" or "our clients tell us it works." Vibes are not evaluation. If the agency does not have a repeatable evaluation methodology - and cannot show you a redacted example from a prior project - they will ship a bot that seems fine at launch and quietly degrades. You will find out from a customer complaint, not a dashboard.

The RAGAS framework and OpenAI's evals library are the two most common starting points. Ask which the agency uses and how they've customised it.

Build vs buy vs hybrid: when an agency is the right answer

You have three real options: buy a productised chatbot (Intercom Fin, Ada, Zendesk AI, HubSpot's Breeze), build in-house, or hire an agency. Each has a lane.

Buy productised when your use case is standard customer support on a well-structured help centre, you're happy with a per-resolution or per-seat pricing model, and you don't need deep integration with your operational systems. Intercom Fin, at time of writing, quotes $0.99 per resolution and handles a real chunk of tier-one support well. It's a serious product.

Build in-house when you have a competent AI engineering team already (not one engineer who read a LangChain tutorial), the bot is strategic enough to warrant permanent ownership, and you'll need to iterate faster than an agency retainer allows. This is the right answer for maybe 15% of mid-market firms.

Hire an agency when you want the bot in production in weeks not quarters, you need someone accountable for the outcome (not just the build), you want RAG done properly without your team learning on your budget, and you value optionality - agencies can hand over cleanly at 6 or 12 months if you decide to bring it in-house.

The hybrid model, increasingly common, is: agency builds and runs for 6-12 months, transfers ownership with documentation and training, keeps a lightweight advisory retainer. This lets you move fast now and own the asset later.

Red flags when evaluating an AI automation agency

Six specific things to watch for in pitches:

1. They lead with the model. "We use GPT-4o" is not a differentiator. The model is a commodity. The differentiator is retrieval quality, evaluation rigour, and integration craft.
2. No production references. Ask for two clients whose bots have been live for over six months. Ask what broke and how it got fixed. Silence tells you what you need to know.
3. They quote a fixed price without discovery. A real chatbot scope needs a week or two of discovery: what's the knowledge base, what channels, what integrations, what compliance regime, what evaluation criteria. A pre-discovery fixed price is a template being sold as bespoke.
4. No mention of refusal or escalation. A bot that answers everything is a bot that hallucinates. Ask specifically: when does the bot say "I don't know" or hand off to a human?
5. No self-hosted or private deployment option. For sensitive data, you may need Azure OpenAI, AWS Bedrock, or a self-hosted open-weights model. If the agency only knows the OpenAI API, they're limited.
6. They can't explain their monitoring stack. Langfuse, LangSmith, Helicone, or a custom logging pipeline - it doesn't matter which, but they should have one.

How the first 90 days should run

A well-run chatbot engagement has a clear shape. Weeks 1-2 are discovery: knowledge audit, use case definition, success metrics, DPIA if required, evaluation dataset construction. Weeks 3-6 are the initial build: retrieval pipeline, prompt engineering, first integration, internal alpha with your team hammering it. Weeks 7-10 are hardening: refusal patterns, escalation flows, evaluation runs, security review, gradual rollout to a subset of real users. Weeks 11-12 are launch and monitoring setup: full rollout, dashboards live, first retrospective scheduled.

By day 90 you should have: measurable performance against the golden dataset, real user conversations logged and reviewed weekly, a documented process for adding new knowledge, and a clear plan for month 4-6 improvements based on production data. If any of that is missing, you have a demo in production, not a system.

Frequently asked questions

How long does it take to launch an AI chatbot with an agency?

For a well-scoped mid-market build, expect 8-14 weeks from kickoff to production. The first two weeks are discovery and evaluation dataset construction. Weeks three to eight are the build: retrieval pipeline, prompts, integrations, refusal logic. Weeks nine to twelve are hardening, security review, and phased rollout. Simpler internal bots on a single knowledge source can ship in 4-6 weeks. Complex multi-agent systems in regulated sectors can take 20+ weeks. Any agency promising two-week production launches for a customer-facing bot is either using a template product or cutting corners you'll regret in month three.

What's the difference between an AI chatbot and an AI agent?

A chatbot answers questions from a knowledge base and can trigger simple actions like creating a ticket or updating a CRM record. An AI agent plans multi-step tasks, uses tools autonomously, and can operate across systems - for example, receiving a request, checking inventory, drafting a quote, sending it for approval, and following up. Agents are more powerful and more expensive to build and operate, with materially higher failure modes. For most 2026 use cases, a well-built chatbot with a few tightly scoped tool calls delivers more value with less risk than a fully autonomous agent. Agents earn their place when the workflow is complex and predictable.

Can we use our existing help centre or do we need to restructure content?

You can start with existing content, but expect a knowledge audit to surface gaps. Most help centres have inconsistent structure, outdated articles, missing edge cases, and content written for humans skimming rather than machines retrieving. A good agency will produce a content quality report in week two showing which articles retrieve well, which need rewriting, and which topics are missing entirely. Plan for 10-20% of your content team's time for the first three months to address gaps. The bot is only as good as the knowledge base, and improving the knowledge base improves human self-service too.

How do we handle GDPR when the chatbot uses OpenAI or Anthropic?

Three things need to be in place. First, an enterprise agreement with the model provider confirming your data is not used for training - both OpenAI and Anthropic offer this. Second, a Data Protection Impact Assessment covering the data flows, retention periods, and lawful basis, particularly relevant for customer-facing bots handling personal data. Third, transparent user disclosure that they're interacting with AI and how their data is processed. For higher-sensitivity use cases, consider Azure OpenAI (data stays in the Microsoft tenancy) or AWS Bedrock with Anthropic. The ICO's guidance on AI and data protection is the authoritative reference and any agency you hire should be fluent in it.

What ongoing costs should we budget for after launch?

Budget three cost lines. Model API costs typically run £200-£3,000 per month for mid-market volumes, scaling with conversation count. Infrastructure (vector database, logging, monitoring) usually £100-£500 per month if self-hosted, more if using managed services like Pinecone at scale. Agency retainer £2,000-£8,000 per month for monitoring, weekly knowledge refreshes, prompt iteration, and model migrations. Add internal time: someone on your side needs to own the bot, review failure logs weekly, and coordinate content updates. Total ongoing cost for a mid-market customer support bot typically lands £3,000-£12,000 per month all-in. Chatbots are not one-off projects.

How do we measure ROI on a chatbot investment?

The right metrics depend on the use case. For customer support: deflection rate (conversations resolved without human), average handle time reduction for escalated conversations, CSAT on bot-resolved conversations, cost per resolution vs human channel. For sales enablement: rep time saved, quality of information retrieved, adoption rate. For internal bots: query volume, time-to-answer reduction, employee satisfaction. Avoid vanity metrics like total conversations - a bot that gets 10,000 conversations but resolves 2% is worse than one that gets 1,000 and resolves 60%. Set the baseline before launch and measure at 30, 90, and 180 days. Expect 6 months to hit steady-state performance.

What happens if we want to switch agencies or bring it in-house?

This is the single most important question and the one agencies least want asked. Insist on three things in the contract: all code, prompts, evaluation datasets, and configuration are your property and delivered in a working repository, the knowledge base and vector database are hosted in your cloud accounts (or transferable), and the agency will provide a defined handover period with documentation and training at reasonable day rates. Vendor lock-in in AI is easy to create accidentally through proprietary orchestration layers or opaque prompt engineering. A serious agency will welcome this conversation because they've built handovers before. A hesitant agency is telling you something.

Should we use n8n, LangChain, or a custom framework for the automation layer?

All three have a lane. n8n is excellent for the workflow automation surrounding the chatbot - ticket creation, CRM updates, notifications, escalations - and can be self-hosted for data control. LangChain or LlamaIndex are useful for the retrieval and orchestration inside the bot itself, though many teams now find them heavy and prefer lighter custom code. A custom framework earns its place when you have specific requirements the frameworks fight you on. A common production stack: FastAPI or Next.js for the API surface, custom retrieval code using pgvector, and n8n handling downstream workflows. The choice matters less than whether the agency can defend it clearly.

Choosing well matters more than choosing fast

The chatbot market in 2026 is crowded with agencies that can demo well and ship poorly. The gap between the two shows up in month three, not month one, which is why the buying decision is so hard. The best signal is not the demo - it's how the agency talks about evaluation, monitoring, failure modes, and handover. Serious builders lead with those. Sales-first shops lead with the model name and the UI.

Scope the first project narrowly. One use case, one channel, one clear success metric. Prove the operating model works, then expand. Chatbots that try to do everything on day one usually do nothing well by day 90.

At AI Advisory we build chatbots the way we'd want one built for our own operations: RAG-first, evaluation-driven, self-hostable where the data demands it, and handed over cleanly when the client is ready. If you're scoping a chatbot project and want a second opinion on the approach - or a proper build partner - get in touch.

Further reading

Sources referenced for context not directly cited in the body:

Ready to put this into production? book a discovery call.

• Artificial intelligence | ICO