Politik hit #2 on the App Store.Read more

AI & strategy

Hire an AI Developer: What to Check Before You Sign

Most AI features in 2026 are built by product engineers who wire models in through APIs, not by researchers who train them. Here is what to check before you sign with a freelancer or agency, what it costs on the market, and the red flags that should end the call.

Nate Laquis10 min read

What to Check Before You Hire an AI Developer

Before you hire an AI developer, check four things: AI features they shipped that real users touch today, how they test answer quality with evals, what each request will cost you, and a contract that puts the code, the prompts and the model API keys in your name. A developer who can show all four is worth talking price with. One who can only show a demo is not.

This article is for a founder who wants something specific built: an assistant inside the product, search and answers over the company's documents, an agent that takes actions, or extraction that turns invoices and contracts into clean data. You have a budget, a deadline, and three proposals that all sound the same. Here is how to tell them apart.

What an AI Developer Does in 2026

The person you need is almost never a machine learning researcher. Most AI products shipping today call a model from OpenAI, Anthropic or Google over an API. The work is product engineering around that call: retrieval that finds the right paragraph in your documents, prompts that hold up across thousands of inputs, tool calls that let an agent do things, evals that catch regressions, and the usual app work of auth, billing, logging and a screen people want to use.

So when you hire an AI developer for a product feature, you are hiring a strong full stack engineer who has done that integration work many times. They know how chunking choices change retrieval quality. They know when a cheaper model is good enough. They know that a prompt change on Tuesday can break an answer that worked on Monday, and they have a test set that tells them.

When you do need an ML engineer

There are cases where API work is not enough. You need a machine learning engineer when the core of the product is a model that does not exist yet: a vision model trained on your own labeled images, a ranking or recommendation system on years of behavior data, forecasting on proprietary signals, or a fine-tuned open-weight model you host yourself because a contract forbids sending data to a vendor. That is a different hire with a different skill set. We wrote a full guide on how to hire AI engineers that covers the AI engineer versus ML engineer split for full-time roles, so we will not repeat it here.

A quick test: if a smart engineer could get a rough version working in a week with a model API and your data, you need an AI product developer. If nobody can tell you whether it will work until a model is trained, you need ML expertise first.

Seven Things to Check Before You Sign

Run every candidate, freelancer or agency, through these. Ask for evidence, not descriptions.

Two developers reviewing work together at a shared desk with a laptop and an external monitor

1. AI features in production with real users

Ask for links to live products, not videos. Then ask what broke after launch. Anyone who has shipped an AI feature has a story about the model answering confidently from the wrong document, a user pasting a 300-page PDF, or a provider changing behavior under them. If the answer is that nothing went wrong, nothing shipped.

2. How they handle evals

An eval is a set of real inputs with known good outputs that runs every time a prompt, model or retrieval setting changes. Ask to see one from a past project. A good answer names the size of the set (fifty to a few hundred cases is normal for a first version), how cases get added when users report bad answers, and how they score them: exact match for extraction, a rubric or a second model as judge for open answers, a person for the hard ones. Tools like Braintrust, LangSmith and Promptfoo are common. A spreadsheet works too. What matters is that it exists.

3. Cost per request and model routing

Ask them to estimate what one user action will cost in tokens before they write code. A good developer will walk through it: prompt size, retrieved context, output length, how many calls per action. Then ask about routing. Most products send easy work, such as classifying a message or summarizing a short note, to a small cheap model and save the flagship for the hard step. The price gap between a vendor's smallest model and its flagship is often more than ten times per token, so routing decides whether your margins survive growth.

4. Hallucination guardrails

Ask how the system behaves when the answer is not in your data. You want to hear about citations a user can click, a refusal when retrieval comes back empty or weak, structured outputs validated against a schema for extraction, and human review on anything with legal or financial weight. Nobody can promise zero wrong answers. A good developer tells you how wrong answers get caught and what the user sees.

5. Data privacy

Ask which vendor terms they rely on. The API plans from the major vendors do not train on your data by default, and enterprise tiers add zero retention options and regional hosting. If you handle health records, you need a BAA with the model vendor, and not every model or feature is covered. Ask where embeddings live, what gets logged, and how a user deletes their data. If the developer has never read a vendor's data processing terms, that is your answer.

6. Fallbacks when a provider is down

Every model API has outages and rate limits. Ask what happens to your product when the primary provider returns errors for an hour. The good answer is a thin layer between your app and the vendors, with retries, timeouts and a second provider ready for the most important calls. Gateways such as OpenRouter, Portkey or the Vercel AI SDK make this an afternoon of work, if someone plans for it.

7. Who owns the prompts and the code

Prompts, eval sets and retrieval settings are part of your product. They should live in your repository, not in the developer's account on some prompt platform. Ask directly. Some agencies keep a proprietary framework and license it to you. That can be fine, but you should know before you sign, not when you try to switch teams.

Red Flags That Should End the Call

Some answers tell you enough to stop.

  • A demo-only portfolio. Slick videos of chatbots answering five perfect questions. Demos are easy. Production is where the work is.
  • "We fine-tune everything." Fine-tuning has its place, mostly for tone, format or narrow classification at scale. For answering from your documents, retrieval beats it and costs far less to change. A team that reaches for training first is selling the expensive path. Our breakdown of fine-tuning versus RAG versus prompt engineering covers when each fits.
  • No eval plan. If quality is judged by the developer trying a few prompts and saying it looks good, you will find the failures in production, from customers.
  • A fixed quote without seeing your data. The data decides the build. Clean PDFs and scanned faxes are different projects. So are 200 documents and 2 million. A quote that arrives before anyone has looked is a guess with a number on it.
  • Their API keys, their cloud account. You end up unable to see your own costs or move without their help.
  • One model, forever. Anyone who has bet the architecture on a single vendor has not watched prices and rankings change every few months.

One more, softer: listen for how they talk about what the model cannot do. A developer with production scars will tell you the limits before you ask.

Freelancer, Agency or In-House

Each option fits a different situation. These are market ranges for 2026, US unless noted, and they move with seniority and location.

OptionMarket rateBest forWatch out for
Freelancer$100 to $225 an hour for experienced US engineers. $35 to $80 offshore.One well-scoped feature added to an app that already worksOne person covers AI, backend, frontend and design. Vacations and illness stop the project.
Agency$120 to $250 an hour blended in the US and Western Europe. A first AI product often lands between $40,000 and $200,000.A new product, or a feature that needs design, backend and AI work at once, on a deadlineSenior people on the sales call, juniors on the code. Ask who writes it.
In-house hireRoughly $160,000 to $250,000 base for a senior AI engineer in a major US market, plus equity and benefitsAI is the core of the company and work continues for yearsTwo to four months to hire. One person cannot cover everything at the start.

Freelance marketplaces show much lower numbers, sometimes $15 to $40 an hour. Some of those people are good. Most are learning on your project, and the checks above matter even more there.

When Hiring an Agency Is the Wrong Call

We are an agency, so take this from someone with a reason to say the opposite. An agency is the wrong choice in more cases than agencies admit.

  • The feature is small and your team is strong. If you already have engineers and the job is a summarize button or a support reply draft, one of them can learn the pattern in two weeks with a vendor's docs. Paying an agency to do it also means paying them to learn your codebase.
  • AI is the whole company. If the model behavior is your edge and you will tune it every week for years, that knowledge has to live inside your company. Hire, and use outside help only to get the first version out while you recruit.
  • You have not talked to users. If you cannot name the ten people who will use it next month, a build of any size is premature. Spend two weeks on interviews and a clickable prototype first.
  • The budget covers the build and nothing after. AI features need care after launch: new eval cases, model upgrades, prompt fixes when a vendor changes behavior. If there is no money for the first six months after launch, a cheaper, smaller scope with a freelancer you can keep is the better plan.

An agency earns its fee when you need a whole product team at once, with design, mobile or web, backend and AI working off one plan, and you need it live in a few months rather than after a hiring cycle.

Run a Paid Test Before the Big Contract

The cheapest way to know if a developer is good is to pay them for a small piece of real work. Two forms work well.

Close-up of a code editor on a laptop screen showing functions and line numbers

Paid discovery

One to three weeks, usually a few thousand dollars to the low tens of thousands on the market. The developer looks at your real data, writes a short spec, estimates cost per request, names the risks, and gives you a fixed or capped quote for the build. You own the output whether or not you continue. If a team will not do discovery and wants to start building on day one, ask what their quote is based on.

A spike on the hardest part

Pick the part most likely to fail. For document extraction, that is your ugliest twenty files. For RAG, it is the fifty questions your users ask most, with the answers you expect. Ask for a working version in a week or two, with an eval that scores it. You learn three things at once: whether the idea works on your data, how the developer communicates, and whether their code is something another engineer could pick up.

Judge the test on the eval numbers and the honesty of the write-up, not on how polished the screen looks.

What Goes in the Contract

Most AI contract problems come from three clauses that were left vague. Get them in writing.

Person signing a paper contract at a desk

Intellectual property

All code, prompts, eval sets, retrieval configuration and documentation transfer to you on payment. If the developer brings an existing library or framework, it should be listed by name with a perpetual license for you to use and modify it. The repository lives in your GitHub organization from the first commit.

Model API keys in your account

Your company opens the accounts with OpenAI, Anthropic, Google or whichever vendor you use, and adds the developer as a member. Same for the vector database, hosting and monitoring. You see the bills, you can cut access in a minute, and nothing breaks when the engagement ends. Ask for usage alerts and spend limits on day one.

Data handling

Spell out which data the developer can access, where it can be copied, which model vendors it may be sent to, and that it gets deleted from their machines at the end. If you have customer data under GDPR or HIPAA, the developer signs a DPA or BAA as needed, and the model vendor setup has to match.

Also worth a line each

  • Acceptance criteria tied to the eval, such as extraction accuracy on a held-out test set.
  • A support period after launch at an agreed rate, and a handover with a recorded walkthrough.
  • Who pays model usage during development. Usually you, through your own account.

What It Costs

The hourly rate is half the story. The other half is scope and the monthly model bill once real users arrive.

On the market, a single AI feature added to an existing app tends to run $15,000 to $60,000. A new product built around AI, with its own app, accounts and billing, tends to run $60,000 to $200,000 for a first version. Agents that take actions in other systems sit at the upper end, because every integration needs its own error handling and tests. Model usage is separate and scales with users, so ask for a per-user monthly estimate before you sign.

For a full breakdown by product type, see our guide on how much it costs to build an AI product. For app costs in general, including design, backend and launch, use our app development cost page.

How We Build AI Features for Founders

This is what we do. We build AI features and AI products for founders, from the first scoping call to the App Store or the live web app, and the checks above are the ones we expect to be held to.

Dropify is a good example. We built it as a Next.js web app with a Python ML backend and OpenAI in the stack: an AI product research engine that scans marketplaces and social trends, an ads intelligence dashboard, and a private supplier network with automated order routing. Before building, we analyzed the workflows of more than fifty successful dropshippers to find where the friction was. It went from idea to revenue in 12 weeks, reached $10K in monthly recurring revenue in its first month, had more than 500 active users at launch and kept 85% of them through month one.

If you are about to sign with someone, we are happy to look at your plan and tell you where we think the risk is, even if you build it with someone else. Book a free strategy call

hire ai developerai development agencyai freelancer ratesllm evalsai development contract

AI in your product, shipped.

Thirty minutes. You leave with a plan and a number.

Book a free call