Why Most Agency Engagements Fail Before a Line of Code Is Written
The majority of failed software agency engagements share one thing in common: the client skipped the vetting process. They liked the sales pitch, the portfolio looked polished, the price was competitive, and they signed. Three months later they are staring at a half-built product, a codebase full of shortcuts, and an agency that has stopped returning calls promptly.
I have seen this pattern play out dozens of times. A founder gets three proposals, compares prices, picks the one that feels right, and moves forward. That is not vetting. That is shopping. Vetting is the deliberate process of stress-testing an agency's claims before you hand them your budget, your timeline, and your product vision. It requires specific questions, structured evaluation, and a willingness to walk away from agencies that cannot back up their pitch with evidence.
The cost of getting this wrong is severe. Direct financial losses typically range from $50,000 to $200,000. But the indirect costs are worse: six to twelve months of lost time, a codebase that needs to be rewritten, a demoralized team, and missed market windows that do not reopen. Every dollar you spend vetting agencies before signing saves you ten dollars in recovery costs later.
This guide gives you the complete framework. It covers what to look for, what to ask, and exactly how to evaluate the answers. If you follow this process, you will not eliminate all risk, but you will eliminate the avoidable failures that account for the vast majority of bad agency experiences.
Red Flags in Initial Conversations That Should End the Process
Your first two or three conversations with an agency tell you almost everything you need to know. The sales process is the agency's best behavior. If something feels off now, it will be dramatically worse once they have your deposit and their attention shifts to the next prospect. Here are the specific red flags that should end your evaluation immediately.
No Discovery Process
If an agency gives you a fixed price within 48 hours of your first conversation, without conducting a thorough discovery session, they are not pricing your project. They are guessing. A responsible agency invests real time understanding your requirements, your users, your business model, and your technical constraints before committing to a number. This usually takes the form of a paid discovery phase lasting one to three weeks and costing $3,000 to $15,000 depending on project complexity.
Agencies that skip discovery are doing one of two things. They are either dramatically padding their estimate to absorb the unknown, which means you overpay. Or they are quoting too low because they do not understand the scope, which means you get hit with change orders throughout the project. Neither outcome serves you.
Fixed Price for a Vague Scope
This is related but distinct. Some agencies will hear a loose description of what you want, say "that will be $80,000 fixed price," and present it as certainty. In reality, they have no idea what $80,000 of work looks like for your project because you have not defined the scope in enough detail. A fixed price is only meaningful when attached to a detailed scope of work with explicit acceptance criteria. Without that, a fixed price is a fiction that both parties pretend is real until the change orders start arriving.
They Cannot Show Code Samples
Every legitimate agency has code they can share. It might be anonymized, it might require an NDA, it might be from an internal project or an open-source contribution. But they can show you something. An agency that claims they cannot share any code due to client confidentiality, with no alternative offered, is hiding something. Either their code quality is poor, their senior engineers did not actually write the code they are taking credit for, or their development process does not produce work they are proud of. None of those possibilities should give you confidence.
The Sales Team Disappears After Signing
Pay attention to who is in your sales meetings. If you are talking to a polished business development team but have not met a single engineer or project manager who would actually work on your project, that is a warning. Ask directly: "Will anyone in this meeting be involved in the day-to-day work on my project?" If the answer is no, ask to meet the team that will. An agency that resists this request is telling you that the people who sold you are not the people who deliver.
The Portfolio Deep-Dive: Go Beyond Screenshots
Every agency has a portfolio page with attractive screenshots and brief case studies. These are marketing materials, not evidence of quality. A polished screenshot tells you nothing about code quality, architecture decisions, scalability, or whether the client was satisfied with the process. You need to go deeper, and a serious agency will welcome the scrutiny.
Ask for GitHub Access or a Code Walkthrough
This is the single most revealing request you can make. Ask the agency to walk you through the codebase of a completed project. If they use GitHub or GitLab, ask for read access to a repository. You do not need to understand the code yourself. What you are looking for is their willingness to be transparent. Agencies with clean, well-structured codebases will be eager to show them off. Agencies with messy, undocumented code will deflect.
If they grant access, have your technical advisor or a hired independent reviewer look at the following: Is there a consistent folder structure? Are there automated tests? Is the commit history clean and descriptive, or is it full of commits like "fix" and "update"? Is there a README that explains how to set up the project? These basics reveal whether the team follows professional engineering practices or just ships code that happens to work.
Verify the Portfolio Projects Are Real
It sounds obvious, but verify that the projects in the portfolio actually exist and that the agency actually built them. Search for the product online. Download the app if one exists. Look up the client company on LinkedIn. If the portfolio piece is a mobile app, check the reviews. If it is a web platform, see whether it is still live and functioning. Agencies occasionally exaggerate their involvement in projects or showcase work done by team members at previous companies.
Ask About Post-Launch Outcomes
A finished product is not a success story. A product that launched, acquired users, and continued to function reliably is a success story. Ask the agency what happened after launch. Did the client continue working with them? Did the product scale? Were there significant technical issues in the first three months? An agency that tracks post-launch outcomes demonstrates that they care about results, not just deliverables. An agency that loses track of projects after handoff is a build shop, not a partner.
Reference Checks: The Questions That Actually Reveal the Truth
Reference checks are the most underused tool in agency vetting. Most people ask vague questions and get vague answers. If you ask a former client "How was your experience with Agency X?" you will get "It was good, they did solid work." That tells you nothing. The quality of your reference check depends entirely on the specificity of your questions.
The Questions You Should Ask Every Reference
Start with these, and adapt based on the conversation:
- "Would you hire them again for a project of similar scope?" This is the most important question. A yes with enthusiasm is a strong signal. A yes with qualifications ("for certain types of work") is worth probing. A hesitation or redirect is a red flag.
- "How did the actual timeline compare to the estimated timeline?" Every project runs over. You want to know by how much. A 10 to 20 percent overrun is normal. A 50 to 100 percent overrun suggests the agency does not estimate well or does not manage scope creep.
- "What happened when something went wrong?" This is where you learn about character. Did the agency own the problem and fix it at their cost? Did they blame the client? Did they go silent? The answer to this question predicts how your hardest moments will go.
- "How was the code quality after handoff?" If the reference had another team take over the codebase, ask how that transition went. Was the code well-documented? Could the new team work with it immediately, or did they need weeks to understand the structure? This tells you whether the agency builds for maintainability or just for delivery.
- "Who actually did the work on your project?" Ask whether the senior people they met during the sales process were the ones writing code. In many agencies, senior engineers are involved in sales and architecture but the actual coding is done by junior developers. That is not inherently bad, but you should know the reality.
Go Beyond the Provided References
The references an agency gives you are curated. They will always be satisfied clients. To get a fuller picture, do your own research. Look up the agency on Clutch, GoodFirms, and Google Reviews. Search for the agency name on Reddit and Hacker News. Look at the agency's LinkedIn page and see which companies they list as clients, then reach out to contacts at those companies directly. The references the agency did not choose to give you are often the most informative ones.
Technical Assessment: Having an Expert Review Their Work
If you are a non-technical founder, the technical assessment is the part of vetting where you are most vulnerable to being misled. An agency can describe their architecture in impressive-sounding language, and without technical knowledge you have no way to evaluate whether it is genuinely sound or just well-presented. The solution is simple: bring in your own technical evaluator.
Who Should Do the Technical Review
If you have a CTO or technical co-founder, they should lead this. If you do not, hire a fractional CTO or an independent senior engineer for a focused review. This typically costs $500 to $2,000 for a thorough evaluation and is one of the highest-ROI investments you can make during the vetting process. The reviewer should be someone with no relationship to the agency, no incentive to approve or reject them, and enough experience to evaluate architecture, code quality, and engineering processes.
What the Technical Reviewer Should Evaluate
Give your reviewer a clear brief. Ask them to assess the following areas:
- Architecture decisions: Are the technology choices appropriate for the project requirements? Is the system designed to scale, or will it need a rewrite at a few thousand users? Are there obvious single points of failure?
- Code quality: Is the code clean, readable, and well-organized? Are there automated tests? Is there consistent error handling? Does the code follow the conventions of its language and framework?
- DevOps and infrastructure: How is the code deployed? Is there a CI/CD pipeline? Are environments properly separated (development, staging, production)? Is there monitoring and logging in place?
- Security practices: Are there obvious vulnerabilities? Is authentication handled properly? Are API keys and secrets managed securely? Is data encrypted at rest and in transit?
Your reviewer should produce a plain-language summary with a clear recommendation: proceed, proceed with conditions, or walk away. If you need guidance on how to read a technical proposal, that resource will help you interpret the evaluation alongside the agency's own documentation.
Contract Review: The Clauses That Protect You
The contract is where your vetting process becomes legally enforceable. Everything you learned during evaluation, every promise the agency made during the sales process, needs to be reflected in the contract. Verbal assurances are worthless if the engagement goes sideways. Here are the specific clauses that protect you and the ones you should push back on.
IP Ownership
This is non-negotiable. You must own 100% of the intellectual property created for your project. The standard language is "work for hire" with full IP assignment upon payment. Watch for agencies that retain a license to reuse components, frameworks, or architecture from your project. Some agencies build reusable internal tools and include them in client projects, which means parts of "your" codebase are actually shared IP. Get clarity on what is custom-built for you versus what is the agency's pre-existing tooling.
Source Code Escrow
For projects where the agency hosts or manages the production environment, insist on source code escrow. This means a neutral third party holds a copy of your complete codebase that is released to you if the agency goes out of business, breaches the contract, or becomes unable to perform. Without escrow, you are dependent on the agency's continued existence and goodwill for access to your own product.
Termination Clauses
Your contract must address what happens if you end the engagement early. Key questions: What do you owe for work completed but not yet invoiced? How quickly must the agency deliver all code, credentials, and documentation after termination? Is there a notice period, and what happens to work in progress during that period? A good termination clause protects both sides. You should be able to exit without penalty beyond payment for completed work. The agency should not be left without compensation for legitimate work done in good faith.
Change Order Process
Every project evolves. Requirements change, priorities shift, and new information emerges. Your contract needs a clear process for handling scope changes. The standard approach: the client submits a change request, the agency estimates the cost and timeline impact within a defined window (48 to 72 hours), and no work begins until both parties approve the change order in writing. Without this process, you will either get surprise invoices or, worse, an agency that quietly absorbs small changes until they present you with a massive "catch-up" bill at the end of a sprint.
Warranty Period
A standard contract includes a 30 to 90 day warranty period after delivery during which the agency fixes bugs at no additional cost. This only covers defects in the delivered work, not new feature requests. Make sure the contract defines what constitutes a bug versus a change request, because that distinction will matter when you find issues after launch. If an agency refuses to include any warranty period, reconsider whether they are confident in their own work quality.
Team Structure Transparency: Know Who Actually Builds Your Product
One of the most common sources of frustration in agency engagements is the gap between who you meet during sales and who actually works on your project. Sales conversations feature senior partners and experienced architects. The day-to-day work often falls to mid-level or junior developers you have never met. This is not always a problem, but it is always something you should know about in advance.
Questions That Reveal the Real Team Structure
Ask these questions before you sign, and get the answers in writing:
- "Who specifically will be assigned to my project, and what are their roles?" Get names, titles, and LinkedIn profiles. Research their backgrounds. How long have they been at the agency? What projects have they worked on?
- "What percentage of their time will be dedicated to my project?" Many agencies staff developers across multiple projects simultaneously. A developer splitting time between three clients gives each one roughly 30% of their attention. That is not the same as a full-time dedicated resource, and it should not be priced the same way.
- "Can I interview the developers who will work on my project?" A confident agency will say yes. A 30-minute technical conversation with the lead developer tells you more about the team's capability than any portfolio or case study. If the agency pushes back on this, ask yourself what they are protecting you from seeing.
- "What happens if a key team member leaves during the project?" Staff turnover is a reality. A good agency has a knowledge transfer process, documentation standards, and bench depth to handle transitions without derailing the project. An agency that has not thought about this will leave you scrambling if their lead developer takes another job.
The Subcontracting Question
Some agencies subcontract portions of their work to freelancers or offshore teams. This is not inherently bad, but you need to know about it. Ask directly: "Do you use any subcontractors or offshore resources?" If they do, ask what quality control processes they have in place, who reviews the subcontracted work, and whether the subcontractors are under the same contractual obligations as the agency's employees. Hidden subcontracting is a dealbreaker because it means the agency is not accountable for the people doing the work.
Communication Assessment: Sales Behavior Predicts Project Behavior
Here is a principle that has proven true in every agency engagement I have observed: how an agency communicates during the sales process is the ceiling of how they will communicate during the project. Sales is when they are most motivated to impress you. If their response time is 48 hours during sales, expect 72 hours or more during the project. If they miss a deadline on delivering the proposal, they will miss deadlines on delivering features.
What to Track During the Sales Process
Keep a simple log of your interactions and note the following:
- Response time: How long does it take them to reply to emails and messages? Under 24 hours is good. Same-day is excellent. More than 48 hours without explanation is a warning sign.
- Follow-through: When they say "I will send you that case study by Friday," do they? Track every commitment and whether it was met. Small follow-through failures during sales become large follow-through failures during the project.
- Clarity: Are their communications clear and specific, or vague and full of jargon? An agency that communicates clearly during sales will communicate clearly during the project. An agency that hides behind buzzwords is either incapable of plain language or actively avoiding specificity.
- Proactive communication: Do they surface risks or concerns without being asked? An agency that mentions a potential complication with your proposed approach during the sales process is demonstrating the kind of honesty you want during the build. An agency that agrees with everything you say is telling you what you want to hear.
Test Their Communication with a Curveball
Midway through the sales process, introduce a change. Shift a requirement, adjust the timeline, or ask about adding a feature that was not in the original discussion. Watch how they respond. Do they thoughtfully explain the impact of the change on scope, cost, and timeline? Or do they just say "sure, we can do that" without addressing the implications? The first response shows you a team that manages scope responsibly. The second shows you a team that will say yes to everything and figure out the consequences later, usually at your expense.
If you are evaluating multiple agencies simultaneously, this comparison becomes especially powerful. The same curveball delivered to three agencies will produce three very different responses, and those differences tell you almost everything about what the working relationship will look like. For more on structuring this evaluation process, see our guide on how to choose the right development agency.
The Complete Agency Vetting Checklist
Use this checklist to evaluate every agency on your shortlist. Score each item as pass, partial, or fail. An agency with more than three "fail" marks should be removed from consideration. An agency with all passes and no more than two partials is a strong candidate.
Initial Conversation
- Discovery process: Does the agency propose a structured discovery phase before quoting a price?
- Scope clarity: Does the agency ask detailed questions about your requirements, users, and business model?
- Honest timelines: Does the agency give estimates as ranges with stated assumptions, not guarantees?
- Code samples: Can the agency provide code samples or repository access for review?
- Team introductions: Has the agency introduced you to the people who will actually work on your project?
Portfolio and Technical Assessment
- Relevant experience: Has the agency built at least two projects similar to yours in domain or technology?
- Code quality: Has an independent technical reviewer assessed the agency's code and given a positive evaluation?
- Architecture rationale: Can the agency articulate trade-offs in their past architecture decisions?
- Post-launch track record: Can the agency describe what happened with past projects after delivery?
- Verified portfolio: Have you confirmed that portfolio projects exist and that the agency built them?
Reference Checks
- Repeat business: Would at least two references hire the agency again without hesitation?
- Timeline accuracy: Did past projects deliver within 20% of the estimated timeline?
- Problem handling: Can references describe how the agency handled a specific problem or failure?
- Code maintainability: Can references confirm the codebase was maintainable after handoff?
- Independent research: Have you found reviews or references outside the agency's curated list?
Contract and Process
- IP ownership: Does the contract assign 100% IP ownership to you upon payment?
- Source code escrow: Is there a provision for source code escrow or guaranteed code access?
- Termination clause: Can you exit the engagement with reasonable terms and full code delivery?
- Change order process: Is there a documented process for handling scope changes?
- Warranty period: Does the contract include a 30 to 90 day post-delivery warranty?
Communication and Team
- Response time: Has the agency consistently responded within 24 hours during the sales process?
- Follow-through: Has the agency met every commitment made during the sales process?
- Dedicated team: Are named, full-time (or near full-time) resources assigned to your project?
- Subcontracting transparency: Has the agency disclosed whether subcontractors will be involved?
- Staff turnover plan: Does the agency have a documented process for handling team transitions?
Print this list. Use it for every agency conversation. The agencies that score highest are the ones worth your investment. If you have already started your search and want help structuring a formal technical RFP, that guide walks you through the document that makes agencies compete on substance instead of sales polish.
Start Your Search with Confidence
Vetting a software development agency takes effort. It requires specific questions, independent technical review, thorough reference checks, and careful contract analysis. Most founders skip parts of this process because it feels slow when they are eager to start building. But the founders who invest two to three weeks in rigorous vetting consistently avoid the six to twelve month recovery cycles that follow a bad agency choice.
The framework in this guide is the same one we recommend to every founder who asks us for advice on selecting a development partner. It works because it focuses on evidence over impressions, specifics over generalities, and behavior over promises. An agency that passes this evaluation has earned your trust through demonstrated competence, not just persuasive selling.
If you are in the early stages of evaluating agencies and want a second opinion on your shortlist, or if you need help defining your requirements before you start the conversation, we are happy to help. Book a free strategy call and we will walk through your vetting process together, review the proposals you have received, and help you make a decision you will not regret.
Need help building this?
Our team has launched 50+ products for startups and ambitious brands. Let's talk about your project.