How to Evaluate Software Development Outsourcing Companies

Every vendor in this market describes itself the same way: senior engineers, rigorous vetting, agile delivery, a partnership rather than a transaction. The words are free, and after four sales calls they blur into one. The problem in evaluating a software development outsourcing company is not a shortage of information, it is that nearly all of it is supplied by the party being evaluated.

Our software development outsourcing page covers the questions worth putting to a vendor directly. This post covers what surrounds them: how to build a shortlist, score it without fooling yourself, verify a team rather than a capability, run a reference call that produces facts, read an SLA and an exit clause, and test a finalist with real work.

Decide what you are buying before you evaluate anyone

There are two distinct purchases here, and evaluating one with the other’s criteria is the most common way a selection goes wrong.

A staffing purchase puts individual engineers inside your team, under your management, working from your backlog. You are evaluating a recruiting and retention engine plus the specific humans it produces; the firm’s project management methodology barely matters, because you are supplying it. Weight the individual technical interviews, engineer tenure, bench depth, replacement speed and time zone overlap.

A delivery purchase hands over an outcome. You are evaluating estimation discipline, project leadership, change control and what the vendor does when it is late. The individuals matter less, because the vendor owns the result and can rotate people to protect it. Weight the estimate behind the proposal, the named project lead, how change orders are handled, and what happens contractually when a milestone slips.

Most firms sell both, and excellence at one does not imply the other. Decide which you need first. Our engagement models page compares the four shapes these arrangements take, and staff augmentation vs. dedicated team vs. fixed price walks through the tradeoffs. If the work is an AI product specifically, the criteria shift again: see how to choose a generative AI development company.

Build a shortlist small enough to run properly

Four to six vendors is right. Fewer gives you no comparison; more turns it into a scheduling exercise, when depth is what makes it work.

Disqualify on hard constraints before you spend a call on anyone:

• Working-day overlap. If your engineers need answers during their afternoon, a two-hour overlap is disqualifying, whatever else is true. Where the work happens is its own decision, covered on our nearshore software development page.
• Real depth in your stack. Not a logo wall. Ask how many engineers are working in your primary language right now, and on what.
• Regulatory reality. Under PCI DSS, HIPAA or GxP, a vendor who has never shipped under an audit will learn on your budget.
• Size fit. You want to matter. A vendor’s smallest client and its largest are both uncomfortable positions.

Write the scorecard before the first call

A weighted scorecard is not rigor for its own sake. Evaluations drift toward whoever presented most recently and most confidently; a scorecard written before you meet anyone is a record of what you actually needed. Weight the criteria against your binding constraint, then score each vendor 1 to 5, noting the evidence behind each score:

Criterion	Weight	Evidence that counts
Named engineers in your stack	25%	Your own technical interview of the individuals proposed
Pilot performance	20%	A paid piece of real work in your repository
Commercial and exit terms	15%	The draft MSA and SOW, read before the final call
Retention and continuity	15%	Attrition on comparable accounts, confirmed by a reference
Communication and overlap	15%	A working session in your tools, not a scheduled demo
Domain and regulatory fit	10%	Work shipped under the same constraints

One rule makes this work: no score without an artifact, for every vendor on the list including us. A vendor’s assertion is not evidence. A case study is evidence only that a case study was written. A resume is not evidence until you have interviewed the person attached to it.

Build your own cost baseline before you read anyone’s number. UK government procurement makes this a formal step: the Sourcing Playbook requires a Should Cost Model Estimate on complex outsourcing projects, because it “provides a better understanding of the costs associated with different service delivery models and helps to protect government from ‘low cost bid bias’.” The same logic applies at any scale: work out what the outcome should cost you, including your own management time, before a proposal anchors you. For the inputs behind the number, see how much IT staff augmentation costs and nearshore developer rates in Latin America.

Verify the team, not the capability

Capability claims describe a firm. You do not get a firm, you get five people, and that distance is where most disappointment lives. Asking to interview the proposed engineers is the baseline; a vendor who resists has answered a different question than the one you asked. Go further and verify four things about each named person:

1. Tenure. How long have they been with the vendor? A team assembled from last month’s hires is a different risk than one that has worked together for three years.
2. Allocation. What else are they on, and how much of their week are you actually buying? “Dedicated” has a wide range of meanings.
3. Substitution. Name key personnel in the statement of work and require written consent before anyone is swapped. That clause is the difference between a team and a staffing queue.
4. Continuity. Does a replacement shadow the person leaving, or simply inherit the tickets?

US federal procurement encodes the same instinct: under FAR 15.305, a past performance evaluation “should take into account past performance information regarding predecessor companies, key personnel who have relevant experience, or subcontractors that will perform major or critical aspects of the requirement.”

The failure mode all of this prevents is the oldest one in professional services: the people who sold you are not the people who show up. A principal runs the pitch, fields every hard question impressively, and is never seen again once the SOW is signed. The countermeasure is not suspicion, it is specificity.

For our part: you receive 3–5 profiles, typically within 5–10 business days for standard stacks, and you run the technical assessment and the 1:1 yourself. Every engineer clears four gates before meeting a client (technical assessment, live coding review, English proficiency and culture fit) from a pool where fewer than 2% of applicants are hired. Our Employee NPS is 76, worth asking any vendor for, because retention decides whether the team that starts your project finishes it. More on staff augmentation services and dedicated development teams.

Run reference calls that produce facts

The references a vendor supplies are pre-selected and usually briefed. That does not make them worthless, it makes them a floor: proof that three clients will say something warm on a recorded call. Clearing it is table stakes, so go past it.

Federal procurement is explicit that supplied references are not the boundary of the inquiry. FAR 15.305 directs that the government “shall consider this information, as well as information obtained from any other sources, when evaluating the offeror’s past performance.” The same rule carries a fairness principle worth importing: an offeror without a relevant record “may not be evaluated favorably or unfavorably on past performance.”

On the call itself, the goal is to extract specifics. Adjectives are noise. Questions that work:

• Tell me about a week when the engagement was behind. What did they actually do?
• Who rotated off mid-engagement? How did you find out, and how long until the replacement was productive?
• What did you end up doing yourselves that you had expected the vendor to do?
• When you escalated, who did you escalate to, and did it change anything?
• What would you write differently in the contract if you were signing it again today?

That last question is the most productive in the set: it invites a specific regret rather than a verdict.

Then find the references nobody gave you. Ask your own network who has used the vendors on your shortlist, and in technical interviews ask the engineers what they worked on before: an unfiltered view of how the vendor allocates people.

Read the SLA and the exit clause before you fall in love

By the time a contract arrives, most buyers have emotionally decided. Read these two documents while you can still walk away.

The SLA. Work out what is measured, who measures it, over what window, and what happens when it is missed. Service credits are a partial refund, not a remedy: a credit does not recover a launch date. Delivery and operations also need different instruments. An availability target suits a system in production, covered under managed IT services; for a build, the real commitments are acceptance criteria, defect thresholds and what a slipped milestone triggers. See outcome-based delivery.

The exit clause. Our pillar page tells you to ask what happens if you want to leave. Here is what to check in the document: the notice period, whether transition assistance is an obligation with a duration and a price, not a goodwill sentence, whether documentation and run books are deliverables, how and when your data is returned and deleted, and which subcontractors may touch your code. Own your repositories, cloud accounts and credentials from day one and the worst case stays survivable.

Governments write this down for the same reason you should. The Sourcing Playbook’s resolution planning policy exists because “resolution planning information will help to ensure government is prepared for any risk to the continuity of critical public services posed by the insolvency of critical suppliers.” Vendors can fail, get acquired, or decide your account is no longer strategic. None of those events are hostile; plan for continuity anyway.

Treat the vendor as part of your supply chain while you are at it. NIST’s Cybersecurity Supply Chain Risk Management Practices for Systems and Organizations (SP 800-161r1) is written for organizations in exactly this position, giving guidance on “identifying, assessing, and mitigating cybersecurity risks throughout the supply chain at all levels of their organizations.” It is a usable checklist for the security half of a vendor review.

Run a small paid pilot

Everything up to this point is evidence about a vendor. A pilot is evidence from one, and the only part of an evaluation that cannot be staged.

Pay for it. A free pilot is a sales activity: it attracts whichever strong engineers are free that week and carries no accountability. A paid pilot is a small commercial relationship with real obligations on both sides.

A useful pilot runs two to four weeks on a real item from your backlog rather than a sandbox exercise, in your repository, against real acceptance criteria, staffed by the named engineers who would continue if you proceed. What to measure while it runs:

• Time from access granted to first pull request.
• How long a question sits before it is answered, the best single predictor of how the engagement will feel at scale.
• Whether pull requests include tests, and whether the code reads like your codebase or like theirs.
• Whether anyone pushes back on a requirement that makes no sense. Silent compliance is a warning sign, not a courtesy.
• What happens when they are blocked: same-day escalation, or quietly working on something else?

A bounded first engagement can also buy a decision rather than a build. GTreasury, a SaaS treasury and risk management platform, brought us in for eight weeks to produce a transition strategy and modernization roadmap for decades-old Delphi code, including a census of hundreds of plugins to establish what to remediate first (read the case study). The build that followed rebuilt more than 400 product screens in under eight months with a dual-track team, onboarded in under two weeks (read the case study).

Score, decide, and write down why

Fill in the scorecard within a day of each call, while your memory is still specific. Then write a short decision memo: who you picked, the two criteria that decided it, what you are knowingly accepting as a weakness, and what would have to become true to change your mind.

Set expectations for the start, too. Two to four weeks from request to a productive engineer is the normal case, not the pessimistic one. Start at 2–5 engineers and scale to 20–50+ once the relationship has earned it. Re-score the same scorecard at 90 days, against reality.

Four traps worth naming

• Optimizing the rate rather than the total cost. The cheaper engineer who needs twice the review and produces half the throughput costs more. This is the low cost bid bias the Sourcing Playbook warns about.
• A shortlist that is all one shape. Four vendors of the same size, model and geography is one option evaluated four times.
• Treating certifications as outcomes. We hold SOC 2 Type II and ISO 27001, and they say something real about controls. They say nothing about whether the code will be good.
• No internal owner. Outsourcing does not create the technical counterpart who reviews and accepts work. Without one, the evaluation was academic.

Where we would lose this evaluation

Run this framework on us and here is where it goes against us. If you need engineers onshore for contractual or clearance reasons, we are the wrong answer. If the work is the core intellectual property that differentiates your company, hire rather than outsource. If you have no engineering management bandwidth, augmentation will disappoint you, because augmented engineers need direction we are not there to provide.

If none of those apply: 16+ years, 1,000+ engineers, 2,500+ projects, 98% client retention, and a replacement guarantee if someone is not the right fit. Tell us what you are building and what is in the way, and we will come back with a team shape and a timeline you can score against everyone else. Talk to us.