Shopify Developer Evaluation Template

Hiring a Shopify developer, freelancer or agency, without a structured evaluation process is how founders end up with expensive rebuilds six months later. This template is a practical framework: a document you can fill out while evaluating candidates, with specific questions to ask, red flags to watch for, and scoring criteria that surface real differentiators.

Use this to evaluate 3-5 candidates side by side. It works for solo freelancers, small agencies, or boutique shops.

Section 1: Basic information

Section 2: Fit-for-project questions

Each of these should be asked directly. Score 1-5 based on how substantively the candidate answers.

2.1 Have you shipped a project like mine in the last 18 months?

“Specifically, a [type of project, migration, rebuild, integration, subscription, B2B, international] for a brand in [vertical].”

Red flag: generic answer about “many similar projects” without specifics. Green flag: names a recent client, describes the specific technical scope, mentions what was hard about it.

Score: /5

2.2 Who specifically would work on my project?

“Not ‘our senior team’, I mean the named engineers, designers, and project manager. Can you share their resumes or LinkedIn profiles?”

Red flag: dodges with “we’ll assign the right team.” Green flag: shares specific names, bios, and years-on-platform data.

Score: /5

2.3 What’s your process for scoping?

“Do you do a paid discovery phase? What’s included? What does the deliverable look like?”

Red flag: no discovery phase, or “we include it for free.” Green flag: structured 2-4 week paid discovery with defined deliverables.

Score: /5

2.4 What happens when scope changes mid-project?

“How do you handle change requests? Do you use change orders? How are they priced?”

Red flag: “We just roll with it” (means scope creep becomes arguments later). Green flag: formal change-order process with documented pricing.

Score: /5

2.5 Walk me through a project that didn’t go as planned.

“I want to hear about one. What went wrong? What did you learn? How did you handle the client relationship?”

Red flag: “We haven’t had that happen” (they’re either lying or not reflecting). Green flag: honest retrospective with specific lessons.

Score: /5

2.6 What does post-launch support look like?

“I don’t just want to know what you’ll build, I want to know what happens after. What’s your retainer structure? Response times? SLAs?”

Red flag: “We can figure that out later.” Green flag: documented retainer tiers, response-time commitments, named support contacts.

Score: /5

2.7 Can I see a sample discovery doc or SOW from a past project?

“Redacted is fine. I want to see how you document scope.”

Red flag: can’t or won’t share. Green flag: shares a detailed doc that reads like a real engineering plan.

Score: /5

2.8 What’s your approach to [specific technical challenge unique to my project]?

Pick something real: “subscription pause/skip flows,” “checkout extensibility for trust badges,” “multi-region tax handling via Markets,” “ERP integration with NetSuite,” etc.

Red flag: generic answer that doesn’t engage with the specifics. Green flag: specific architecture recommendations, tradeoff discussion, and known-issue flags.

Score: /5

Section 3: Reference checks

Don’t skip these. Plan for 2-3 reference calls per shortlisted candidate.

3.1 References provided by candidate

These are their happiest clients. Useful but not complete. Ask:

• What did they actually build for you?
• What surprised you during the project?
• What would you change about how it went?
• Would you hire them again for a different project?
• Who wasn’t on the proposed team but ended up doing work?

3.2 References you find yourself

Search LinkedIn for developers who left the agency in the last 18 months. Ask for a 15-minute chat. Most say yes. They’ll tell you things the sales team won’t.

Questions:

• What was the working environment like?
• How was code quality maintained?
• How often did senior engineers actually touch client projects?
• What kinds of projects went smoothly? Which didn’t?

3.3 Clients not on their website

Most agencies have clients who aren’t featured. Sometimes that’s because the project went badly. Search for “[agency name]” on Twitter/X and LinkedIn, you’ll often find mentions from clients who aren’t in the case studies.

Section 4: Sample work review

Ask for access to live examples of their work, not screenshots, actual URLs.

4.1 Store speed test

Run a PageSpeed Insights test on 3 stores they’ve built. Look at Core Web Vitals. Consistently fast stores signal engineering discipline. Slow stores signal cut corners.

4.2 Mobile UX review

Check their built stores on mobile. DTC traffic is 60-80% mobile. A desktop-first agency will produce desktop-first work.

4.3 Design quality

Compare three stores they’ve built. Is there visible design quality variation, or do they all look distinct and brand-appropriate? Cookie-cutter agencies produce stores that all look similar despite different clients.

4.4 Conversion structure

Without access to their analytics, you can still evaluate structure: clear CTAs, logical flow, trust elements, strong product pages. Agencies that build for conversion show it in their portfolios.

Section 5: Scoring and decision framework

Total score

Sum the fit-for-project scores (out of 40). Plus adjustments for:

• Reference quality (+/- 5)
• Sample work quality (+/- 5)
• Proposed team composition (+/- 5)

Total: /55

Cost-adjusted value

A candidate scoring 40/55 at $80k is usually better value than one scoring 50/55 at $250k. Calculate score ÷ proposed fee (in $10k units) to compare.

Working style fit

This is harder to score but matters. After the sales process, ask yourself:

• Did they listen, or pitch?
• Did they ask good questions, or just answer yours?
• Did their proposal reflect your specifics, or feel templated?
• Did you enjoy the interactions?

You’re going to work with this team for months (or years). Fit matters as much as capability.

Common evaluation mistakes

• Overweighting the sales pitch. The polished deck and charismatic pitch often represent the agency’s best day. The day-to-day engineering work is different.
• Underweighting the team composition question. Agencies that won’t tell you who’ll work on your project are telling you who’ll work on your project: whoever’s available.
• Treating price as the primary variable. For the same scope, prices across credible agencies typically vary 2-3x. That variance usually reflects real differences in team seniority, process quality, and post-launch support.
• Skipping the discovery phase. It costs $10-30k up front and saves $50-200k in avoided scope-creep and rebuilds.
• Only interviewing agencies. Sometimes a senior freelancer is the right call. Include one in your shortlist.

Finding candidates to evaluate

Shopify Partner directory

Filter by Shopify Plus Partner tier and project type. Quality is variable but it’s a vetted starting point.

Peer recommendations

Ask three brands you respect who they hire. This is the highest-signal sourcing channel.

Curated agency lists

Published agency roundups from established brands (like curated Shopify partner rosters from Netalico) or editorial listicles. These surface agencies with documented track records. Searches for “top Shopify developers” or “best Shopify Plus agencies” produce these lists, read 2-3 to triangulate.

LinkedIn search

Search “Shopify Partner” plus your vertical plus your region. Surfaces agencies and senior freelancers actively posting in your space.

Slack communities

DTC-focused Slack groups often have Shopify-partner recommendation channels. Peer-surfaced, high signal.

Final note

The right developer for your project is the one whose specialty matches your work, whose team you can actually verify, and whose references hold up under scrutiny. No template replaces judgment, but a template prevents the judgment from being purely gut-feel. Use this as a scaffold, not a rulebook, and trust your read on the people as much as the scorecard.