How AI assistants tell you to choose an SEO agency

gpt-5.4-mini took three clean runs at “How should a small business choose an SEO agency? Give specific criteria and name any agencies you would point to,” all on 2026-09-23, en-CA locale, no browsing, nothing carried over between them.

This prompt asks for two things at once: a set of evaluation criteria, and specific agency names. The two halves of the answer behaved very differently across the three runs, and a data limit in how we captured the responses affected what we can honestly report about the second half.

All three runs produced what amounts to the same checklist, organized differently but covering the same ground each time: has the agency worked with businesses like mine, do they explain a concrete first 30, 60, or 90-day plan rather than vague promises, do they report on real business outcomes like leads and revenue rather than only rankings and traffic, do they use white-hat, explainable methods, who specifically will work on the account and how many clients that person handles, and does the agency's pricing match the scope of what a small business actually needs.

All three runs also independently listed the same red flags: guaranteed number-one rankings, vague or secret methods, no case studies or references, and long contracts with no exit terms.

We need to disclose something about how this particular prompt's data was recorded. Our capture recorded each response's true length, 4,046, 4,105, and 4,019 characters, but only stored the first 4,000 characters of the actual text for all three runs. That means every one of these three responses is missing its final section, in the model's own words, not something we edited out.

This matters specifically for the “name any agencies you would point to” half of the prompt. Run 1's stored text ends mid-sentence, right as it introduces a section called “Agencies I'd point to,” with the sentence “These are examples of agencies often recognized in the SEO space. Whether they're right for you depends” and nothing after it. Run 2's stored text ends mid-word in its pricing discussion, before reaching any agencies section we can see. We are reporting this limit plainly rather than guessing at what either run said next.

Run 3's stored text is the only one that reached a named-agencies section before the 4,000-character cutoff. It split its list into two tiers: for small business and local SEO, it named WebFX, HigherVisibility, Victorious, Straight North, and Thrive Internet Marketing Agency, each with a short qualifier, for example calling Thrive a “common SMB option, though you should review team quality closely by account.” For more strategic or premium SEO, it named Seer Interactive and then Directive, with Directive's description cut off mid-word by the same 4,000-character limit.

We are not treating this as the model's complete or definitive agency list, only as the one partial list our capture happened to reach before running out of stored characters. Whether runs 1 and 2 would have named the same seven agencies, an overlapping set, or a completely different set, in the same pattern we saw with the dental-marketing and Vancouver Google Ads prompts, is not something we can answer from this dataset.

This shows that the criteria half of this question is something the model answers with a stable, template-like framework across runs, similar to how the Google Ads pricing prompt in this same panel produced consistent numbers. It does not show a reliable, complete answer to the naming half of the question, because our own capture limit cut every run short before we could confirm the full picture.

A small business reading this should treat the evaluation framework as a reasonably stable checklist to bring to any agency conversation, and should treat the seven named agencies as one partial, unverified list from one AI model on one date, not a vetted shortlist.