Enterprise SEO agencies, tested on the Fortune 100

Before I wrote a word about enterprise SEO agencies, I wanted to see the Fortune 100 the way ChatGPT sees it. So on 24 October 2026 I sent the same requests to the 100 largest US companies that an AI crawler sends: robots.txt first, then the homepage, once as a normal browser and once under the name of each AI bot.

Almost nobody closes the door in robots.txt. Two companies out of 79 that answered. But on 18 of 73 homepages the server answered the AI bots differently from the browser, and that decision was made by Akamai, CloudFront or Sucuri, not by anyone on the SEO team.

That is the whole enterprise SEO problem in one test. The site is too big for one person to own, so the thing that decides your visibility ends up in somebody else's ticket queue. An enterprise SEO agency is worth its retainer when it can find that ticket.

Key takeaways

• robots.txt is not where AI access is decided any more. 2 of 79 Fortune 100 sites block AI crawlers there. 18 of 73 homepages treat AI user agents differently at the edge, and on 3 of them our test Googlebot got through while the AI bots did not.
• 9 Fortune 100 homepages are close to empty without JavaScript. Comcast, Broadcom and Deere serve 4 words, Delta 8, UPS 16. GPTBot, ClaudeBot and PerplexityBot do not run JavaScript, so that is all they read.
• Published retainers start around $5,000 a month and go past $25,000. The same agency can be quoted at $15,000 in one list and a $50,000 minimum in another, so ask for the number in writing.
• 11 of the 13 «best enterprise SEO agencies» lists in Google's top results were written by agencies, and 9 of them put themselves first. Google's own AI Overview names a different set of agencies than the lists do.
• Ask any agency one question first: who at our company owns the CDN and firewall rules, and how will you work with them?

What makes SEO «enterprise»

It is not the size of the budget. I have seen $3,000-a-month projects with enterprise problems and $20,000-a-month projects without them.

It is the number of people who can break your search visibility without knowing they did. On a 50-page site, the marketer who writes the title also publishes the page. On a site with 80,000 URLs across 12 languages, the title comes from a template owned by product, the page renders through a framework owned by engineering, the CDN rules belong to security, and legal can veto the FAQ. Every one of them changes search results. None of them reports to the SEO lead.

In practice I see four traits, and two of them are enough to call a project enterprise:

That last row is why enterprise agencies charge what they charge. The audit is the easy part. Getting the fix through someone else's sprint is the job.

What AI crawlers see on Fortune 100 sites

Here is the test in more detail, because it is the most useful thing on this page and you can repeat it on your own site in an afternoon.

I took the 100 largest US companies by revenue from the Fortune 500 (the 2025 edition: the 2026 list is out, but only its top 10 are public). For each company's main site I:

• read robots.txt and checked 11 AI user agents against it: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot, Bytespider and meta-externalagent;
• requested the homepage with a normal Chrome user agent, then with 7 AI user agents and Googlebot, all from the same address, 1.5 seconds apart;
• counted how much text the homepage returns before any JavaScript runs, and checked the basics in that raw HTML: one H1, structured data, hreflang, canonical;
• checked for /llms.txt.

For comparison, I ran the same script on 33 SEO agencies from the lists and the top of this search, and on our own site.

One caveat before the numbers, because I would want it if I were reading this. My requests carried the bots' names but did not come from OpenAI's or Anthropic's IP ranges. A good firewall checks that a bot is real and stops a fake one, and that is correct security, not an AI policy. So I count separately the cases where Googlebot from the same address got through and the AI bots did not. That pattern is a rule written against AI bots by name. The other cases need a look at the server logs, and that is exactly what an agency should ask for in week one.

The firewall decides, and the SEO team does not own it

The response headers tell you who made the call. IBM, Nike and American Express sit behind Akamai. Lockheed Martin answers through Amazon CloudFront. Berkshire Hathaway, the company with the plainest website in the Fortune 100, runs Sucuri. On Nike our GPTBot request got Akamai's «Access Denied» page, and our Googlebot got a 403 as well. On Lockheed Martin, Googlebot came through and GPTBot and ChatGPT-User were refused.

None of this shows up in robots.txt. And robots.txt is the file every SEO audit checks.

Ahrefs measured AI-bot blocking across about 140 million sites in May 2025 and found GPTBot disallowed on 5.89% of them. It is a good study, and it says plainly that firewalls and IP blocks were out of scope. That is the layer where the large companies make the decision. Since 1 July 2025 Cloudflare has blocked AI crawlers by default on new domains, as MIT Technology Review reported at the time, so a site can change its AI policy without anyone in marketing touching a file.

Is blocking AI crawlers wrong? Not necessarily. Some companies have good reasons. The problem is when nobody decided. On LinkedIn a few months ago I wrote that «"The AI misquoted us" is quickly becoming a real support ticket, not a hypothetical.» A company that does not know which AI bots it lets in will not know why it is misquoted either.

Empty homepages: what a bot reads without JavaScript

Vercel looked at its own network traffic in December 2024 and stated that none of the major AI crawlers render JavaScript; Google's Gemini is the exception because it rides on Googlebot. Google renders your React app. ChatGPT reads what your server sends before any script runs.

For 9 Fortune 100 companies that is almost nothing:

A homepage is not where most AI citations come from, so do not panic over this table. But the homepage is usually built on the same framework as everything else. If it ships empty, I would check the product and support pages next, because those are what AI answers actually quote.

The H1 numbers point the same way. 24 of 74 Fortune homepages have no H1 in the raw HTML. Some of them add it later in the browser, which is fine for Google and invisible to the crawlers that do not render.

The agencies do on their own sites what they sell

This was the part I did not expect. The 33 agencies are tidier than the Fortune 100 on every basic check: 25 of 28 with exactly one H1, 23 with Organization markup, 15 of 33 with llms.txt against 11 of 100. Not one blocks AI crawlers in robots.txt.

That is not a reason to hire them. A clean marketing site with 300 pages proves very little about running 80,000. But it does tell you something about the gap an enterprise agency is hired to close: the basics that an agency gets right on a small site are hard to keep right on a big one, because ten people touch the templates.

We ran the same script on seoquick.com.ua. All ten user agents got a 200, robots.txt is open, llms.txt is there, one H1, Organization markup, hreflang for three languages and 1,695 words before JavaScript. Fixing our own site was cheap. That is the point.

We also ran it on 50 of the largest Ukrainian brands from a Ukrainian address. The pattern repeated: one robots.txt block (a single bot), five sites where the edge stops AI crawlers by name while Googlebot passes. On three of them, a supermarket chain, an agribusiness and a fuel retailer, GPTBot and ClaudeBot get a 403 while the AI bots that fetch pages for a live user are let in. That is what a one-click «block AI training bots» switch looks like from outside.

Want this test on your site?

Send me your domain. I will run the same checks on your key templates and tell you which AI crawlers get in, which get stopped, and who in your company made that call. You will get the raw results, not a pitch.

Free, and it commits you to nothing.

Book a call

What an enterprise SEO agency should do in the first 90 days

When I review a proposal from any agency, ours included, I look for work in this order. If the plan starts with a content calendar, the agency is pricing a small-site project at an enterprise rate.

Weeks 1–3: read the logs, not the crawl

A crawler like Screaming Frog shows what a crawler can reach. Server logs show what Googlebot, GPTBot and ClaudeBot actually requested, how often, and what they got back. On a big site those are very different lists. Vercel measured 404s at about 35% of ChatGPT's and Claude's fetches on its network, against 8% for Googlebot. AI bots waste their visits on dead URLs, and only the logs show you which ones.

Ask the agency who will get access to the logs, in what format, and how long the export takes at your company. If they have never asked a security team for log access, you will find out in week two.

Weeks 2–4: map who owns the edge

Firewall, CDN, bot management, rate limits, geo rules. Write down every layer between the request and the page, who owns it, and what it does with Googlebot, Bingbot and the AI user agents. Then agree a policy on purpose: which AI crawlers you let in, which you do not, and why.

Month 2: fix what the server sends

Server-side render the content, title, H1, structured data and internal links on the templates that matter. This is engineering work, so the agency needs someone who can write the ticket in your developers' language and test it on staging, not just send a PDF.

Month 2–3: one market at a time

hreflang, regional folders, which version should rank where. 21 of 74 Fortune homepages declared hreflang, so most of the largest companies run either one market or several sites. If you have several markets, the agency should show you the pages ranking in the wrong country before it proposes anything new.

Month 3: templates, not pages

On an enterprise site you do not optimize 40,000 product pages. You optimize one template and the data that fills it. I wrote about how Wise built its programmatic pages and how Rtings.com structures its product reviews: in both cases the work that mattered was the data model, not the copy.

All the time: authority, because content is no longer scarce

This is my personal bias, so take it as one. I wrote last month: «If crafting great content is accessible for everyone then links and mentions become the most important asset today.» For a large brand that means digital PR at a steady pace, not a link package. On one of my own projects we published 12 press releases a week, and I calculated that the coverage replaced about $3.7 million a year of what we would otherwise have paid for links and media. A good release earned more than 55 links and mentions; a weak one still earned 5–10.

Run the test on your own site

You do not need an agency for the first look. Three commands from any terminal will tell you most of what my script found. Replace the domain with yours.

First, does robots.txt mention AI crawlers at all:

curl -s https://www.example.com/robots.txt | grep -i -E "gptbot|oai-searchbot|chatgpt-user|claudebot|perplexitybot|google-extended|ccbot"

Second, does the edge answer an AI user agent the way it answers a browser. Compare the status codes:

for ua in "Mozilla/5.0 Chrome/130" \
  "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
  "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)" \
  "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"; do
  curl -s -o /dev/null -w "%{http_code}  $ua\n" -A "$ua" https://www.example.com/
done

Third, how much text is there before JavaScript runs:

curl -s -A "Mozilla/5.0 Chrome/130" https://www.example.com/ \
  | sed -e 's/<script[^>]*>.*<\/script>//g' -e 's/<[^>]*>/ /g' | wc -w

The last one is rough, it counts some navigation and footer words, but a result under 100 on a page that looks full in the browser tells you what an AI crawler gets. Run it on your homepage, a category or product template and a support article. If the three numbers differ a lot, you have found your first ticket.

What these commands cannot tell you is what the real bots get from their real IP addresses. For that you need the server or CDN logs, filtered by user agent and verified by IP. That is the first thing to ask your engineering team for, and the first thing an enterprise agency should ask you for.

The tools I use for this, with a disclosure

SEOquick is a partner of both SE Ranking and Semrush, and the links in this section are referral links: if you subscribe through them, we earn a commission at no cost to you. We pay for both and use both every day, so here is what each one does in this kind of work.

For an enterprise program you will probably want both: one as the daily workhorse, the other as a second opinion on the numbers. We compared them on 26 real sites against Search Console in SE Ranking vs Semrush; neither replaces your own logs and Search Console, and both are far better than guessing.

Red flags in an enterprise SEO proposal

I read a lot of proposals, sometimes from agencies pitching the same client as us. These are the ones that tell me the agency has not worked at scale:

• A keyword count as the main deliverable. «We will optimize 500 keywords» means nothing on a site where one template serves 40,000 of them.
• No mention of logs. An enterprise plan that relies on a crawler alone is guessing about what bots actually do.
• Content volume before technical access. Thirty new articles a month on a site where the product pages ship empty to AI crawlers is money spent on the wrong floor of the building.
• AI visibility as a separate upsell. If GEO is a line item added at the end, the agency has not connected it to rendering, edge rules and structured data, which is where it actually happens.
• Reports in positions, not in pages and revenue. Positions for 500 keywords move every day. What the board wants to know is which templates gained or lost traffic and what that was worth.
• Nobody named who will talk to engineering. If every fix goes out as a PDF to your marketing team, it will sit there.

What enterprise SEO agencies charge

Few enterprise agencies publish prices. These are the figures I could find on the agencies' own pages or quoted in the lists that rank for this search, with the source for each. Where two sources disagree, both are shown.

Google's AI Overview for «enterprise seo agency» puts the typical range at $7,000 to $25,000+ a month. That matches the table. Six of the 13 lists in the top results give no price for anyone.

Why does iPullRank appear at $15,000 in one place and $50,000 in another? Probably because one figure is a program and the other is a minimum engagement, and list writers copy whichever number they found first. This is why I tell people to get the figure in writing from the agency itself.

And the in-house comparison, because enterprise buyers always ask. In my newsletter this week I put the 2026 numbers side by side: «about $16,000 a year for an average freelancer retainer, $38,500 for an agency, and at least $118,000 for a full-time US hire with benefits.» Those are averages across all business sizes. At enterprise scale the hire is a team, not one person, and the agency is usually brought in next to that team, not instead of it.

Who writes the «best enterprise SEO agencies» lists

I read the 13 lists that rank on the first pages of Google for this search. 11 of them were published by SEO agencies. All 11 included themselves. 9 put themselves at number one; Animalz got there by sorting alphabetically, MADX is second behind Terakeet, and Fuel Online's page names only Fuel Online. Two of those lists come from Ukrainian agencies, and both are number one on their own lists.

Only 4 of the 11 say on the page that the publisher is in the list it wrote. Two lists were not written by agencies: Traffic Think Tank, a Semrush-owned education site, and the freelance writer Nathan Ojaokomo. Neither one ranks itself.

Across the lists, these agencies come up most often:

Now look at the AI Overview Google shows above all of these lists. It has its own table of «top enterprise SEO agencies»: WebFX, ResultFirst, SEO Brand, Loopex Digital and Workshop Digital. Only WebFX is in the top of the list consensus. The sources under the overview are mostly agency service pages, the pages that say plainly what the agency does, for whom and at what scale.

To check whether that was a fluke, I pulled the AI search data for those agencies from SE Ranking's AI Search module: how often the agency's site is cited as a source in AI answers (AI Overviews, AI Mode, ChatGPT, Perplexity and Gemini, US) and how often the brand is mentioned. Snapshot of October 2026.

WebFX is cited more than all the others combined, and it is the agency the AI Overview names first. Graphite, named in 7 of the 13 lists, has 17 citations. Three agencies that no list mentions, ResultFirst, Workshop Digital and SEO Brand, made the AI table on the strength of their service pages. Praise from peers and citations from AI are two different currencies.

I find that more interesting than any ranking. The AI did not pick the agencies their peers praise most. It picked the ones that described themselves most clearly. It is the same thing I wrote about AI Overviews in general: «Whoever writes the clearest, least ambiguous version of the truth on their site has the best shot at being the version Google's AI actually repeats correctly.»

A shortlist, with facts only

I am not ranking these. Every agency below is named in at least five of the 13 lists, and the price column only shows what was published. SEOquick is in the table because we do this work; I put us where a reader can compare us, not on top.

What we have done at enterprise scale

I have been in SEO since 2008, and SEOquick has been running since 2012. Our biggest engagements look nothing like a monthly report with 20 keywords. They look like this:

We do not have Fortune 100 logos to show you. What we have is the habit this page is about: we measure before we write, we fix our own site first, and when the work needs a developer, a video producer or a PR writer, they are on the same team as the SEO lead, not in another agency's inbox.

How to choose: questions for the first call

The lists agree on the criteria: experience at scale (10 of 13 lists), case studies (9), AI search readiness (7), reporting tied to revenue (7). All true, and all easy to claim. These questions are harder to fake:

• «Who owns our firewall and CDN rules, and how will you work with them?» If the answer is «we will check robots.txt», keep looking.
• «What does GPTBot see on our top three templates?» A serious agency will want to test it, not guess.
• «How do you get a fix into our sprint?» Ask for an example ticket they wrote for another client's developers.
• «Which of your published numbers can I see in a contract?» If you found a price in a list, ask the agency to confirm it.
• «Who will actually work on our account, and what else are they on?» Names, not roles.
• «What would you not do in the first 90 days?» The answer tells you whether they have a plan or a menu.

If you want a second opinion on a proposal you already have, I will read it against this list. That is the most useful thing I can do for you before you sign anything.

Questions people ask

How much does an enterprise SEO agency cost?

Published retainers run from about $5,000 a month to $25,000 and more; Google's AI Overview gives $7,000–25,000+. Minimum projects of $25,000 and $50,000 exist. Get the figure in writing from the agency, because the lists contradict each other.

Should we block AI crawlers?

That is a business decision, not an SEO default. Blocking training crawlers like GPTBot while allowing the agents that fetch a page for a live user (ChatGPT-User, Claude-User, Perplexity-User) is a reasonable middle ground, and it is what several large Ukrainian brands do today. What is not reasonable is not knowing. Check your edge layer, not only robots.txt.

Agency or in-house team?

Most enterprises need both. The in-house lead owns priorities and internal politics; the agency brings people the company would not hire full time: log analysis, JavaScript SEO, international, digital PR. A US in-house hire costs at least $118,000 a year with benefits, so the agency is usually the cheaper way to add those skills.

Related reading

• How to choose an SEO agency
• How much SEO services cost
• The best B2B SEO agencies
• The best SaaS SEO agencies
• What to put in an SEO agency contract
• Digital PR case study
• GEO and AI visibility services
• SEO for English-speaking markets

The measurement scripts and raw results are ours, from 24 October 2026; prices come from the agencies' pages and the lists named above as of the same date. If you run the test on your own site and get something odd, write to me. I collect these.