Which problems most often keep websites out of search and AI answers?

The most common problem we measured on websites is a weak layer of structured facts about who the business is and what it sells. We scanned 216 Shopify storefronts on June 28, 2026, and 72 percent failed Entity Integrity. They earned less than half of its points.

Our second study scored 303 non ecommerce websites in July 2026 and found the same weak spot. Their structured data pillar averaged 35.3 percent of its available points, the lowest of the five pillars. A smaller group of sites failed before any of that could be measured, because the crawler could not read them at all.

Visibility Mesh published this report on September 27, 2026 and updated it on November 9, 2026. Every figure keeps its study's population and date. None of it shows that a problem caused a loss of traffic or of AI mentions.

Which studies is this report built on?

This table lists the two frozen studies behind the report.

Both studies used version 1.0 of the AI Visibility Score, which our AI Legibility Framework v1.0 describes. The storefront study reports five metrics and the non ecommerce study reports five pillars, so we quote each one in its own terms. The Forbes column built on the June study explains the storefront findings for business owners. The non ecommerce study read homepages only. Neither study read Search Console, analytics or AI answers, so the figures describe what a crawler could read on the day of the scan.

Which problems most often stop a website from being read?

Each of the five metrics was failed by a large share of the storefronts. This table shows the share of the 216 Shopify storefronts that failed each one on June 28, 2026. The published percentages are truncated, not rounded.

None of the 216 storefronts earned 80 percent of the points on any of the five metrics.

The non ecommerce study points at the same layer. Across the 303 websites the engine wrote 11,633 findings. Thin structured data and a missing block of questions and answers made up 54.4 percent of them.

Google stopped showing FAQ rich results in Search on May 7, 2026. We still count a missing FAQ as a finding. A short block of questions and answers is text a machine can quote, whether or not Google draws a rich result for it.

How common are entity and structured data problems?

Entity problems were the most common failure in the storefront study, at 72 percent. Entity Integrity checks whether the brand, its products and their attributes are stated consistently in the data AI systems read.

The non ecommerce web shows the same gap in a different form. Of 299 homepages the scanner could reach in July 2026, 52.2 percent carried any JSON-LD. Organization markup appeared on 42.5 percent, and FAQPage markup on 1.3 percent.

We had both a score and a homepage check for 275 non ecommerce sites. Sites with any JSON-LD averaged 53.7, and sites without it averaged 41.0. Part of that 12.7 point difference is built into the score, since the score rewards structured data. Nobody should read it as proof that markup lifts a website.

How many websites could not be read at all?

The engine could not read 13 of the 229 storefronts it scanned on June 28, 2026. Bot walls, empty JavaScript renders and error pages stopped it. We left those 13 out of the scoring instead of counting them as zeros. In the non ecommerce study, 47 of 350 websites produced no score. Most of them had heavy bot protection or a homepage built only in JavaScript. Four of the 47 never finished scanning, so we cannot say why those four failed.

A bot wall that stops our crawler does not prove that it stops Googlebot. Cloudflare, for example, keeps a directory of verified bots that site owners can choose to let through. We have not classified the reasons across all our scans yet, so we publish no single share for the wider web.

How do I check my own site for these problems?

Work through the steps in this order, starting from the door, with a browser open on your own site.

1. Open yourdomain.com/robots.txt and look for a Disallow rule under GPTBot, OAI-SearchBot, ClaudeBot or PerplexityBot. Our robots.txt guide shows how to read the file line by line.
2. Open a product or service page, right click and choose View page source. Search for a sentence you can see on the page. If the browser cannot find it, the text arrives only after JavaScript runs.
3. In the same source, search for "application/ld+json" and read the blocks. Count the Organization blocks on the homepage and the Product blocks on a product page, and remove any second copy.
4. Compare the price, availability and name in the markup with what the page shows. The structured data validation guide shows how to test the markup with free tools.
5. Read the first paragraph of your most important page and check that it says plainly what you sell and who it is for.
6. On Shopify, theme markup is edited in Online Store > Themes > Edit code. On WordPress, check the SEO plugin's schema settings before you touch the theme, since the plugin often prints the second block.

What does this look like on a real kind of site?

Consider an accounting firm with a WordPress site and a security plugin set to its strictest mode. The homepage loads instantly for a person. The plugin challenges any visitor it does not recognize, so a crawler receives a challenge page instead of the homepage.

A site like that can end up with no score, like the 47 in the non ecommerce study. The owner only has to change a setting. In the plugin's bot settings, the owner lets verified search and AI crawlers through and keeps the strict mode for everyone else. Then the owner checks the homepage source for one Organization block with the firm's real name, address and phone number.

Are online stores more or less readable than other websites?

Our two frozen studies do not settle it. The median storefront in the June study scored 46 out of 100, and the median non ecommerce website in the July study scored 49.1. The samples differ in platform, date and depth, and the July study read homepages only. We do not treat that small gap as a finding about stores. Our dated reading of 245 Shopify storefronts on August 22, 2026 is another fixed point.

How should I cite this report?

Cite the original study for each figure, with its population and date. For the storefront figures, cite Visibility Mesh, The State of AI Visibility 2026, which scored 216 Shopify storefronts on June 28, 2026. Both studies are published under CC BY 4.0.

To see where your own site stands, the free scan reads five of its pages with the same engine.

Questions people ask

How many websites cannot be read by crawlers because of bot protection or empty JavaScript pages?

The engine could not read 13 of 229 Shopify storefronts scanned on June 28, 2026. Of 350 non ecommerce websites targeted in July 2026, 47 produced no score. Bot walls, empty JavaScript renders and error pages caused most of them. We publish no single share for the wider web yet.

How common are entity integrity problems on websites?

Entity problems were the most common failure in our storefront study. Of 216 Shopify storefronts scanned on June 28, 2026, 72 percent failed Entity Integrity and earned less than half of its points. On 299 non ecommerce homepages reached in July 2026, 42.5 percent carried Organization markup.

Are online stores more or less readable to AI than other websites?

Our frozen studies do not settle it. The median Shopify storefront scored 46 out of 100 on June 28, 2026, and the median non ecommerce website scored 49.1 in July 2026. The samples differ in platform, date and depth, so we do not treat the gap as a finding about stores.

Does a low score prove my site gets fewer visits or AI mentions?

A low score does not prove that. The score measures what AI systems can read on a site on the day of the scan. It does not forecast citations or traffic.

Sources

• The State of AI Visibility 2026, Visibility Mesh, 216 Shopify storefronts scored on June 28, 2026.
• The State of AI Visibility on the Non Ecommerce Web, Visibility Mesh, published July 13, 2026, on the Research and Press page.
• Google, Search Central documentation updates, entry of May 8, 2026 on the FAQ rich result ending May 7, 2026, read November 9, 2026.
• Cloudflare, Verified bots, read November 9, 2026.