Google AI Overviews appears to be slower to remove citations/content from its index that it serves for answers, compared to web search. This includes when a page or section is not indexed or the robots.txt file is out and when Google deindexes pages or sections of the site, due to manual action or another reason.
Lily Ray noticed how pages blocked by robots.txt still seem eligible to appear in AI previews, and she even shows the snippet/description. In this case, it does not appear in web search, but in AI previews.
Here are these screenshots of Lily on X:
Here is the preview of the AI that appears:
Here the snippet is not shown in web search because it is blocked:
Here is the robots.txt file:
My first instinct was that it was a timing issue and that the AI previews were delayed.
Then I saw Glenn Gabe post on X that Google was still showing sites with manual actions over time new update to the site reputation abuse policy in AI previews but not in Google search results.
Note that later today the link no longer appeared for this AI preview. There is a delay but it’s hard to say how long the delay is… Here’s a screenshot of what I saw later that day:
Looks like there’s a bit of a disconnect with citations in AIO!
-Lily Ray 😏 (@lilyraynyc) November 20, 2024
Jason Kilgore also spotted it and wrote about LinkedIn“Ready to learn more? Advisor is STILL listed in AIO for “long-term care insurance”. Apparently manual actions are not yet reflected in AIO.”
So maybe, just maybe, Google is slower with AI previews than with web search. I know Google is also slower with image search but you were hoping the AI previews were fast…
Lily’s example was also deleted about 24 hours later:
24 hours or less 😅 @rustybrick https://t.co/jlo94YbEmL pic.twitter.com/Gca3TNT2zq
-Lily Ray 😏 (@lilyraynyc) November 21, 2024