You need to sign in to do that
Don't have an account?
charlie_v
Robots.txt file won't refresh
Hi there.
This morning I finally got around to attaching the Site Robots.txt file to my site. A quick check of Google Webmaster Tools verified it was there, and they were seeing it.
Unfortunately, I inadvertently set all user agents to Disallow instead of Allow. So I made the correction, but now that change doesn't seem to be propagating to the web. I've waited 45 minutes, and it's still not out on the web. I've verified the change is correct using the "preview" button in my Sites configuration.
How long before the updated robots.txt file gets out on the web?
updates on robot.txt might take up to a day since the txt file would be cached for a day like static resources (the difference is that this file can't have a time stamp so your updates can't be reflected immediately)