Hi all,

We have a public site where all pages are available to the guest user profile.  When accessing this site through the browser, we've never seen an authorization required page.  However, when google bots attempt to crawl the site we get intermittent 401 Authorization Required responses.  

I've tried hitting the site from seoconsultants check header tool, and I get intermittent 401 and 302 responses.  The 302 that I've just received is saying that the page has been moved to : cust_maint/site_down/maintenance.html.   Web analyzer gives the same results, yet using fiddler shows that the browsers are receiving nothing but 200 responses.

Has anyone else seen behaviour like this?  Its proving difficult to track down as every time I raise a case with support, they close it saying they don't support the google client.  However, that's not the issue here - I need to understand why the Salesforce server is returning the responses that it is.