NOT VERIFIED We were unable to access the verification file due to a restriction in robots.txt
There is no restriction in robots.txt !! AND the file has been in place and working since the start of sitemaps, now its coming up with this rubbish !!
Does your site redirect 404 errors to a custom 404 page? If so, you might check to see if you have a robots.txt file that excludes access to that custom page.
If your server's response to a request for a non-existent page redirects to a page that is restricted by a robots.txt file, you might see this error message.
If you find that this is the case, you could turn off the redirect just long enough to verify and then turn the redirect on again.
Ok, my cgi scripts were in error/ which was in robots.txt my custom pages were also inside error/ before google had all my custom pages listed, thats why I added error/ to robots.txt
Now error/ (cgi scripts only) location in robots.txt errorpages/ my custom error pages, location not in robots.txt
Now when I click verifi I get this
"We've detected that your 404 (file not found) error page returns a status of 200 (OK) in the header. "
Should I redo the whole of my site just to please google ??????
It's not just something Google made up, it's the standard. The other search engines have problems with these kinds of things as well, Google is just the one to put it on the "front page" :-). Also -- without this check it is not possible to verify that you are the site owner (with the current system, since any validation-file that Google would want will return 200 "ok" - even if it does not exist - if anyone else decides to add your site to their statistics).