New Message: Correct Syntax For Robots.txt?

webmaster at webmaster at
Thu Jun 22 19:54:58 CDT 2006

A new message was posted:


By: ken tompkins (ken at

I'd like to confirm the correct syntax for robots.txt. I've read the doc and am not 100% sure. What I have entered as the part of the site I want ignored seems not to be working -- hence this question.

Say I want to have the search engines ignore a complete site. Do I put the whole address into the robots.txt form as

or do I simply put


because ALL of the sites on the server have the address?

I have, as I said, tried both of these but to no obvious effect.


ken tompkins

This is a Manila site...

More information about the Manila-Server mailing list