Google Search Appliance: Exclude Part of Your Site from Search Results
The Search Appliance obeys the exclusion parameters you place in the document "/robots.txt" on your Web server, at the document root level. This file, if present, must be served with an HTTP 200 (OK) status code in order for the Search Appliance to use it. An HTTP 404 (Not Found) status code simply indicates that your site is to be crawled as usual because robots.txt does not exist. If a request for "/robots.txt" returns an error code other than 404, then your site will not be crawled, and its documents will not appear in search results.
To block the Google Search Appliance, include the following lines: