The Search Appliance determines that two documents are unique if they have different URLs. If you have worked with Web applications or CGI scripts, you know that a single "page" can have an indefinite number of URLs, simply by adding arbitrary characters after a '?' in the URL (the query string). Web application frameworks in PHP, ASP, and ColdFusion, to name a few, offer mechanisms to pass user session tokens in the URL when requesting a page. Since these tokens are random and frequently changing, the Search Appliance will recrawl such session-enabled pages indefinitely, assuming each URL represents a unique page. This overinflates our limited search index, and it incurs an unnecessary load on both your Web server and the Search Appliance.
Common Session Tokens Passed via URL
Web application framework: PHP Token name: PHPSESSIONID
Web application framework: ASP Token name: ASPSESSIONID
Web application framework: ColdFusion Token name: CFID
Web application framework: ColdFusion Token name: CFTOKEN