SEO-Lexicon
Freshbot
Freshbot is a spider-software, which is used by Google. It is responsible only for the finding and the indexing of new websites. Freshbot searches for new websites and forums on the internet nonstop. Hereby, it only accesses HTML- and text-documents. Other file formats, like PDF, Word, etc, are not affected. New sites are not captured that voluminously. It only checks pages, which are directly linked with the page that was found. The indexing of the new website depends on the actualization of the page. The page is being checked regularly on actualizations. If the actualization is missing, so the site gets a worse ranking in the index. Pages, which are hardly ever actualized, fail in the index. After a certain time, however, they will again appear in Google Dance in the main index. In the logfiles of the web server you can identify freshbots based on the IP-numbers in the area between 64.x.x.x and 66.x.x.x.
