SEO-Lexicon
Deep Crawl
Deep crawl can be paraphrased as a deep indexing. Some search engines use this special technique. Hereby, every subpage will be scanned and indexed. Search engines without Deep crawl, index subpages up to a certain folder-deepness. There are different criteria, which tell the search engine whether to scan the subpage or not. Next to the page rank of the page you might find the link-popularity. Search engines with Deep crawl automatically include every subpage with its indices of a domain, which is already indexed. Here, the information is read out even from the lowermost layer. This procedure is very time-consuming. For the avoidance of any overloads of the server, search engines work with latencies. in the server-logfiles you can find information about this: if there was a deep crawling on your own page and how deep the indexing of the subpages was.
