focused crawler
A focused crawlerSoumen Chakrabarti, Focused Web Crawling, in the Encyclopedia of Database Systems. is a web crawler that collects Web pages that satisfy some specific property, by carefully prioritizing the crawl frontier and managing the hyperlink exploration process.