We want the crawler of the search engine to download as many resources as possible from a particular Web site. A crawler would normaly ascend to every path in each URL that it intends to crawl. For example, when given a seed URL of ****.org/a/b/page.
