Large-scale real-time fetch service
US20130117252A1 Патенты
System and method for fetching embedded object content as part of a batch crawl. A fetch server receives a request on a request thread to retrieve content for objects embedded in a document, such as a web page. The fetch server attempts to locate the content of the object in cache first and in disk storage next. If the content is not located in the cache the fetch server may switch the request to a worker thread. If the content is not located in the disk storage, the fetch server may schedule a request to retrieve the content of the embedded object through a batch web crawl. Scheduling a reque
Притязаний: 20. Знаков текста: 48 807.