MACROMEDIA COLFUSION MX 7 - INSTALLING AND USING COLDFUSION..., Руководство пользователя

Страница: 121 / 170

Networking options 121
-delay
Type : Web crawling only
Syntax :
-delay num_milliseconds
Specifies the minimum time between HTTP requests, in milliseconds. The default value is 0
milliseconds for no delay.
-header
Type : Web crawling only
Syntax :
-header string
Specifies an HTTP header to add to the spidering request; for example:
-header "Referer: http://www.verity.com/"
Verity Spider sends some predefined headers, such as Accept and User-Agent, by default. Special
headers are sometimes necessary to correctly index a site.
For example, earlier versions of Verity Spider did not support the Host header, which is needed
for Virtual Host indexing. Also, a Proxy-authentication header was needed to pass a username
and password to a proxy server. In the current version of Verity Spider, the Host header is
supported by default, and the
-proxyauth
option is available for proxy server authentication.
Therefore, the
-header
option is maintained only for backwards compatibility and possible
future enhancements.
Note: Misuse of this option causes spider failure. If this happens, rerun the indexing task with
modified
-header
values.
-hostcache
Syntax :
-hostcache num_hostnames
Specifies the number of host names to cache to avoid DNS lookups. Without this option, the
host cache continues to grow.
The default value is 256.
-noflowctrl
Type : Web crawling only
Disables round-robin indexing of websites with network flow control.
By default, Verity Spider uses round-robin indexing of websites to avoid overwhelming a web
server and to improve indexing performance. Verity Spider connects to each web server in a
round-robin manner, using up to the value for the
-connections
option. This means that one
URL is fetched from each web server, in turn.
Note: Using the
-noflowctrl
option can result in a significant drop in performance.