Causes the Search Appliance to wait the specified number of seconds between
page fetches. Normally set this to 0, and the Search Appliance will
fetch and process pages as quickly as it can. Increase the Walk
Delay if the web server cannot handle being hit rapidly. Increasing
this value forces the walk to take at least the following number of
seconds to complete: the Walk Delay number times the number of pages
on the site.
Decimal numbers may be specified - 0.1 will cause it to walk no
more than 10 pages per second, etc.
Note: Using a delay larger than 0 forces Threads
(3.4.10) to 1 to avoid possible fetch timeouts. Thus a
non-zero delay defeats the advantage of multiple threads. Also note
that if Robots robots.txt is Y, then a Crawl-delay
value in a site's robots.txt will override this setting.