Here you may specify URLs for individual pages to include in the
index. These pages are fetched and stored in the database like others
but the hyperlinks on them are not followed during a walk.
If you change this and click "Update" instead of "GO" the added pages
will be fetched immediately and added to the existing database.
Pages removed from the list will not be removed from the database
until the next rewalk.
Note that since this setting is intended for "one-off" individual
URLs, robots.txt will not be fetched for these pages.