...
The URL of the indexer page is configured in the properties.txt
file (metaurl
parameter) and is the default start page in the Setup Tool. The search engine crawler retrieves all the URLs from the indexer page and creates requests for each of the documents. Requests are always sent to the front-end frontend in order to benefit from existing caching and to take authorization and personalization into account. With the default configuration, the crawler only indexes links from the indexer page but does not follow links in the retrieved documents. This means the default index depth is 1. An index depth of 1 ensures that the indexing process is efficient and that no outbound links (links outside the channel) are found and indexed.
...