Indexes websites for inclusion in Perplexity's search results; does not crawl content for AI model training.