Absolutely! You can use a robots.txt file to limit what parts of your site the Swiftype search agent will crawl for engine indexing.
For additional details about robots.txt support, you can refer to our documentation.
Absolutely! You can use a robots.txt file to limit what parts of your site the Swiftype search agent will crawl for engine indexing.
For additional details about robots.txt support, you can refer to our documentation.