I would like to use Swiftype for a search engine on my site but my site is pass word protected. Is that an issue when submitting my domain for indexing?
Swiftype does work with .htaccess files, but you’ll need to open access to our crawler’s user-agent string. The user-agent string can be faked by malicious clients, so please understand that you might be opening access beyond your comfort level.
But, if you want to password protect your site from all users and open access to the Swiftype crawler, the following .htaccess rules will work. Note that you’ll need to change the
AuthUserFile line below:
Order deny,allow Deny from all AuthType Basic AuthName "Password Protected Area" AuthUserFile /path/to/.htpasswd Require valid-user SetEnvIfNoCase User-Agent .*Swiftbot.* search_robot Allow from env=search_robot Satisfy Any
If you’d like to receive your account specific User-Agent string, please contact our support team from your Swiftype dashboard.