I would like to use Swiftype for a search engine on my site but my site is pass word protected. Is that an issue when submitting my domain for indexing?
Swiftype does work with .htaccess files, but you’ll need to open access to our crawler’s user-agent string. The user-agent string can be faked by malicious clients, so please understand that you might be opening access beyond your comfort level.
But, if you want to password protect your site from all users and open access to the Swiftype crawler, the following .htaccess rules will work. Note that you’ll need to change the AuthUserFile line below:
Order deny,allow
Deny from all
AuthType Basic
AuthName "Password Protected Area"
AuthUserFile /path/to/.htpasswd
Require valid-user
SetEnvIfNoCase User-Agent .*Swiftbot.* search_robot
Allow from env=search_robot
Satisfy Any
If you’d like to receive your account specific User-Agent string, please contact our support team from your Swiftype dashboard.