Sitemap Spider Food

Just a reminder, you no longer have to go to Google or Yahoo to submit your sitemap file location manually. The major engines have (mercifully) agreed to a common sitemap protocol that allows the spiders to find your sitemap file by way of your robots.txt file (for a change, robots.txt tells the spiders where to go instead of where NOT to go).

Just place this line in your robots.txt file and you’re set:


Naturally, you need to change the to whatever your actual domain is.

Sitemap files are terrific for getting all of the pages on your site found by the search engines. If you’ve got a large site and want a free way to spider it and generate a sitemap for you, take a look at


One thought on “Sitemap Spider Food

  1. Yes this is a nice feature, but I would still use Google Webmaster Console tools, because with this robots.txt option you don’t get feedback on if their are issues when the sitemap is parsed. You could have problems and not even know it!!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s