Mar 20 2008

Cross-host submission for Sitemaps is officially launched

Posted by Tran Tinh in Google, Microsoft, Yahoo
0 Comments

Google, Yahoo and Microsoft are the top three search engines on the internet nowadays and they have never collaborated with each other to make internet much easier to manage and scaled in the past. Further, have been struggling with each other to draw more clients to use their search engines rather than their competitors’. Internet users are confused when they are facing with different structures and algorithms operated by these companies.

Your blogs once submitted to Google are not sure to be crawled by Yahoo or Microsoft Live Search. Therefore, there have been a agreement among three Mr. Giants to make this issue possible and it becomes true now. The first persons who will applaud for this news must be our webmasters.

Today, Yahoo!, along with Google and Microsoft, is announcing Cross-host submission for Sitemaps, which will make it easier for webmasters to manage their Sitemap submissions to the major search engines. With this announcement, webmasters can now submit Sitemaps that correspond to several differently-hosted websites using a single mechanism.

For background, a Sitemap file contains the URLs for the pages on a site along with meta-data, such as priority, last crawled date and change frequency for the content. To ensure validity of this metadata, Sitemaps have previously been required to be on the same host and path as the URLs they contain. This requirement forced the Sitemaps files to be hosted on the same servers as the actual site content.

With today’s announcement, a Sitemap can now be hosted on a different host and path than the URLs it contains. For example, say you have a Sitemap (sitemap-www.xml) for the URLs on http://www.example.com but you want to put that Sitemap on http://sitemaps.example.com. That is now possible. To make the Sitemap valid and preserve data security you need to refer to it from the robots.txt file on the site where the URLs it contains are located. For example, add the following line to http://www.example.com/robots.txt:

Sitemap: http://sitemaps.example.com/sitemap-www.xml

Our collaboration with Google and Microsoft began back in November 2006 when we announced joint support for the Sitemaps protocol. Since then, we’ve learned a lot about how webmasters and site owners manage their sites and feeds. We know that segregating user facing content from feeds, like Sitemaps, is important. We’ve also learned that managing feeds for large websites or websites using third party feed publishing services is critical. We hope this enhancement helps address those needs.

We’ll continue to work on addressing the needs of our webmasters through new standards and protocols. If you have other thoughts about how we can collaborate with other search engines on standards such as robots.txt, we’d love to hear from you. Leave us a comment below, or give us your feedback here.

Details about the Sitemaps protocol, including our recent addition, are available on the protocol website. Or, if you’re at SMXW this week, bring your questions to our panelists and speakers.