Sitemaps offer better coverage for your Custom Search Engine
Stay organized with collectionsSave and categorize content based on your preferences.
Tuesday, May 06, 2008
If you're a webmaster or site owner, you realize the importance of providing high quality search
on your site so that users easily find the right information.
Wejust announcedtoday thatAdSense for Searchis now powered by Custom Search.Custom Search(a Google-powered search box that you can install on your website in minutes) helps your users
quickly find what they're looking for. As a webmaster, Custom Search gives you advanced
customization options to improve the accuracy of your site's search results. You can also choose
to monetize your traffic with ads tuned to the topic of your site. If you don't want ads, you
can useCustom Search Business Edition.
Now, we're also looking to index more of your site's content for inclusion in your Custom Search
Engine (CSE) used for search on your site. We figure out what sites and URLs are included in
your CSE, and—if you've providedSitemapsfor the relevant sites—we use that information to create a
more comprehensive experience for your site's visitors. You don't have to do anything specific,
besides submitting a Sitemap (viaWebmaster Tools)
for your site if you haven't already done so. Note that this change will not result in more pages
indexed on Google.com and your search rankings on Google.com won't change. However, you will be
able to get much better results coverage in your CSE.
Custom Search is built on top of the Google index. This means that all pages that are available on
Google.com are also available to your search engine. We're now maintaining a CSE-specific index
in addition to the Google.com index for enhancing the performance of search on your site. If you
submit a Sitemap, it's likely that we will crawl those pages and include them in the additional
index we build.
In order for us to index these additional pages, our crawlers must be able to crawl them. Your
Sitemap will also help us identify the URLs that are important. Please ensure you are notblocking us from crawlingany pages you want indexed. Improved index coverage is not instantaneous, as it takes some time
for the pages to be crawled and indexed.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],[],[[["\u003cp\u003eAdSense for Search is now using Google Custom Search, offering webmasters advanced customization and monetization options for their site search.\u003c/p\u003e\n"],["\u003cp\u003eGoogle is working to index more of a website's content for Custom Search Engines (CSEs) to improve search result coverage, especially if a Sitemap is provided.\u003c/p\u003e\n"],["\u003cp\u003eSubmitting a Sitemap through Google Webmaster Tools helps Google understand the important pages on your website to be included in your CSE.\u003c/p\u003e\n"],["\u003cp\u003eThis improved indexing for CSEs does not impact a website's ranking on Google.com or the number of pages indexed on Google.com, only on the site's own search.\u003c/p\u003e\n"],["\u003cp\u003eWebmasters need to ensure their robots.txt is not blocking Google from crawling pages intended for inclusion in the CSE for this improved indexing to be effective.\u003c/p\u003e\n"]]],["AdSense for Search is now powered by Custom Search, allowing webmasters to use a customizable, Google-powered search box on their sites. Webmasters can improve site search accuracy and monetize traffic, or opt for the ad-free Custom Search Business Edition. Google will now index more site content for Custom Search Engines (CSEs), using provided Sitemaps to improve search results. Submitting Sitemaps and ensuring pages are crawlable helps include more content in the CSE-specific index, enhancing site search performance.\n"],null,["# Sitemaps offer better coverage for your Custom Search Engine\n\nTuesday, May 06, 2008\n\n\nIf you're a webmaster or site owner, you realize the importance of providing high quality search\non your site so that users easily find the right information.\n\n\nWe\n[just announced](https://adsense.blogspot.com/2008/05/adsense-for-search-now-powered-by.html)\ntoday that\n[AdSense for Search](https://www.google.com/adsense/afs)\nis now powered by Custom Search.\n[Custom Search](https://www.google.com/cse)\n(a Google-powered search box that you can install on your website in minutes) helps your users\nquickly find what they're looking for. As a webmaster, Custom Search gives you advanced\ncustomization options to improve the accuracy of your site's search results. You can also choose\nto monetize your traffic with ads tuned to the topic of your site. If you don't want ads, you\ncan use\n[Custom Search Business Edition](https://www.google.com/sitesearch).\n\n\nNow, we're also looking to index more of your site's content for inclusion in your Custom Search\nEngine (CSE) used for search on your site. We figure out what sites and URLs are included in\nyour CSE, and---if you've provided\n[Sitemaps](/search/docs/crawling-indexing/sitemaps/overview)\nfor the relevant sites---we use that information to create a\nmore comprehensive experience for your site's visitors. You don't have to do anything specific,\nbesides submitting a Sitemap (via\n[Webmaster Tools](https://search.google.com/search-console))\nfor your site if you haven't already done so. Note that this change will not result in more pages\nindexed on Google.com and your search rankings on Google.com won't change. However, you will be\nable to get much better results coverage in your CSE.\n\n\nCustom Search is built on top of the Google index. This means that all pages that are available on\nGoogle.com are also available to your search engine. We're now maintaining a CSE-specific index\nin addition to the Google.com index for enhancing the performance of search on your site. If you\nsubmit a Sitemap, it's likely that we will crawl those pages and include them in the additional\nindex we build.\n\n\nIn order for us to index these additional pages, our crawlers must be able to crawl them. Your\nSitemap will also help us identify the URLs that are important. Please ensure you are not\n[blocking us from crawling](/search/docs/crawling-indexing/robots/intro)\nany pages you want indexed. Improved index coverage is not instantaneous, as it takes some time\nfor the pages to be crawled and indexed.\n\n\nSo what are you waiting for?\n[Submit your Sitemap](/search/docs/crawling-indexing/sitemaps/overview)!\n\nRajat Mukherjee, Group Product Manager, Search"]]
