Stay organized with collectionsSave and categorize content based on your preferences.
Thursday, November 13, 2008
Since welaunched enhanced indexingwith theCustom Search platformearlier this year, webmasters who submitSitemapsto Webmaster Tools get special
treatment: Custom Search recognizes the submitted Sitemaps and indexes URLs from these Sitemaps
into a separate index for higher quality Custom Search results. We analyze your Custom Search
Engines (CSEs), pick up the appropriate Sitemaps, and figure out which URLs are relevant for your
engines for enhanced indexing. You get thedual benefitof better discovery for Google.com
and more comprehensive coverage in your own CSEs.
Today, we're taking another step towards improving your experience with Google webmaster services
with the launch ofOn-Demand Indexingin Custom Search. With On-Demand Indexing, you can now tell us about the pages on your websites
that are new, or that are important and have changed, and Custom Search will instantly schedule
them for crawl, and index and serve them in your CSEs usually within 24 hours, often much faster.
How do you tell us about these URLs? You guessed it... provide a Sitemap to Webmaster Tools,
like you always do, and tell Custom Search about it. Just go to theCSE control panel,
click on the Indexing tab, select your On-Demand Sitemap, and click the "Index Now" button. You
can tell us which of these URLs are most important to you via thepriorityandlastmodattributes that you provide in your Sitemap. Each CSE has a number of pages
allocated within the On-Demand Index, and with these attributes, you can us which are most
important for indexing. If you need greaterallocationin the On-Demand index, as well as more customization controls,Google Site Searchprovides a range of options.
Some important points to remember:
You only need to submit your Sitemaps once in Webmaster Tools. Custom Search will automatically
list the Sitemaps submitted via Webmaster Tools and you can decide which Sitemap to select for
On-Demand Indexing.
Your Sitemap needs to be for a websiteverified in Webmaster Tools,
so that we can verify ownership of the right URLs.
In order for us to index these additional pages, our crawlers must be able to crawl them. You
can use "Webmaster Tools>Crawl Errors>URLs restricted by robots.txt"
or check your robots.txt file to ensure that you're notblocking us from crawlingthese pages.
Submitting pages for On-Demand Indexing will not make them appear any faster in the main Google
index, or impact ranking on Google.com.
We hope you'll use this feature to inform us regularly of the most important changes on your
sites, so we can respond quickly and get those pages indexed in your CSE. As always, we're always
listening for yourfeedback on Custom Search.
Written by Rajat Mukherjee, Group Product Manager, Search
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],[],[[["\u003cp\u003eCustom Search now utilizes Sitemaps submitted to Webmaster Tools, offering enhanced indexing and higher quality results.\u003c/p\u003e\n"],["\u003cp\u003eOn-Demand Indexing allows you to quickly index new or important pages in your Custom Search Engine within 24 hours, often much faster.\u003c/p\u003e\n"],["\u003cp\u003eSubmit your Sitemap through Webmaster Tools and use the "Index Now" button in your Custom Search Engine control panel to utilize On-Demand Indexing.\u003c/p\u003e\n"],["\u003cp\u003eEnsure your website is verified in Webmaster Tools and that your robots.txt file isn't blocking Google from crawling the pages you want indexed.\u003c/p\u003e\n"],["\u003cp\u003eOn-Demand Indexing only affects your Custom Search Engine and does not impact your Google.com ranking or indexing.\u003c/p\u003e\n"]]],["Custom Search now recognizes Sitemaps submitted to Webmaster Tools, indexing URLs from them for higher quality results. With the new On-Demand Indexing feature, users can submit Sitemaps to inform Custom Search about new or updated pages. These pages are then scheduled for crawling and indexing, typically within 24 hours. Users can prioritize URLs via the `priority` and `lastmod` attributes. Submitted pages will not appear faster in the main Google index. Sitemap needs to be for verified website.\n"],null,["# On-Demand Sitemaps for Custom Search\n\n| It's been a while since we published this blog post. Some of the information may be outdated (for example, some images may be missing, and some links may not work anymore).\n\nThursday, November 13, 2008\n\n\nSince we\n[launched enhanced indexing](/search/blog/2008/05/sitemaps-offer-better-coverage-for-your)\nwith the\n[Custom Search platform](https://www.google.com/cse)\nearlier this year, webmasters who submit\n[Sitemaps](/search/docs/crawling-indexing/sitemaps/overview) to Webmaster Tools get special\ntreatment: Custom Search recognizes the submitted Sitemaps and indexes URLs from these Sitemaps\ninto a separate index for higher quality Custom Search results. We analyze your Custom Search\nEngines (CSEs), pick up the appropriate Sitemaps, and figure out which URLs are relevant for your\nengines for enhanced indexing. You get the *dual benefit* of better discovery for Google.com\nand more comprehensive coverage in your own CSEs.\n\n\nToday, we're taking another step towards improving your experience with Google webmaster services\nwith the launch of\n[On-Demand Indexing](https://googlecustomsearch.blogspot.com/2008/11/on-demand-indexing-for-fast-moving-web.html)\nin Custom Search. With On-Demand Indexing, you can now tell us about the pages on your websites\nthat are new, or that are important and have changed, and Custom Search will instantly schedule\nthem for crawl, and index and serve them in your CSEs usually within 24 hours, often much faster.\n\n\nHow do you tell us about these URLs? You guessed it... provide a Sitemap to Webmaster Tools,\nlike you always do, and tell Custom Search about it. Just go to the\n[CSE control panel](https://www.google.com/coop/manage/cse/),\nclick on the Indexing tab, select your On-Demand Sitemap, and click the \"Index Now\" button. You\ncan tell us which of these URLs are most important to you via the `priority` and\n`lastmod` attributes that you provide in your Sitemap. Each CSE has a number of pages\nallocated within the On-Demand Index, and with these attributes, you can us which are most\nimportant for indexing. If you need greater\n[allocation](https://www.google.com/support/customsearch/bin/topic.py?topic=16792)\nin the On-Demand index, as well as more customization controls,\n[Google Site Search](https://www.google.com/sitesearch)\nprovides a range of options.\n\nSome important points to remember:\n\n1. You only need to submit your Sitemaps once in Webmaster Tools. Custom Search will automatically list the Sitemaps submitted via Webmaster Tools and you can decide which Sitemap to select for On-Demand Indexing.\n2. Your Sitemap needs to be for a website [verified in Webmaster Tools](https://support.google.com/webmasters/answer/34592), so that we can verify ownership of the right URLs.\n3. In order for us to index these additional pages, our crawlers must be able to crawl them. You can use \"Webmaster Tools \\\u003e Crawl Errors \\\u003e [URLs restricted by robots.txt](https://www.google.com/support/webmasters/bin/answer.py?answer=35235)\" or check your robots.txt file to ensure that you're not [blocking us from crawling](/search/docs/crawling-indexing/robots/intro) these pages.\n4. Submitting pages for On-Demand Indexing will not make them appear any faster in the main Google index, or impact ranking on Google.com.\n\n\nWe hope you'll use this feature to inform us regularly of the most important changes on your\nsites, so we can respond quickly and get those pages indexed in your CSE. As always, we're always\nlistening for your\n[feedback on Custom Search](https://groups.google.com/group/google-custom-search).\n\nWritten by Rajat Mukherjee, Group Product Manager, Search"]]
