Website Crawl Settings - Microsoft

By default, Microsoft adjusts its crawl frequency and timing based on the server load it detects from your website. If your server encounters bandwidth-related performance problems—such as receiving too many requests within a short period—you may consider lowering how often Microsoft and other search engines crawl your pages. Excessive rapid requests can slow down your website and may even cause server performance issues, especially on heavily trafficked sites or those with inefficient coding.

Modifying Microsoft’s Crawl Settings

Microsoft gives you the ability to define a custom crawl rate for your site and even configure a crawl schedule by hour (a crawl pattern). This lets you instruct Microsoft to crawl more aggressively during off-peak periods and scale back during hours when most of your visitors are active.

To adjust Microsoft’s crawl settings:

1. Sign in to your Microsoft Bing | Webmaster Tools.
2. If you have not already added your website, enter your site information in Microsoft Bing | Webmaster Tools and click Add.

   
    

3. You must verify your website. Choose one of the available verification methods.

   
    

   If you selected the XML File method, follow the steps below.

   1. Download the XML file and upload it to your site’s document root.
      You may upload the XML file directly through the File Manager. Refer to the article How to Upload Files Using File Manager.
   2. After uploading, click the Verify button.
4. Once verified, a “Site addition successful” page will appear.

   
    

5. Click Done.
6. You will then be taken to the Microsoft Bing homepage.
7. Select the hamburger menu icon on the upper left.

    

   
8. Choose Configuration and then click Crawl Control.

9. On the Crawl Control page, you can use the provided presets in the Time when your site receives most of the traffic section to indicate when your site is busiest.

   To adjust crawl rate hours according to your local time, select your appropriate time zone from the Time zone drop-down menu and configure your preferred crawling pattern.

   
    

10. Alternatively, you may create a custom hourly crawl pattern by adjusting the cubes to reflect your desired intensity on the timeline (x-axis). More cubes increase crawl frequency, while fewer cubes slow it down.

    
     

11. Click Save Changes to apply your new settings.

For additional details, you may review Microsoft Bing’s documentation.