Bingbot is the crawler Bing uses to discover and fetch web pages. Its user agent strings contain the token bingbot/2.0 and a link to bing.com/bingbot.htm. Like other major crawlers, it reads robots.txt before crawling, and site owners can manage it through Bing Webmaster Tools. If you block Bingbot, your pages cannot be crawled for Bing's index.
Need help with SEO?
Comppound runs SEO and AI search for SaaS companies. We plan what to publish, produce it with AI, and every page passes automated checks, and a person reviews samples and anything high-risk.
- Published to your site and tracked in Search Console
- One person answers for your account
- Plans from $400 a month, terms from 3 months
Bing's other crawlers
Bingbot is the main crawler, but Bing's documentation lists others. Check for them before writing a firewall or bot rule, so you do not block something by accident.
- AdIdxBot, used for Bing Ads
- BingPreview, used for page previews
- MicrosoftPreview, also a preview crawler
- BingVideoPreview, used for video previews
Controlling and verifying it
Bingbot honors robots.txt, including the crawl-delay setting. Bing Webmaster Tools also has a Crawl Control setting for adjusting hourly crawl rates. Blocking everything is rarely the goal, so make any rule as narrow as you can.
User agent strings are easy to fake, so a matching string does not prove a request came from Bing. Bing offers a Verify Bingbot tool that takes an IP address and tells you whether it belongs to a Bingbot server.
User-agent: bingbot
Disallow: /internal-search/Accidental blocks
Firewalls and CDNs can block crawlers without anyone on the team noticing, and traffic from Google alone will not reveal the problem. Check server logs for Bingbot requests and their response codes after changing any bot rule, and check Bing Webmaster Tools too.
Example
A bot rule that went too far
Hypothetical project tool Ganderly tightened its CDN bot rules after a scraping incident. Google traffic held steady, so nobody noticed that Bing traffic was falling away.
A check of server logs showed requests with the bingbot token being challenged. The team used the Verify Bingbot tool on the IP addresses, confirmed they were real, and added an exception.
Common mistakes
Allowing bots on the user agent alone
Scrapers often pretend to be Bingbot. Verify the IP address before giving a request special treatment.
Testing only for Googlebot
Bot rules and robots.txt groups can treat crawlers differently. Check Bingbot separately whenever these change.
Frequently Asked Questions
Use Bing's Verify Bingbot tool with the IP address from your logs. The user agent alone can be faked.
Yes, Bing says Bingbot honors crawl-delay in robots.txt. You can also adjust crawl rates using Crawl Control in Bing Webmaster Tools.
Keep reading
- Bing: Bing is Microsoft's search engine. Its index also powers other products, so Bing visibility can matter beyond Bing's own results page.
- Bing Webmaster Tools: Bing Webmaster Tools is Microsoft's free console for site owners. It shows how Bing crawls and indexes a site and offers SEO reports and URL submission.
- User agent: A user agent is the string a browser or crawler sends to identify itself. In SEO it is how robots.txt rules and server logs tell bots apart.
- robots.txt: robots.txt is a plain-text file at a site's root that tells crawlers which paths they may or may not request.
- Brand SERP: A brand SERP is the results page people see when they search your company or product name. For a SaaS company it is often the last page a buyer checks before booking a demo.
- Brave Search: Brave Search is a privacy-focused search engine from the maker of the Brave browser that runs its own independent web index.
- Continuous Scroll: Continuous scroll was Google's 2021 to 2024 results layout that loaded more results as users scrolled. Google returned to paginated results in 2024.
- Discussions and Forums: Discussions and forums is a Google results feature that surfaces threads from Reddit, Quora and other forums for queries where first-hand views help.
