How to block baidu spiders

As a veteran in baidu seo, I can say that baidu spider spider is super lazy, and hard to dealing with. To end this mad spider, I will show you how to block baidu spiders.

Most Chinese SEOers feel baidu spider is crazy because they normal won’t follow the rules you made. For example, spiders should crawl the pages following the rules you build in search engine webmaster (such as sitemap, URL submission etc.) When you go throw server log, you can see that baidu spiders crawl you site like they are in the zoo. Only crawl what they like to crawl, and ignore all other pages which you think is important.

Let’s take look how to use robots.txt to stop baidu spiders.

User-agent: Baiduspider

Disallow: /

Put code above into your site’s robots.txt file, will prevent baidu web spider crawling the site.

Please do notice that, I said Baidu WEB spider. Not all baidu spiders.

How many kinds of spiders do they have? Let’s take a look of the list below.

  • Baiduspider (for web pages)
  • Baiduspider-image (for images)
  • Baiduspider-video (for videos)
  • Baiduspider-news (for news)
  • Baiduspider-favo (baidu favorites)
  • Baiduspider-cpro (baidu adsense)
  • Baiduspider-sfkr (baidu sem spider)

If you want to Block Baidu, you should block them all like this

User-agent: Baiduspider

Disallow: /

User-agent: Baiduspdeir-image

Disallow: /

User-agent: Baiduspider-video

Disallow: /

User-agent: Baiduspider-news

Disallow: /

User-agent: Baiduspider-favo

Disallow: /

User-agent: Baiduspider-cpro

Disallow: /

User-agent: Baiduspider-sfkr

Disallow: /

 

Baidu official | 4 methods to improve Index rate in Baidu

Baidu index rate (sample)

Hello guys, Park again. today I will translate (also including some of my thinkings)  a Baidu official article about how to improve index rate. Most of time, the Baidu index rate never go up not because the content you provide, mainly reason is the Baidu spider never crawled your pages.

I will show 4 methods about how to improve your index rate (actually it’s crawling rate) in Baidu below. Before you reading this, please understand that, baidu spider is super lazy. It’s efficiency is much lower than Google bot.

  1. Active submit
  2. Sitemap submit
  3. Automatic submit
  4. Manual submit

Active Submission: Baidu officially recommended

In this method, baidu will provide an port code (we called it token value) for each registered site. Site owners could use the port code to input a special function(php function) into their sites. When the page is updated or a new page is published, the function will be active, and the new URL will submit to Baidu automatically.

Sitemap Submission

Same as google, but Google sitemap submission seems more efficient.

Automatic Submission: I highly recommended

I really like this submission, which is simple. Put a short javascript code into every page you got. When the page is opened, the URL will be automatically submitted to Baidu. Here is a small trick of mine, If the page was not indexed for a long time. We can use a click simulator to open the page over and over again.

Manual Submission

Do not use it, it’s stupid and waste of time.

At last, I want to explain why index rate is so important in Baidu. Index means your pages have chances to be explored by search engine users. which indicates that the page is scored by search engine. So, index rate will be the main element to determine the average page score of you site.  which decides how many users can see your site. (hope you guys understand what I’m saying)