# W3C Link checker
User-agent: W3C-checklink
Disallow:

User-agent: Baiduspider
Disallow: 

User-agent: Googlebot
Disallow:

User-agent: Mediapartners-Google*
Disallow:

User-agent: *
Disallow: /