| Baiduspider |
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) |
| PerplexityBot |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot) |
| Claude-SearchBot |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; [email protected]) |
| DuckAssistBot |
DuckAssistBot/1.1 (https://duckduckgo.com/duckassistbot) |
| Bravebot |
Mozilla/5.0 (compatible; Bravebot/1.0; +https://brave.com/search/) |
| Meta-ExternalAgent |
Mozilla/5.0 (compatible; Meta-ExternalAgent/1.0; +https://developers.facebook.com/docs/sharing/webmasters/crawler) |
| Kimi-SearchBot |
Mozilla/5.0 (compatible; Kimi-SearchBot/1.0; +https://kimi.ai/) |
| Qwenbot |
Mozilla/5.0 (compatible; Qwenbot/1.0; +https://qwen.alibaba.com/) |
| GPTBot |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot |
| cohere-ai |
Mozilla/5.0 (compatible; cohere-ai; +https://cohere.com/crawler) |
| YouBot |
Mozilla/5.0 (compatible; YouBot/1.0; +https://you.com/bot) |
| YiBot |
Mozilla/5.0 (compatible; YiBot/1.0; +https://01.ai/) |
| DeepSeekBot |
Mozilla/5.0 (compatible; DeepSeekBot/1.0; +https://www.deepseek.com/) |
| MoonshotBot |
Mozilla/5.0 (compatible; MoonshotBot/1.0; +https://kimi.ai/) |
| KimiBot |
Mozilla/5.0 (compatible; KimiBot/1.0; +https://kimi.ai/) |
| Hunyuan |
Mozilla/5.0 (compatible; Hunyuan/1.0; +https://hunyuan.tencent.com/) |
| Googlebot |
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| MistralAI-User |
Mozilla/5.0 (compatible; MistralAI-User/1.0; +https://mistral.ai/) |
| ChatGLM-Spider |
Mozilla/5.0 (compatible; ChatGLM-Spider/1.0; +https://zhipuai.cn/) |
| Applebot |
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15 (Applebot/0.1) |
| PanguBot |
Mozilla/5.0 (compatible; PanguBot/1.0; +https://www.huaweicloud.com/) |
| ClaudeBot |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; [email protected]) |
| Perplexity-User |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexitybot) |
| CCBot |
CCBot/2.0 (https://commoncrawl.org/faq/) |
| Google-Extended |
Mozilla/5.0 (compatible; Google-Extended; +http://www.google.com/bot.html) |
| Amazonbot |
Mozilla/5.0 (compatible; Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot) |
| ChatGPT-User |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot |
| GrokBot |
Mozilla/5.0 (compatible; GrokBot/1.0; +https://x.ai/) |
| Claude-User |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; [email protected]) |
| xAI-Grok |
Mozilla/5.0 (compatible; xAI-Grok/1.0; +https://x.ai/) |
| OAI-SearchBot |
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot |
| Bytespider |
Mozilla/5.0 (compatible; Bytespider; [email protected]) AppleWebKit/537.36 |
全TM是ai
这都是哪些公司的爬虫🦎