Skip to content

Commit 0c02109

Browse files
committed
Added more AI bots
1 parent 1fff03b commit 0c02109

1 file changed

Lines changed: 15 additions & 3 deletions

File tree

robots.txt

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,25 @@ Disallow: /20*
1515
User-agent: ClaudeBot
1616
Disallow: /20*
1717

18-
User-agent: BardBot
18+
User-agent: CCBot
1919
Disallow: /20*
2020

21-
User-agent: J1Bot
21+
User-agent: cohere-ai
2222
Disallow: /20*
2323

24-
User-agent: CCBot
24+
User-agent: AI2Bot
25+
Disallow: /20*
26+
27+
User-agent: Diffbot
28+
Disallow: /20*
29+
30+
User-agent: Bytespider
31+
Disallow: /20*
32+
33+
User-agent: Google-Extended
34+
Disallow: /20*
35+
36+
User-agent: Applebot-Extended
2537
Disallow: /20*
2638

2739
Sitemap: https://csswizardry.com/sitemap.xml

0 commit comments

Comments
 (0)