Visual robots.txt Generator

Rules

🤖

No rules added yet

Click "Add Rule" to get started

Generated robots.txt

# Add rules to generate robots.txt content

How to use:

  1. Add rules using the "Add Rule" button
  2. Set User-Agent (use "*" for all crawlers)
  3. Choose Allow or Disallow
  4. Specify the path (e.g., "/", "/admin", "/private")
  5. Copy the generated text to your robots.txt file

Common robots.txt Examples

Block all crawlers:

User-agent: *
Disallow: /

Block admin area:

User-agent: *
Disallow: /admin/
Disallow: /wp-admin/

Allow Google, block others:

User-agent: Googlebot
Allow: /

User-agent: *
Disallow: /

Block images:

User-agent: *
Disallow: /images/
Disallow: *.jpg
Disallow: *.png