# Block all crawlers by default User-agent: * Disallow: /ccic/ Disallow: /chip-aws/chip-aws-v0.6/level2 # Explicitly allow Wget User-agent: Wget Allow: /ccic/ Allow: /chip-aws/chip-aws-v0.6/level2