User-agent: * Disallow: Disallow: /wp-admin/*.php Disallow: /wp-admin/includes/*.php Disallow: /wp-includes/*.php Disallow: /wp-content/cache Disallow: /trackback Disallow: /comments Disallow: /wiki/ Disallow: *.pdf Disallow: *.doc Disallow: *.docx Crawl-delay: 100