User-Agent: YaDirectBot Disallow: User-agent: * Disallow: /multitran$ Disallow: /web_services$ Disallow: /promt$ Disallow: /ectaco.htm User-agent: Yandex Disallow: /multitran$ Disallow: /web_services$ Disallow: /promt$ Disallow: /ectaco.htm Disallow: /index.htm User-agent: Purebot Disallow: / User-agent: Purebot/1.1 Disallow: / User-agent: SolomonoBot Disallow: / # such disall. with a dollar sign forbids '/page', # but doesn't forbid '/page.htm'