User-agent: * Disallow: * Disallow: /data Disallow: /_srv-msg Disallow: /srv-bin User-agent: gsa-crawler (This names the user-agent that the file targets.) Disallow: /*.asmx