Convert a robots.txt file to include the equivalent
ACAP crawler records

To start using ACAP on your website to communicate with crawlers, you need to add ACAP crawler records to your robots.txt file. If you're not sure whether or not you have a robots.txt file on your website, you probably need to consult your webmaster / website manager and follow their advice on how best to proceed. The remainder of this page is addressed to webmasters and others with responsibility for maintenance of your website.

If you already have a robots.txt file, you will probably want to keep all the existing crawler (robot / spider / User-agent) records as they are, so that any crawlers that don't understand ACAP can continue to use conventional crawler records to determine which resources you are allowing them to crawl and index, if any. But in order to be ready for when crawlers start to understand ACAP – and to encourage them to do so – we suggest that you now convert all your existing crawler records to use the ACAP extensions to the robots.txt syntax. By doing this you are immediately letting crawlers know that you are enabling ACAP on your website. In other words, we suggest that you start off by extending your robots.txt to contain both conventional records and ACAP records that communicate the same information.

Once you have done this, you will probably want to start refining your robots.txt file, to use the new forms of permission and prohibition expression that are available in the ACAP extensions to the robots.txt syntax. To do this you will need to consult the documents that specify the extensions to the Robots Exclusion Protocol that ACAP is proposing, which can be found here: >.

To get things started, we have developed a very simple tool (a CGI script) that makes it really easy to convert your existing robots.txt file. All you need to do is to upload your robots.txt file using the simple form below. Our server will respond with a page containing the converted robots.txt file which you can then save to your local file system from your web browser.

Upload robots.txt file here...

To select the file for upload, click on the 'Browse' button, browse your file system until you find your robots.txt file, select it, then click the 'Open' button in the browse dialogue.

Once you have the correct path and filename in the browse box above, click on the 'Convert' button below to upload it to our server. The conversion process should take only a few moments, after which the converted file will be displayed in your browser. Note that, due to a known bug in Mozilla Firefox, you cannot save it directly from the browser window, but must copy the content of the window to the clipboard, paste into a blank text document (e.g. in Notepad) and save from the document editor.