Forumer - Alphawave

Adding a robots.txt file - RapidWeaver Forum

Hi everyone, Could anyone advise how to add a robots.txt file to a RapidWeaver site? It's not generating one automatically.

You can't access it through RW. It lives on your server/web host. You normally need to create one first. Just name it “robots.txt”.

Old Hard to Find TV Series on DVD

You can use http://tools.seobook.com/robots-txt/generator/ to create robots.txt file and upload it to your website root folder… @Tour2pondy.

So I created a simple robots.txt file. I'm not sure what I need to put in it … I want the crawler to only see the published pages, and it seems ...

Add an HTML page to your project. Set the page to not use a theme in the inspector. Name the page robots.txt; Populate the robots.txt rules ...

My website hosting company recommended I include a “Robots.txt” file in my site. (The SEO tools from my website company is an automated ...

Who can tell me how to set up RapidBot so that it allows access only to the main index page (HOME page) and disallows access to everything ...

When I create a new webpage in RW I am prompted to decide whether or not to Enable Robot Meta Tags. As a subset to allowing the Robot Meta ...

txt file in the root of the site. If it doesn't and you don't have the robots meta tag on pages, search engines will default to indexing them.

This plugin's single purpose was to create a robots.txt file. Although I could create that file manually, RapidBot made it easier and prevented ...