Virtual Robots.txt

Virtual Robots.txt is a simple WordPress plugin for easily editing the robots.txt content on your website. Because the goal of any CMS is to be as self-contained as possible, and because WordPress itself contains a default "virtual" robots.txt file generated by the wp_options table, it makes sense to maintain a virtual robots.txt whenever possible to make site management and migration more controlled.

Instead of simply using the built-in robots_txt filter, however, this plugin creates a new hook for the main parse_query action to ensure that no other WordPress plugin (or theme) can alter the contents of robots.txt after it has been edited. Although the current version does not directly "disable" the robots_txt filter, in effect that filter is completely ignored by ending the hook sequence before robots_txt can be called.

Unlike similar plugins (or WordPress itself), it includes a default "allow all" ruleset upon activation. The theory behind this is that playing things safe is always the best course of action for inexperienced webmasters; allowing all robots to access all site content is not only recommended by Google but also makes sense in the vast majority of cases. While the form value will persist on plugin deactivation, the option value will be completely deleted in the case of the plugin being removed from your site, in an effort at clean code and to avoid any confusion or conflicts.

Compatibility:

• Meant for Linux servers
• Apache works automatically, Nginx requires try_files rule (see FAQ)
• Minimum PHP version: 5.5
• Designed for: PHP 7+ and MySQL 5.7+
• Can be used as a "Must Use" plugin (mu-plugins)

Future plugin goals:

• Localization (translation support)
• Hook/filter/sequence experimentation
• Option to switch between Virtual and Physical robots.txt
• More features (based on user suggestions)
• Code maintenance/improvements

Code inspired by:

• WordPress Robots.txt File
• Virtual Robots.txt

NOTE: We released this plugin in response to our managed hosting clients asking for better access to their server environment, and our primary goal will likely remain supporting that purpose. Although we are 100% open to fielding requests from the WordPress community, we kindly ask that you consider all of the above mentioned goals before leaving reviews of this plugin, thanks!