What's WP
Our everything is about WordPress - best choice of WordPress users!
How to Optimize WordPress Robots.txt File for SEO Friendly?

How to Optimize WordPress Robots.txt File for SEO Friendly?

When it comes to boosting website SEO, robots.txt is an indispensable factor to be discussed. So, what is robots.txt, why is it so important, and how can you optimize WordPress robots.txt file for SEO friendly? If you have no idea about the answers to these questions, then, the post is prepared for you. In the following, we are going to reveal all answers in simple and easy language.

What Is Robots.txt?

Robots.txt is a simple text file that controls how search engines behave with your website. It directs search engines about where they are allowed to index and where they can’t index. When search engines’ robots crawl on your website, they will firstly follow orders from robots.txt.

By default, search engines are designed to index as much information as they can and robots.txt is a restriction for the practice. You may wonder why you should limit them to acquire information as the more search engine robots crawl, the more chances your websites get to have a good ranking place.


In total, there are two reasons for using the file. For one thing, opening all data to search engines makes your site vulnerable and easy to be hacked. For another thing, allowing search engines to access your site unrestrictedly can consume a large amount of bandwidth; and in return, make you website slow.

How to Optimize WordPress SEO with Robots.txt?

After reading all information above, you may have a clear understanding about the definition of robots.txt and why you should use it for your WordPress website. Excessive constraints on search engines’ behavior may have negative impact on website SEO while too much freedom for them may put websites in danger.

WordPress SEOTherefore, you need to work out a solution that can give consideration to both website security and SEO. In the following, our editors give you an example that can restrict search engine behaviors and promote SEO. We show you the whole process to achieve this as below.

Firstly, log into your hosting control panel, for example, cPanel. Find and open file manager in file directory besides Backups. Now, you can see your website files that usually include Robots.txt. However, if you fail to find the file, just choose to create a new file and name it as Robots.txt.

Then, you need to right click on your Robots.txt file and click on code edit after it opening. Later, choose utf-8 as character encoding and click on edit. After this, copy and paste the following codes in the file.

Optimize Robots.txt Codes


Lucy has been a very experienced SEOer, technical writer, web developer, c# developer since 2002. Now she owns a startup in San Francisco, CA, focusing on running a couple of blogs to share knowledge and experience with global readers and deliver exceptional results to global sponsors by leveraging the power of Internet.


Want to Learn More About WordPress?

Get Our Free Themes, Plugins, Support, SEO Tips, and Other Tutorials!

Email *