Results 1 to 4 of 4

Robot.txt

This is a discussion on Robot.txt within the Blogging forums, part of the Internet Marketing category; What is Robot.txt file in seo ?...

  1. #1
    Junior Member
    Join Date
    Jul 2017
    Location
    mumbai
    Posts
    6

    Default Robot.txt

    What is Robot.txt file in seo?

  2. #2
    Junior Member
    Join Date
    Jul 2017
    Posts
    21

    Default Re: Robot.txt

    Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.

    The location of robots.txt is very important. It must be in the main directory because otherwise user agents will not be able to find it they do not search the whole site for a file named robots.txt.

  3. #3
    Senior Member RH-Calvin's Avatar
    Join Date
    Jun 2013
    Posts
    1,874

    Default Re: Robot.txt

    Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
    Cheap VPS | $1 VPS Hosting | Windows with Remote Desktop
    Cheap Dedicated Server | $29 Dedicated Server with Free IPMI Setup

  4. #4
    Junior Member
    Join Date
    Apr 2017
    Posts
    15

    Default Re: Robot.txt

    Robots.txt is a text file which contains list of all webpages that can be easily accessible by search engine.