Results 1 to 13 of 13

What is Web crawler?

This is a discussion on What is Web crawler? within the Search Engine Optimization forums, part of the Internet Marketing category; Hello friends, What is Web crawler?...

  1. #1
    Senior Member
    Join Date
    Jul 2013
    Posts
    394

    Default What is Web crawler?

    Hello friends,

    What is Web crawler?

  2. #2
    Member
    Join Date
    Oct 2015
    Location
    Banagladesh
    Posts
    75

    Default Re: What is Web crawler?

    A web crawler (also known as a web spider or web robot) is a program or automated script which browses the World Wide Web in a methodical, automated manner. This process is called Web crawling or spidering. Many legitimate sites, in particular search engines, use spidering as a means of providing up-to-date data.

  3. #3
    Senior Member
    Join Date
    Aug 2015
    Location
    india
    Posts
    168

    Default Re: What is Web crawler?

    A web crawler is a program which is used by search engines to crawl and fetch the data from different webpages and sends the appropriate webpages to search engine database.

  4. #4
    Senior Member RH-Calvin's Avatar
    Join Date
    Jun 2013
    Posts
    3,400

    Default Re: What is Web crawler?

    A web crawler is search engine program that are responsible to read through webpage source and provide them with a cache certificate. These crawlers are also known as search engine spiders and search engine bots are are used to perform certain tasks prescribed by search engines.
    Cheap VPS | $1 VPS Hosting | Windows with Remote Desktop
    Cheap Dedicated Server | $29 Dedicated Server with Free IPMI Setup

  5. #5
    Senior Member Cherin's Avatar
    Join Date
    Apr 2015
    Posts
    208

    Default Re: What is Web crawler?

    Web Crawler is program used to read and analyse the updated web pages and helps in indexing the Website on unique content basis. Crawlers helps in resolving the Website Deadlinks or any errors.

  6. #6
    Junior Member
    Join Date
    Nov 2015
    Location
    Bangalore
    Posts
    6

    Default Re: What is Web crawler?

    Web crawlers are essentially used to collect/mine data from the Internet.

  7. #7
    Senior Member cybnetics's Avatar
    Join Date
    Nov 2015
    Location
    Delhi
    Posts
    105

    Default Re: What is Web crawler?

    Web Crawler crawl websites and collects useful information of website for a limited time of period.

  8. #8
    Senior Member dennis123's Avatar
    Join Date
    Apr 2013
    Location
    Bangalore
    Posts
    2,568

    Default Re: What is Web crawler?

    Crawler – a program that automatically follows all of the links on each web page.

  9. #9
    Senior Member Nexevo Technologies's Avatar
    Join Date
    Aug 2015
    Location
    Bangalore
    Posts
    852

    Default Re: What is Web crawler?

    Spider is crawl the website or a program which automatically fetch the web pages.It is also known as web crawler.

  10. #10
    Member
    Join Date
    Dec 2015
    Location
    Los Angeles, California
    Posts
    41

    Default Re: What is Web crawler?

    A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index. The major search engines on the Web all have such a program, which is also known as a "spider" or a "bot." Crawlers are typically programmed to visit sites that have been submitted by their owners as new or updated. Entire sites or specific pages can be selectively visited and indexed. Crawlers apparently gained the name because they crawl through a site a page at a time, following the links to other pages on the site until all pages have been read.

  11. #11
    Member
    Join Date
    Dec 2015
    Posts
    57

    Default Re: What is Web crawler?

    web crawler is the search engine bot which is crawl website to keep in the data base.

  12. #12
    Senior Member
    Join Date
    Jan 2012
    Posts
    2,014

    Default Re: What is Web crawler?

    A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing.

  13. #13
    Senior Member hostechsupport's Avatar
    Join Date
    Feb 2017
    Posts
    414

    Default Re: What is Web crawler?

    Web Crawling is the methodin which search engines collect relevant information about different websiteson World Wide Web. It is the process in which indexing is done through crawlingweb pages & index them accordingly.
    HostechSupport
    Best Linux/ Windows Server Management
    24x7 Skype: hostechsupportindia