09-12-2011, 07:26 PM
hi,
i know that in the robots.txt file i can tell google what pages on my site not to crawl.
are there any other uses of this file?
can it be useful for seo in any way?
i dont think theres any pages on my site that i dont want search engines to crawl so im thinking i might not need this file unless there are other reasons for it
thanks.
i know that in the robots.txt file i can tell google what pages on my site not to crawl.
are there any other uses of this file?
can it be useful for seo in any way?
i dont think theres any pages on my site that i dont want search engines to crawl so im thinking i might not need this file unless there are other reasons for it
thanks.