Search Engine Optimization Crawling and Indexing
Today we will learn how to making your website easy to crawl and indexed by the search engine. In crawling your website, SEO (Search Engine Optimization) performs is very important role. Crawling means find out your links and crawling your site. If the search engines is not able to crawling your website, the site will not get ranked top. Also, If your site will not be crawled by search engine, then site will not be accessible easily.
If the your website is accessible it means all the links and pages of your site indexed and can easily rank the target keywords.
The Working of Search Engine Optimization depends on these three facts mainly as follows:
- Crawling
- Indexing
- Ranking
Crawling
Crawling means search engine can track your website links and useful pages. This process is done by Spider of Google crawler. We always keep in mind to stop the web page those we didn’t want to crawl by using rel=”nofollow” attribute of links.
With “nofollow” property, we can stop both Links on the page or specific pages itself to crawl. Usually we stop crawling of a certain part of our site is by using Robots.txt file.
- Add the rel=”nofollow” tag to links on a page you do not want crawled.
- Use <a href=”signin.php” rel=”nofollow”>sign in</a> to stop crawling links.
We can also you meta tag to control the crawling by the google search engine. You can place this meta tag in the head section of your web page. By using content=”noindex” you can stop to index the web page by a search engine you can also stop to index the images on the page.
Keep a Simple URL Structure:
A site’s URL structure should be as simple as possible. For example, if you’re searching for information about Opencart Marketplace, a URL like https://store.webkul.com/OpenCart-MarketPlace.html will help you decide whether to click that link. A URL like http://www.example.com/index.php?id_sezione=360&sid=3a5ebc944f41daa6f849f730f1, is much less appealing to users.
Consider using punctuation in your URLs. The URL http://oc.webkul.com/Booking-Reservation/ is much more useful to us than http://oc.webkul.com/BookingReservation/. Google recommends that you use hyphens (-) instead of underscores (_) in your URLs.
Indicate paginated content:
Sites paginate content in various ways. For example:
- News and/or publishing sites often divide a long article into several shorter pages.
- Retail sites may divide the list of items in a large product category into multiple pages.
- Discussion forums often break threads into sequential URLs.
If you paginate content on your site, and you want that content to appear in search results, use rel="next" and rel="prev" attributes to links.
In the <head> section of the first page (http://www.example.com/article-part1.html), add a link tag pointing to the next page in the sequence, like this:
<link rel=”next” href=”http://www.example.com/article-part2.html”>
Indexing
Indexing is the process of adding Webpages into Google search. Depending upon what meta tag you used for your page (Index or NO-index), Google crawl and index your pages. No-index means that page will not be added in Web search. By default any WordPress post or pages are indexable.
A good idea for better search engine ranking is to let only vital part of your blog/Website be in index and do not index un-necessary archive like tags, categories and all such pages.