Detect Googlebot device in sitecore for
dynamic pages.

Hello friends, You must heard about googlebot !!!  Yes it’s Google’s web crawling bot (sometimes
also called a “spider”). Is the process by which Googlebot discovers
new and updated pages to be added to the Google index. It is very important
for SEO prospective..

Sometime in the programming world we just focus on the performance of
the webpage and try to write the best code and just skip the SEO Interest.  Yes google crawling is very important factor to increase the rank of the page.

Sometime in dynamic content or dynamic page where the DOM is not fixed
and content itself ( html) coming through ajax calls may create the problem to

In the recent project we are using AJAX based HTML pages where content(DOM)
updates through ajax calls. Then our SEO team came up with the problem that the google crawler (googlebot) is not crawling the page properly, and the rank of
the page is going down.
So we researched a lot and came with the below solution.

1.) Create 2 version of the same page one for the Google and one for the end user.
  • The First versions of the page is for google is the static page means normal postback not used more complex ajax calls. The whole purpose is to give the proper HTML(Dom) so that google can crawled it properly.
  • The second version of the page is fully AJAX based to the rest of the world.

2.) Create the different device for googlebot to detect the googlebot agent and assign the different presentation to this device as below:
  •  Go to the device >
  •  Create one more device named Googlebot and assign the value as  Googlebot in Browser agent  field.
  •  Then go to the page item > Presentation and assign the different presentation
    (without ajax  call layout) to googlebot device.

So when Google crawl the page, sitecore  detect the agent “googlebot” and rendered the different
presentation for google as we already defined in the presentation section.

So finally we served to the google with the
desired HTML- DOM format that google supposed to crawled , so the page can be  crawled properly.

Now my page ranking is up, and my SEO team is happy 🙂

We can also verify the “googlebot” agent presentation on the page through the browser, after manually setup the user-agent of the browser,

I hope it will help you.

Feel free to post your comment/query or any suggestion.

You can also connect me on twitter
Happy Sitecoring..

Leave a Reply

Your email address will not be published. Required fields are marked *