Indexing ajax loaded contents

Is there a widely used standard way on how to index ajax loaded content (for search engines)?

For example, indexing HTML content that would dynamically be inserted into a page.



You may want to consider using some sort of sitemap generator that aggregates all the content you normally load through AJAX.

Sitemaps are particularly beneficial on websites where:

  • Some areas of the website are not available through the browsable interface, or
  • Webmasters use rich Ajax, Silverlight, or Flash content that is not normally processed by search engines.

From Wikipedia - Sitemaps

Remember that:

Because most web crawlers do not execute JavaScript code, publicly indexable web applications should provide an alternative means of accessing the content that would normally be retrieved with Ajax, to allow search engines to index it.

From Wikipedia - AJAX Drawbacks

In addition you may be interested in checking out the following articles:

One way of doing this is using JS fallbacks for dialog boxes like thickbox: A link would point to the dialog box loading Ajax content, and the fallback href='...' would point to a search engine-readable representation of that content (i.e. the HTML snippet that the AJAX function would load, but surrounded by the necessary HTML body basics).

Example (I pulled rel='box' out of my arse, this is supposed to be the anchor for the box plugin, like rel=thickbox):

<a href='/encyclopedia/definition/mushroom.html' rel='box'>Definition of Mushroom</a>
  • Clicking on the link in a Ajax/JS enabled browser will open a nice dialog box with the article
  • Clicking on the link without JS (or as a search engine) will lead to a new page containing the article (which needs some server side intelligence to detect which channel the request came from).

That's all that comes to my mind in this direction. Ajax and search engines is a widely uncharted field otherwise.

Have Javascript fallbacks. Have a look at Amazon Diamond Search with and without Javascript enabled. Read up on

I don't really know the answer, but it seems to me that ajax-loaded content won't help to improve se positions because search engine can't refer to ajax-loaded content. Another words search engine can't say: "Hey, go here and then click 3rd button from the top to see the content you're interested in.".

I think that good idea is to put this content to xml and put link to this xml at tag (like URL to RSS)...

What about using an alternative content for JS disabled clients (search engines)? I think there is no other way of letting the search engines index your AJAX site properly.

I think actually only Google really implements a specification to index AJAX content. It's the Google AJAX crawling specification.

We have used that for our website, there is an example in our technical blog on how to do that with Django in a clean way.

Need Your Help

Project Reference Problem, Visual Studio 2010

visual-studio visual-studio-2010 .net-4.0 reference project-reference

I have a solution that has one class library project, and the others are WPF applications. I have added class library's reference the but, but both of them cannot use its classes.

What is the simplest way to implement an Entity Framework 6 Add-or-Update method

c# database entity-framework linq entity-framework-6

I have a project that uses Entity Framework 6 that does an Add-or-Update (i.e. Upsert) for a bunch of different entity types. For each of these entities, there is an surrogate integer ID (i.e. pri...