IIS 6.0 Won't Pass Bot Requests to Prerender.io

We have a website coded with AngularJS. Since much of the site is generated via javascript, search engines can't index the pages. So, we have setup Prerender.io to index our site. It does that fine and I can view the rawhtml from their site.

If I enter the following into a browser, Prerender.io will display the rawhtml correctly: http://service.prerender.io/http://www.swiftlearning.com/?_escaped_fragment_=/login

The problem is that I can not get IIS 6 to send the bot requests to Prerender.

When I take this URL: http://www.swiftlearning.com/#!/login and replace the #! with ?_escaped_fragment_= resulting in http://www.swiftlearning.com/?_escaped_fragment_=/login

IIS returns the initial web site page and displays the following URL: http://www.swiftlearning.com/?_escaped_fragment_=/login#!/home

I have setup Wireshark to capture the traffic. It shows that the request comes in and then the response comes from the website, not prerender.io.

I created a web.config file (with what I have already found on StackOverflow) with the following configuration.

<?xml version="1.0"?>
                <add name="X-Prerender-Token" value="dTaPu5H97XTS618Y8edm" />
            <add name="Prerender" type="Prerender.io.PrerenderModule, Prerender.io, Version=, Culture=neutral, PublicKeyToken=null"/>
                <!--# Only proxy the request to Prerender if it's a request for HTML-->
                <rule name="Prerender" stopProcessing="true">
                    <match url="^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent))(.*)" ignoreCase="false" />
                    <conditions logicalGrouping="MatchAny">
                        <add input="{HTTP_USER_AGENT}" pattern="baiduspider|facebookexternalhit|twitterbot" />
                        <add input="{QUERY_STRING}" pattern="_escaped_fragment_" ignoreCase="false" />
                    <action type="Rewrite" url="http://service.prerender.io/http://{HTTP_HOST}{REQUEST_URI}" appendQueryString="false" />
                <rule name="Redirect To Index" stopProcessing="true">
                    <match url=".*" />
                        <add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
                        <add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
                    <action type="Rewrite" url="/" />

I hope that I haven't confused the issue with my explanation. Any help will be appreciated.




Well, it turns out that IIS 6 does not handle redirects. Redirects became possible with IIS 7. So, it boils down to upgrading our server, which for various reasons is highly suggestible, or some hack.

Thanks for looking at the question.

I can not select this as the answer until tomorrow.


Need Your Help

dynamically toggle chosen js between single and multiple input

javascript jquery html jquery-chosen

I am using chosen.js and I would like a select control which can dynamically be switched between multiple and single inputs. Is this possible?

How to make google maps API get the restaurants in the specified area

asp.net javascript google-maps

So far I use http://maps.google.com/maps/geo?q=downtown&amp;output=csv&amp;key=whateverthekeyis to get the longitude and lattitude of some address.