Why the new Google referer string is important for SEO

Google announced last week that as part of a new AJAX-powered Google search interface it would also be passing a new referer string, in the following format:

http://www.google.com/url?sa=t&source=web&ct=res&cd=7&url=http%3A%2F%2Fwww.example.com%2Fmypage.htm&ei=0SjdSa-1N5O8M_qW8dQN&rct=j&q=flowers&usg=AFQjCNHJXSUh7Vw7oubPaO3tZOzz-F-u_w&sig2=X8uCFh6IoPtnwmvGMULQfw

BlogStorm were the first to notice that this string had two new mystery variables, cd= and ct=. They speculated that this might be position information, and this was later confirmed by Google’s Matt Cutts in a comment on the same post.

This change will probably pass a large number of digital marketers by, but it is definitely not insignificant. Google have never willingly shared position data with webmasters before, and the only way to get keyword ranking data on a massive scale has been to scrape the Google search results. This is not only a violation of Google’s terms of service, but is also fairly ineffective – you could only get a limited number of data points for each keyword, and usually only at one particular time of day and from one location.

Everyone knows that Google has been making massive moves towards personalisation. Now we can understand much more precisely how personalisation and geodemographics affect our search engine positions, and get much more insight into how effective (or ineffective) our SEO spend really is. With multiple position points throughout the day on every keyword, and the ability to track position to traffic and even revenue, SEO reporting suddenly becomes quite a lot more accurate and a little bit more accountable.

A large amount of the SEO community might argue that its not rankings you should be concentrating on, but traffic – this is definitely true, but without one there can’t be the other. Hopefully it won’t be long before detailed position reports start appearing in the major analytics packages, and I know that certainly we’ll be looking at the data in as much detail as possible.

But why would Google share this data? They never have before. Personally I wonder if this is an effort to cut down on the amount of scraping that Google must see on a daily basis – certainly I imagine a significant portion of their daily processing power must be used up by SEO’ers looking for ranking data, and despite blocking the most obvious offenders, there are many ways to hide from Google’s “we think you are a robot” screen.

Unfortunately for Google (and us) I don’t think the SEO community are quite done with their scrapers yet. The advanced digital marketer is not just interested in his own position, but those of his competitors too, and there are no signs that Google might be sharing that data in the near future.