Moving a Data Center - Big Project, and Big Results...

It has been almost 10 years since we first moved into Ask's original datacenter. Back then, a copy of the entire Internet fit in just a couple of hundred machines. We could be confident that even with the wildest Internet growth predictions, the vast expanses of emptiness of a datacenter would never be consumed.

 

It turned out that this prediction was indeed correct, but we realized a couple of years back that our ability to satisfy expansion requirements was not going to be limited by physical space, but instead by the availability of electricity. Limited power meant a cap on the number of servers, which restricted our ability to grow the search index. Our engineers were going to have to spend more time on improving efficiency, and less time on new feature development. So we've been in the planning stages for moving to new datacenter spaces since 2006, and we've executed the move of thousands of systems over the past several months.

 

If you are a web site owner who watches your access logs like a hawk, you most likely noticed that the Ask crawler has not been as active crawling your site. We curtailed our crawling after making the very difficult decision to reduce the search index size and freshness during the move in order to limit risks to our live traffic. We tried to do this in the smartest way possible, so that the impact on users was minimal. The internal metrics that we watch indicated that we met our goals. A few commentators noticed the reduced crawling level and speculated that this was an indication that Ask was in the process of exiting the search engine business. To the contrary, this quiet period was a necessary component of preparing Ask for future expansion and improving the overall search experience for our users. Now that the move is mostly complete, you should start to see Ask becoming more active in your logs.

 

We're excited about our new infrastructure position because it supports new and innovative core search technology initiatives, and allows us to continually improve the vital basics such as speed, freshness and relevancy. The direct benefits of the new data center spaces are:

 

 o The new locations are strategically positioned, resulting in an overall better response time for our users.
 o Reliability is higher, given that we're now operating well below peak availability limits.
 o Our new datacenter partners are invested in our success, and consequently are more responsive to change requests.
 o Electricity is cheaper and much more plentiful.
 o More of our power is provided by renewable resources.

 

I hope you'll keep an eye on what we're doing, and watch this space for more technology and infrastructure updates.

 

- Keith Hogan, VP Technology