Among the technically inclined, when I describe how our social search engine combines the best results of Google, Yahoo! and MSN, a common response can be summed simply:

Isn’t that SLOW?

As users of url.com know, our search responses are actually quite fast and usually just a tad slower than any of the top search engines. (ok, I’m biased here, do regular users think our search response times are acceptable? Please comment below)
One reason why metasearch engines are thought to be slow is because, for each search result, the engine has to grab results from the original search engine, process it and then route the results back to the client browser.

What is unique about url.com is that we employ a technology such that your browser actually grabs search results directly from the original search engines such as Google and Yahoo. We do this via ajax apis that Google and Yahoo provide.

This technical architecture also has the nice side benefit that it lightens our server load for each search request, which allow us to serve more requests per server

One of the biggest technical challenges of this novel architecture that we’ve chosen, is that we need to make sure each request to the original search engine is made in parallel, e.g. I don’t have to wait for Google results to come in before I make a request for Yahoo! results.

In technical parlance, we figured out a way to do issue such request in a non-blocking fashion in Javascript. I think this technique is applicable beyond url.com and will be happy to share how we do it in a later post.

Although we’ve put in quite a lot of work into figuring out how to speed up search results, we’re really just at the beginning. We have several experiments underway to make the results faster, and equally important, appear to be faster.

To that last point, we certainly can allow search results to be displayed as soon as we receive them, but we’re not sure if that would result in too much flicker.

Do you have ideas on how we can speed up search results? Or have a better UI to display the search results? Please share.