This is the second day in a row that we have entertained an excellent guest author. Today we welcome Andy Mitchell, who originally posted this interesting article on his blog 3 Dogs Bark.




One idea has been circling the back of my head for nigh-on a year, but I know I’ll never have time to act upon it. So it is shared here in the hope that some other talented type might run with it. Very briefly, it’s a browser extension that hijacks the searches you do at individual websites to provide highly focused results drawn from many sources; opening up an exciting new type of search.

The Foundations Summary

So, the foundations in a nutshell: verticals are useful but harder to use; unless you’re already at a site with a vertical search and want to dig a little deeper.

The Foundations

To lay the foundations, for anyone not familiar with the term, a ‘vertical search’ is simply a search within a given niche. An example could be a people search engine; or it may also be any search within a particular site (e.g. a search over the LonelyPlanet.com site would be in the travel vertical). The benefits are much more focused results. E.g. if you’re looking for a person, then a vertical can guarantee that no matter what the query, all results will be people (so I search ‘web development’, and I get a person who knows something about web development; not an article on web developing).

The drawback, however, is that they require one additional, difficult step. Rather than just entering you query into Google, you have to first find the correct vertical, learn to use it, then enter your query (and learn to use their results). Worse, a search engine for just one website can only return results from the very limited resources within that website. In contrast, Google would draw results from all sites about Russia. Vertical searches advantages are their precision, at the cost of broad results.

The most obvious answer to this is to create a Google-clone that has a UI dedicated to drilling into a niche, and indexes many individual verticals. This has been tried; but never succeeded. Theories abound, but I personally think it comes down to a two things. Few companies have the resources to build a useful Google clone, so it’s never particularly high quality. And more importantly, users simply do not want the hassle of ‘drilling down’; they’re happy for Google to give an easy overview and ’sift’ through the results, rather than do upfront work before even getting those results. They simply prefer instant gratification for minimal input.

However, if you’re already at a content-site that is giving you some useful information; and you just want to dig a little deeper into the topic; you will happily use their search tools (almost every site has some kind of search box). Going back to the Google example above, it simply requires minimal effort. While the results might be limited, they’re easily attained.

The Solution

Enter browser extensions. Browser extensions are powerful beasts; allowing you to directly modify any existing website.

The benefit is best described with a scenario:
You go to LonelyPlanet.com and type in ‘Cambodia’ to get travel tips for Cambodia. The extension hijacks the results, and instantly repeats the search at Footprint.com, RoughGuide.com and STATravel; neatly merging the results into a familiar UI. You are getting a super-simple vertical search that also pans across other searches in the same vertical.

How would this work? The key is understanding the nature of the vertical you’re searching. Something that is easily achieved with Delicious and Alexa. Using their APIs, you can retrieve the ‘tags’ most people use to describe that website. So, it knows LonelyPlanet.com is tagged ‘travel’, and can find other similarly tagged sites.

The Benefit

The advantages have already been covered above, you’re getting highly relevant results, from a wide range of resources, with minimal effort. A typical usage pattern might be to do a general search (’Cambodia’ at Google), make your way onto LonelyPlanet.com, and then dig deeper using LonelyPlanet.com as a powerful vertical search.

A particularly interesting view could be that it supercharges the search power of any website; i.e. ‘search this vertical‘ rather than ‘search this site‘. As this expands the usefulness of the site, many would want to deploy a proper widget to enable this kind of meta search for all users, not just those with the extension involved; thus spreading the userbase much faster.

Revenue? Please. You’ve got a search tool useful to the masses, people looking for things, people telling you what they’re looking for, and very precise context-knowledge – a near perfect match of the advertising system that works so well for Google; only in a new product area.

Wisdom of the Crowds

Why stop there?! What else is an extension able to tell you? It tells you which results a user clicks, and how long they stay at that result. In other words, it’s telling you which results are relevant to a given search term (or at a higher level, which additional verticals are preferred by a user, e.g. searching from LonelyPlanet.com, that most people prefer additional results from Footprint.com rather than Roughguide.com).

If the base idea is unique and does interesting things to the search field; this is even more powerful. Now not only are you doing precise searches drawing on a wide range of relevant results – and lets not forget doing them very easily – you’re able to filter those results based on what real people are saying is useful.

In that sense, it’s kind of playing in the direction of Wikia (see an overview at TechCrunch), providing a highly social element to search. Perhaps Wikia would even be a very good benefactor of this data.

The Long Term – a Google rival?

The data that you would build up, merging multiple verticals; knowing what keywords are linked to what verticals; and knowing which results get human approval for a given keyword/vertical; you could then be in a position to move beyond the browser extension world and create a real search engine. A search engine where you could go along, type in ‘Cambodia’, have the system say “this looks like a Travel search”, provide results taken from LonelyPlanet and FootPrint, and pre-filtered by other humans… that sounds like a massive leap forward to me.

Article by Andy Mitchell
(andy@productivefirefox.com)

Footnote

“Bumble Search” – the name of my first (now defunct) extension – would be an apt name for such a service; cross pollinating search engines everywhere.

* Update

Imran Ali riffed on this idea, and pointed out that explicit control over the meta search would also be useful. So, you could nominate the search engines you wish to include when searching a ‘travel’ site, or even when searching a specific site like LonelyPlanet.com. This would certainly be easier for someone to initially implement (and is actually closer to something we have in development, but don’t let that stop you as it’s unlikely to cross-over directly).

What do you think? Do you agree with Andy or beg to differ? Leave a comment!

This entry was posted on Friday, August 3rd, 2007 at 2:43 pm and is filed under Guest Authors. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.