The Greatest Debate: Algorithms vs. Interfaces
One week ago, Nitin Karandikar wrote this post defending the pursuit of a new algorithm to beat Google. It’s not too long, so I encourage you to please stop, go back, and read it before you go any further.
Okay, now let’s carefully examine his approach. Next week Kaila Colbin of VortexDNA will judge the merits of our two arguments and share some insights of her own.
We are all very familiar with the two pillars of this dichotomy. Nitin calls it Beauty or Brains. (Tricky, since Kaila has both.) It’s also been called steak or sizzle. Cake or the icing on the cake. Nitin was probably a Science major, I was Liberal Arts all the way. When I buy a car, I go by the body style, I never open the hood unless it’s smoking. Nitin probably changes his own oil. Yuk!
Can’t we both just get along?
No, we can’t. And Nitin and I agree on this point. It’s because a start up company has limited resources, and they have decide which side of this fork in the road to take. Just imagine that Nitin works for me. He can only work on a new algorithm or a new UI, he can’t do both.
Now Google has the most sophisticated Algorithm, the fastest crawlers, the largest Index, the brightest engineers (note: I said engineers), and the most money of any Search company on the planet. This, their strength, is almost incalculably ahead of the pack. Nitin argues that if another company starts off with a different architecture, a new technology, Google will not be able to retro fit theirs successfully.
I want to be clear here; it is true that Google is an Ocean Liner and start-ups are the speedboats. The speedboats can turn and maneuver so much faster than the giant ship. But as I look at hundreds of search engines, I think that means that they can bring an awesome new Interface to market faster than Google can or will change theirs. After all, isn’t that why we have The Top 100 Alternative Search Engines? 100 unique improvements to the major search engines, with little to no response, except to buy the ones that show promise. And each time that they do, two more spring up to take their place.
I think that Google’s greatest weakness is their bare bones interface. Just consider (now, please, we’ll wait) my post about the Top 10 ways to visualize search results. Yes, it was posted after Nitin’s post, but he knows they exist. Those interfaces are beautiful, they are engaging. They are, “the most wonderful search engines you’ve never seen.”
An example: use Yahoo! to choose a place to go for lunch. And make it give you a random choice. Good luck! Here is mine. And by the way, it spins, and goes clickety-clack. Which is more fun? Since it sits atop the Yahoo! local search results, the quality of the results is the same. Just a different Interface.
Which brings me to my next point. If I can work out a deal to put my dazzling new Interface atop a decent search engine, why should I pay Nitin to try and build a better algorithm? Keep in mind the immeasurable resources those other companies have invested in their product.
I drive a Skoda, the national car of the Czech Republic, and the only one in North America. (true fact). Ten years ago, VW bought the whole company, and to be efficient, they put the Skoda body on the Jetta/Passat chassis and engine. So, when I searched for a car, I went by what I could see, by the curves and the cool logo. The thrill of having the only one. (People routinely stop me in parking lots.) What do I care about what’s unseen? It’s a flawless VW engine for goodness sake.
I think Nitin would say, buy a Jetta, and then try to build a better frame and engine than one of the top German automakers with decades of experience. But he talks about a better type of engine. That’s like saying it might run on french fry oil. Then put it up for sale because VW would never be able to adapt. I think they could.
Look, we have (in my post) Quintura tag clouds, KoolTorch circles, KartOO maps, 3-D SpaceTime results (I could end this post right here.), and Tagnautica circles.
I remember standing in front of a crowd of people and showing them traditional search results, then Ms. Dewey. Ten text snippets, Ms. Dewey. Same results (powered by MSN), but what an Inter-face!
Come on Kaila, don’t worry about hurting Nitin’s feelings, he can take it. You know and I know that Google is so literally crowded with engineers and PhD’s that it can’t see the forest for the trees. Tweaking code is like hiring a thousand people to look at a thousand trees with a thousand magnifying glasses.
The graphic artists that are developing these colorful, wonderful, attractive interfaces are standing waayy back and designing the homepages that will someday lead us to the Universal/Unifying Interface, the One Page, and the underlying algorithm will be just the “engine” that we can swap out at anytime. So long as the car runs, right?
Kaila?








September 21st, 2007 at 10:25 pm
Very well written, Charles! You’ve got me convinced - well, almost!
You make some great points. The dazzling, attractive new user interfaces that the Alts are creating are indeed unique, creative and valuable, and likely to draw attention. But, to use your own argument, if one or more of these user interfaces really takes off and starts drawing users away, how long would it take the Google hordes to reproduce it? A Google team could retrofit a cool new UI on top (possibly under a different domain) just as easily as a third-party can, once they know what they’re shooting for. If you want an example, I give you SearchMash - a Google property.
On the other hand, changing the underlying search algorithm - say based on Semantic Search or NLP - is a great deal harder, IMHO.
However, I must admit that I’m biased - being a software developer at heart, and “art-challenged” to boot! [Until my wife started buying clothes for me, my idea of variety was various shades of gray ...]
I can’t wait to see what Kaila has to say!
September 23rd, 2007 at 1:18 am
[...] did a debate this week on Algorithms vs. Interfaces, a hot topic in search blogs right now. ASE editor Charles Knight says that Google has the most [...]
September 23rd, 2007 at 2:55 am
[...] unknown wrote an interesting post today!.Here’s a quick excerptOne week ago, Nitin Karandikar wrote this post defending the pursuit of a new algorithm to beat Google. It’s not too long, so I encourage you to please stop, go back, and read it before you go any further. … [...]
September 23rd, 2007 at 1:51 pm
NK,
Truthfully, I think Google could replicate almost anything they want to
replicate.
They’re into space exploration for Gosh sake! The $65,000 question is,
what can be done about it?
If all of the best Vertical Alts combine to take money away from Google,
Google can’t pick them off or re-create dozens of the top verticals.
This has always been my thought: “Collaborate or Perish.” *Together*
they would be too much to copy. And then add a new Algo, and they’re
in a fix.
I do feel that they have a “blind spot” which is exactly why Ford, GM &
Chrysler are now such miserable failures.
CK
September 24th, 2007 at 1:31 am
[...] did a debate this week on Algorithms vs. Interfaces, a hot topic in search blogs right now. ASE editor Charles Knight says that Google has the most [...]
September 25th, 2007 at 5:40 pm
It is not a new interface or a new algorithm that can effectively compete with Google; it is where you search and how you crawl. HTML code was not invented so it can be crawled effectively by search engines; it is just so primitive and inefficient. Structured XML data such as RSS feeds is where we can start to effectively compete with Google. Google is as new to this area as other start-ups. Still new start-ups have to collaborate and innovate constantly to stay ahead of curve.
Even Google admits that it is not able to efficiently crawl and index this huge amount of data that is constantly changing on Internet. Structured data is fundamentally scalable, efficient, self-explanatory and platform independent. It offers the basis for better search algorithms and AI applications and its adaptation to custom built interfaces is just extraordinary. BH
September 25th, 2007 at 7:27 pm
Babak,
Thank you for that comment. You’re right, we must be willing to look in all directions.
Of the “Top 100″ Alternative Search Engines, who do you think is on the right track?
Charles
September 26th, 2007 at 8:48 am
@Babak: You’re absolutely right - HTML is really focused on text and display markup, so it’s a very inefficient way for search engines to gather information from the web. Gathering structured data like RSS (or even from semantic processing of regular web pages, if it can be done) will be far more powerful! It would allow regular search users to write parameterized, database-like queries.
I must disagree with you, however, on the importance of the size of the crawled index. I can give you counter-examples of search engines which work well with far smaller indexes than that of Google. In the end, the user only really cares about finding the few, *relevant* answers to her query (and I don’t believe those are directly correlated).
Counter-examples include: Mahalo, which explicitly addresses only the top 10K search queries, yet has gotten significant traction AND could easily be considered a superlative search engine for those common queries. Similarly, vertical search engines, by definition, have dramatically smaller indexes, yet can be both popular and highly effective.