Guest Author Phil Bradley

Have you ever heard of Black Stump? How about TicoFun, Leit, Tanikalang Ginto or Kanoksearch? Don’t worry if you haven’t, but on the off chance that you have, I’d take a guess that you’re interested in Australia, Costa Rica, Iceland, the Philippines or Thailand respectively. They’re all search engines – the main difference between them and many of the others that you will have heard about are because they have a very specific focus, on a country or a region.

When I’m teaching internet search one of the questions that I’m often asked is ‘how can I find more country based information?’ Traditional search engines are generally not very good at providing this sort of information, and it can be tricky to create a good search strategy to pull it out from the likes of Google. My usual answer to that question is that I don’t bother with the traditional search engines – I’ll go straight to a country search engine.

Country based search engines are by and large one of the hidden secrets of Internet search. Of course, we’ve all heard of the likes of Yandex (Russia) or Baidu (China), but there are another 133 search engines that cover just those two countries alone! A country search engine will provide an insight into a country and its culture in a way that other search engines cannot even dream of. The majority of these engines are index or directory based, so the headings and subheadings really will reflect the interests and concerns of that particular country. For a searcher this provides invaluable information; what is the political situation in a country? Do they have a monarchy or a presidential system? What about the history or geography of a region? What about tourist attractions? Or doing business in a country? Simply by looking at the structure of a search engine the enquirer can very quickly get a good overview, and that’s even before he or she has started their search properly!

Of course, most of the sites that are listed in engines of this nature are going to be in that country, so the searcher has immediately circumnavigated all the thousands of irrelevant hits that the bigger engines will return, which of course in turn means a tighter and more focused search. The directory based structure allows users to quickly cast an eye over the sites listed, and this may alone highlight appropriate places to visit. A site will often act as a portal as well, providing immediate information on exchange rates, the weather, top news stories and so on – again in much more detail that you’ll find with more general engines. All of this isn’t to say that they’re perfect – far from it. Many of these directory search engines rely on website authors finding them and submitting their sites for inclusion, rather than the spidering approach taken by larger free text search engines such as Google. In most instances the search options are also limited, but equally a small set of results will be more manageable so it’s a question of six of one and half a dozen of the other.

Types of search engines

The vast majority of this type of search engine is that they are index or directory based. Clearly it’s very much easier to take this development approach, since it’s only necessary to decide on the headings, subheadings and so on as appropriate for each country and then wait for the submissions to come flooding in (or not, in some cases) rather than attempt to go out and spider the net in order to create a searchable database. Of course, there are problems inherent in this approach, particularly for the searcher. Results will never really be close to comprehensive and unless there is a stringent URL checking procedure in place link rot will be quick to set in. Since it’s usually the responsibility for a web author to approach a search engine to ask for inclusion I’m convinced that many excellent information resources are not included by these local or regional engines.

Consequently, in order to be certain that a comprehensive search has been run it’s necessary to backtrack to one of the many traditional free text engines, making one wonder if the excursion to a local country based search engine was nothing but a waste of time! Other disadvantages are that local advertising will often play a large part (with some search engines that I looked at, it was quite hard to find individual entries, buried as they were under a welter of business logos and banner advertisements) and there exists at least the possibility that some sites will get placement based entirely on a fee, while others will as a result be excluded. Another obvious problem is that many of these engines are in the local language, which if course isn’t a problem at all if you speak it, but a real handicap if you don’t. Many engines to their credit do try and provide English language versions of the pages, but not all do – and indeed why should they?

A small number of engines have gone down the route of the free text approach, allowing the searcher to simply type in the terms that they’re looking for, cross fingers and hope for a result. In my experience the searcher does need to be concerned at this point, as some engines will just automatically limit a search to sites that end their URL in a country code, failing to include sites with a TLD of .com for example. This is a risky strategy that doesn’t always pay off, but it’s something that may trip up an unwary searcher.

The good, bad and the ugly.

Clearly therefore there are problems with some of these engines, but to be fair even the very best search engines can be criticized for what they have or have not done. There are some excellent resources, and the best of these generally share one specific criteria – they cover lots of different countries. Examples are the Open Directory (DMOZ) and Yahoo! Directory regional engines. Coverage (both of country and subject) is usually excellent and a good starting point if you’re not sure exactly what you need – in any list of search engines if you see those mentioned – use them as an initial starting point; very often you won’t need to go any further. The [country_name]crawler.com series of search engines which are part of the Crawler network also give good information and act as a country based portal, with guides, quick links, popular searches, maps, toolbars and other services. The Buscapique collection is also similarly comprehensive in terms of subject and country coverage, although in most instances the interface is only available in the local language. At the other end of the spectrum in my experience as a searcher it’s best to avoid any of the ‘Allof[country_name]‘ engines. The ‘Allof’ brand used to provide a good quality service but seems to have gone out of business half way through 2005 and the domains are now very little other than placeholders full of country adverts, but without any useful content. It’s not unusual to find failed search engine URLs being purchased by pornographers, keen to make a short term profit based on the number of sites linking to the URL, so don’t be at all surprised to sometimes click on a perfectly innocent link only to be faced with pornographic links.

Development of these search engines.

I’ve been tracking this type of search engine for the last five years, and it’s fascinating. Overall the growth is certainly very encouraging; for example in 2005 the UK had about 164 search engines devoted either to it specifically, to particular regions or to specific subject interests such as curry. When I updated this list recently I found that in a 3 year period over 50 of these engines had simply disappeared; the URLs did not work or more commonly the domain name had been purchased by a speculator (this is perhaps the single most common fate of country search engines that fail; their name ends up rather sad and tattered in the Internet equivalent of a remainder shop), but sometimes a pretense would be made to keep the search engine going by providing links to a small number of rather dubious websites.

However, the upside has been that these 50 casualties have been more than adequately replaced by 63 new engines, leading to a total of almost 180 search engines. In almost every instance there has been an explosion of search engine coverage by country – it’s not unusual for a country to double or even triple its coverage in a two or three year period. Sometimes it’s fairly easy to guess why there is good coverage for a country – somewhere like the Bahamas has 20 engines, many of which get much of their income via tourist industry advertisements. Perhaps it’s time to update the old adage from ’sun, sand and sea’ to ’sun, sand, sea and search engines’! However, the promise of a relaxing holiday and sun tan doesn’t always result in a growth in the number of search engines – the Seychelles for example can only muster a paltry 4 engines.

Sometimes the number of search engines in a country can relate (at least in part) to a political situation. Taiwan is an example of this – it supports an astonishing 25 search engines, which is only about half of the number that you find for China. Any new or emerging country will see an increase in the number of engines available; Kosova currently has 5 which specifically cover it (Serbia meanwhile has 11), but I confidently expect that number to rise as confidence in the country increases, and its economic situation improves. On the other hand, contradictions abound, and limited democracy doesn’t necessarily mean limited search engines. Sudan boasts 12 engines, North Korea has 7, Myanmar has 6, Libya has 10, while Zimbabwe only has 5. At the other end of the scale, China has 50 engines and Russia has 85. The size of a country also has little bearing on the number of engines that focus on it. India has a population of roughly 1,130,000,000 supported by 95 engines while there are 137 Canadian search engines, but a population of only 33,000,000.

One might expect that the poorer the country, the less search engines cover it; poorer countries (usually) have poor communications infrastructures, little by way of a tourist trade and the development of internet search engines isn’t usually that high on the list of priorities. However, Tanzania has 13 search engines devoted to it, Niger has 8, Mali 12, Yemen 12, Haiti 10 and Somalia has 8. It’s certainly true that some less well developed countries do not support many engines; 5 for Tibet, Malawi also boasts 5, and there are also only 5 for Angola. It’s worth pointing out as well that many of these engines will not have been developed in country, but will be part of a much larger project with the Open Directory Project country pages and the Yahoo Regional Pages being high on the list.

Perhaps one can however be a little more confident in assuming that the more developed a country, the more search engines it will support? The UN Human Development Report 2007/2008 lists the following countries as the most developed in terms of life expectancy, education, literacy, standard of living and so on: Iceland, (which has 24 engines), Norway (34 engines), Australia (36 engines), Canada (as previously mentioned with 137 search engines), Ireland (52 Irish search engines), Sweden (with 24 Swedish engines). At last! Something which can be predicted with a reasonable level of confidence. Well, not exactly. The ‘top’ countries in terms of the number of search engines that concentrate on them is as follows:
United States 228 [12th]
United Kingdom 177 [16th]
Germany (106) [22nd]
India (95) [128th]
Russia (85) [67th]
France (75) [10th]
Spain(72) [13th]
Greece (54) [24th]
Italy (54) [20th]
Japan (53) [8th]

So only two countries, France and Japan are to be found on both lists! It would seem that in actual fact, it’s almost impossible to predict with any level of success how many search engines a country might expect to support. However, if you’re a brave soul, try to put these countries into order, largest number of search engines first: China, Ireland, Netherlands, New Zealand, Romania. The only clue that I’ll give you is that there’s only a difference of 4 between the top and bottom engines.

Further sources of information

Larger and more traditional search engines do have a part to play in the search process, and it would be disingenuous to pretend that they did not. Local versions of the major engines such as Google are always worth using, and in most instances are intelligent enough to work on the IP address, rather than the domain name. Exalead provides an excellent country and regional search option automatically, and Accoona is also very helpful when it comes to narrowing a search geographically. Ask is superb if you simply wish to check some basic facts on a country and I always find the quick facts/answers section prior to the main body of results is an excellent time saver.

There are several collections of country search engines available on the web, but the majority are limited to a small number of engines, or which include the same search engine several times for each language it is available in. My collection of country and regional search engines now lists over 4,000 search engines, with a coverage of well over 200 countries, areas and regions. I have also tried to enhance the collection with multi search engines, links to books, photographs and blog references to selected countries. Of course it’s not a complete list and search engines are always appearing and disappearing (indeed, much the same could almost be said for countries themselves!), but it will give you a good pointer in the right direction.

The general trend would appear to be clear; three years ago my listing was a little over 2,000 engines, so the situation would appear to be healthy in general for these engines, though the same may not always be said of individual engines. They perform a very useful function and are certainly worth checking out.

Addendum

Oh yes, in case you were still wondering about the order that those 5 search engines should go in, it’s as follows:
Romania (52)
Ireland (52)
New Zealand (51)
China (50)
Netherlands (48)

Phil Bradley is a UK based librarian and Internet Consultant who has specialized in Internet search for the last 15 years and is a well known writer and trainer, both in the UK and abroad; he’s been one of the Microsoft ‘Search Champs’ and also trains staff at the European Parliament. His website is at http://www.philb.com.

Are you interested in being a Guest Author?

1. Enter/Join the Forum.

2. Create a news article.

3. “Ask Charles” to post.

This entry was posted on Thursday, February 28th, 2008 at 3:56 pm and is filed under Guest Authors, Top 100. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.