You can use all search engines all the time
One of the Web's dirty little secrets is that search engines aren't very thorough in their coverage. The largest covers less than 30 percent of the Web. Even worse, there's not a lot of overlap in what the various engines do index. So, which search engine should you use? Because it is not clear which search engine is best, many of us got into the habit checking several different search engines to ensure that we didn't miss anything by using just one. The result of this behaviour was that Web developers saw a market for a "metasearch'' engine and set about creating one.
Don't be put off by the word "metasearch''. Metasearching has nothing to do with anything like metaphysics, except for using the same prefix. In computer science meta is a prefix that means "about'', Metadata is data that describes other data, for example what format some data is stored in -- data about data.
A metafile is a file that contains other files. The HTML meta tag is used to describe the contents of a Web page. A metasearch tells you what other search engines say about your query.
A metasearch engine searches the major search engines like Yahoo, AltaVista, and Lycos, simultaneously. When you use a metasearch engine, you cast a much wider net for your query, and increase the likelihood of quickly snaring an elusive page.
Unlike regular search engines, metasearch engines don't crawl the Web themselves to build listings of Web sites. Instead, they search several other search engines simultaneously -- they send a search request to several search engines all at once. The results are then blended together on to one page.
This lets you see at a glance which particular search engine returned the best results for a query without having to search each one individually.
Metasearch engines can be a real time-saver if you're looking for results in a hurry and you do not know which search engine to use. But as a famous philosopher once said, each solution spawns new problems needing new solutions. Metasearch engines are no panacea for the ills of their underlying search engines -- Metasearch engines have significant drawbacks, too: No search engine or metasearch engine is perfect. If an irrelevant, outdated, or even "spam'' result has slithered its way into a top listing at even one search engine, it will be in a metasearch result also.
Metasearches are quick and dirty. All metasearch engines spend only a short time in each search engine database and are subject to time-outs when search processing takes too long. This means metasearch engines only retrieve the top 10 percent of any results from each search engine, so that the total number of unique links may be considerably fewer than results found by doing a direct search on one of the search engines. This is not necessarily a drawback if you do not mind just finding the most popular pages and want to scan the results quickly, and do not have time to go through 20 pages of search results.
If you use Northern Light, you're out of luck. Most metasearch engines can't search it.
Advanced search features on individual search engines are not usually available. Phrase and Boolean searching may not be properly processed even if it is available. This may not be a problem for you if you never use the advanced search results.
There are two types of metasearchers: Software programs that you download and sit on your desktop, and Web sites that work like traditional search engines.
Personally, I don't think anyone likes add-ons that they must download and may work separately from their browser. I like the simple life -- surfing with just my Web browser.
Search plenty of search engines at once If you are like me, memory management on your computer is a problem. The desktop on my PC has enough icons on it to stretch along the Great Wall of China, each one just taking up more RAM, and laughing at me every time my system crashes. And I have 128 megs of RAM! I do not want to download any more software, so this article only deals with metasearch Web sites, or "metacrawlers'', not with metasearch software utilities like the award-winning Copernic that you can download from their Web site. I will just stick with searching the traditional way, where you go to a Web site and type in a search term.
Useful Metasearch Web Sites Below are a few of the especially well-known, well-used or long-established free metacrawler Web site.
Ixquick: www.ixquick.com Rated as one of the best by U.C. Berkeley and ZDNet, Ixquick searches 14 search engines and directories, including old favourites, plus recent additions such as Direct Hit, Open Directory, Google, and Goto.com. You can customise Ixquick to select just the search engines you want to check. But that's just the beginning. The clever twist is Ixquick's unique "star rating'' system for reporting metasearch results. It only reports top ten results from each search engine, assigning a single star when a search result makes it into the top ten. In other words, if a particular result receives a No. 1 at Excite and No. 2 at Lycos, but much lower results everywhere else, it gets two stars, but another result that was ranked No. 6 at Yahoo, No. 5 at Google, No. 9 at Excite, and No. 3 at Altavista, will receive four stars.
Currently Ixquick is one of the few metasearch tools that supports regular searches, natural language searches, and advanced Boolean searches, and knows which engines can handle which types of searches. If a page is listed in more than one search engine, Ixquick tells you which engines and how it was ranked.
Ixquick lets you search through Usenet, News, Audio (MP3) and picture databases.
ProFusion: www.profusion.com Search technology developer Intelliseek operates this ProFusion metasearch site that offers interesting ways of choosing which databases to search ("Fastest 3'', "Best 3'' as well as a pick-and-choose approach). Also, ProFusion has broken-link detection and search tracking. You can select which of the 14 search engines it will search, including Excite, Britannica, LookSmart, Lycos, MSN, and Netscape. It also allows you to access the Invisible Web by steering you towards category searches of "hidden'' content.
DogPile: www.dogpile.com Dogpile was one of the first metasearch engines and has a lot of name recognition. It displays results from each of the 16 search engines it scours individually. You can customise Dogpile to exclude and change the order of the databases it searches. I recommend that you customise Dogpile to send it to Google, Infoseek, and AltaVista first. It also provides links to more results from each search engine database. When you search on Dogpile you will get a long, though easily readable list of sites. The results are displayed in lists of ten links from each search engine it queries and are not grouped into one list like Ixquick and ProFusion. This sequential mode of searching may seem cumbersome, and it will include duplicate links if more than one search engine lists the same site. Dogpile also offers other searches for Usenet, FTP, Audio (MP3) and Business News. Dogpile is owned by Go2Net, which also owns the metasearch engine, MetaCrawler.
Ask Jeeves: www.ask.com Ask Jeeves is a human-powered directory like Yahoo, but if it cannot find the answer to your question, its metacrawler software will. If it fails to find a match within its own database, then it will display matching Web pages from various search engines.
There is more than one way to search the Web. There are so many kinds of search engines, specialised, topic-specific search engines, search engines searching databases on the invisible Web, search engines using software spiders that automatically "crawl'' through the links on the Web to create lists of sites, search engines that use humans to create lists of Web sites, and metasearch engines that crawl the search engines. Which one should you use? The best search engine to use depends on what you are looking for.
If you only have a vague idea of what you are looking for, or if you want to learn about something, try a human-powered directory like Yahoo! For more difficult searches -- whenever you retrieve a huge list of links and you want to focus on some specific aspect -- use a single search engine where you can search within results on a term or phrase you specify. If you do not know of a specialised search engine for your topic, try using Alta Vista Advanced Search and Northern Light Power Search.
Metasearch engines are useful if you are looking for a unique phrase or if you simply want to test run a couple of keywords to see if they bring up what you want or if you are in a hurry. They are a good place to begin in your hunt for information.
There is no one-size-fits-all way to find what you're looking for on the Web even though metasearch engines almost succeed in being all things to all searchers.
Michelle Swartz's column appears on the first and third Wednesday of every month in The Royal Gazette's Personal Technology and on the Royal Gazette's website, www.accessbda.bm. Questions and comments can be directed to michelle ychristers.net