The Internet has moved from being one vast, disorganised library in which all the books were scattered on the floor to one in which the territory is becoming a bit more mapped out.

Flags are being planted, territories staked out in the forms of portals, hubs and home bases.

Voyagers navigate the realm through the search engines, metasearchers and search bots developed to scour cyberspace.

These launch pads are the tools for exploring the current estimated 5.4 million sites holding an estimated 500 million pages of information. The more narrow and obscure your search is, the more sophisticated you'll have to get in tracking down what you want.

The search engines are as good or as bad as your ability to use them. You have to pick your engine according to your needs. Keep all of them handy. What you don't find on one may pop up on another depending on the methods used to search and catalogue the Internet.

You should also be aware that some have sold their soul to advertising -- you may only get what the engine company has been paid to let you see. Rankings on generated lists are sometimes sold. Some smart companies have also found tricks to ensure their site shows up at the top of search terms.

Competition has, however, increased the number of search companies and these have become better at finding information faster. Most are free, some are charging quite extraordinary fees by Web standards. Web searching is the activity people do the most on the Internet and entrepreneurs are testing various models to figure out how to cash in on the trend.

The best fee-based service I've found is Electric Library, where for about $70 a year you get to search through articles from hundreds of magazines, newspapers and journals. You can even read the entire Economist magazine on the site if you're willing to wait two weeks past the issue date. Web searching is the number one activity on the Internet. The top search term is "sex'' followed by news and sports. At the very basic level of the navigation tools are the indexes or catalogues of sites, compiled in advance. Here you search within a general listing of sites chosen for their completeness.

Indexes are a useful starting point, but too many people set their homepage to start them off at a particular search page and never venture outside. They simply give up, or accept what the engine offers as the end of the Web. Still indexes may be all you really needed.

In the middle are the search engines that claim to search the entire Web. Some -- which will find people, businesses, jobs and addresses -- are specialised.

Yahoo.com, Go.com, Excite.com, Lycos.com, Altavista.com, Snap.com, Hotbot.com, Looksmart.com, Goto.com, and Webcrawler.com are (in order) the top five search engine sites according to Internet measurement service Media Metrix.

My personal favourite is HotBot because it's fast, thorough and deep. I've been using HotBot for over two years now and generally stick with it. Hotbot allows you to start with a general term. Once you've got a list you can then search through results to narrow down your choices. Many search engines now allow users to search through their generated results. AskJeeves has fast become a popular human-powered directory in which users can put in questions in normal English. Northern Light is also popular and allows you to organise your returns, and includes searches of 4,500 publications. For most sites you really have to know how to use the symbols and key words to tell the engines which words to include and which ones to exclude. That's key in searching -- read the sit's help directory.

Metasearchers, such as MetaCrawler, sit on top of the regular engines, allowing users to query multiple sites simultaneously. SavvySearch and AccuFind are other metacrawlers.

Bots (from robots) are the top level of the sophisticated tools for searching the Internet. A bot is a software tool for digging through data. You give a bot directions and it fetches the answers. Regular search engines themselves use bots which crawl from one server to another compiling the lists you'll eventually see when you use the engine.

On the Web, robots have taken on a new form of life. Since all Web servers are connected, robot-like software is the perfect way to perform the methodical searches needed to find information. Bots are being used to develop a new type of artificial intelligence software (more on that in the future). In relation to search engines news bots are available to fetch, and regularly keep fetching, information on specific topics from information sites. You can get a listing of the engines available at the BotSpot (bots.internet.com).

"Intelligent bots can make decisions based on past experiences, which will become an important tool for data miners trying to perfect complex searches that delve into billions of data points,'' according to the site.

To find out more details about the other search tools and how they work try The Mining Co., a great and growing site.

Check out websearch.miningco.com for a complete links to all the search engine sites -- and more -- cited in this column.