Search is one of the most important user interface elements in any large website. As a rule of thumb, sites with more than about 200 pages should offer search. Guidelines for search include:
a search button on every page (Update: search should be shown as a box, not a link, according to my newer studies on search usability.)
global search (searching all of the site) is better than scoped search
boolean queries should be relegated to a secondary "advanced search" page
Our usability studies show that more than half of all users are search-dominant, about a fifth of the users are link-dominant, and the rest exhibit mixed behavior. The search-dominant users will usually go straight for the search button when they enter a website: they are not interested in looking around the site; they are task-focused and want to find specific information as fast as possible. In contrast, the link-dominant users prefer to follow the links around a site: even when they want to find specific information, they will initially try to get to it by following promising links from the home page. Only when they get hopelessly lost will link-dominant users admit defeat and use a search command. Mixed-behavior users switch between search and link-following, depending on what seems most promising to them at any given time but do not have an inherent preference.
Despite the primacy of search, webdesign still needs to grounded in a strong sense of structure and navigation support: all pages must make it clear where they fit in the larger scheme of the site. First, there is obviously a need to support those users who don't like search or who belong to the mixed-behavior group. Second, users who get to a page through search still need structure to understand the nature of the page relative to the rest of the site. They also need navigation to move around the site in the neighborhood of the page they found by searching: it is rare that a single page holds all the answers or even that the search found the most relevant page, so users need to see related pages.
Search should be easily available from every single page on the site. Search-dominant users will often click on a search button right on the home page, but other users may move around until they become lost. Once that happens, you don't want them to have to have to search for the search, so it should be right there on the page. This means any page, since you can't predict when users will give up navigating and look for the search button.
Sometimes special areas of a site are sufficiently coherent and distinct from the rest of the site that it makes sense to offer a scoped search: restricted to search that subsite only (the search scope). In general, I warn against scoped search since our observations have shown that users often don't understand the structure of sites. It is quite common for users to believe that the answer is in a wrong subsite, meaning that they will never find it in a scoped search. Other times, users don't realize where they are and the scope of their search, so they may think that they are searching the entire site or a different subsite than the one they are actually in.
In contemplating a scoped search option, designers should be strongly biased to avoid scoping. If the site in fact has subsites that necessitate scoped search, then all scoped search pages must do two things:
Explicitly state what scope is being searched. This should be indicated at the top of both the query page and the results page.
Include a link to the page that searches the entire site. Again, this link needs to be on both the query page and the results page: on the results page it should be encoded as a link saying something like "Didn't find what you were looking for? Try to extend your search to the entire Foo.com site." Following this latter link should activate the global search with the same query as used for the scoped search and should take the user directly to the results page for the expanded search.
Relegate Booleans to Advanced Search
Boolean search should be avoided since all experience shows that users cannot use it correctly. We have studied many groups of users who have been given tasks like this:
You have the following pets:
Find information about your pets.
Almost all users will enter the query
cats AND dogs. In our studies, they typically do not find anything, since the site never mentions both animals on the same page. Upon encountering a "no hits found" message, the vast majority of users concludes that there is no information available about these pets. Experienced programmers normally use the same erroneous query, but when they get the null result, they typically say "oh, yes, I should have used an OR instead of the AND."
Unfortunately, most users have not been taught debugging, so they are very poor at query reformulation. This is why I recommend minimal use of scoped search and no use of Boolean search in the primary search interface. Advanced search is fine if offered on a different page than the simple search. The advanced search page can provide a variety of fancy options, including Booleans, scopes, and various parametric searches (e.g., only find pages added or changed after a certain date). It is important to use an intimidating name like "advanced search" to scare off novice users from getting into the page and hurting themselves. Search is one of the few cases where I do recommend shaping the user's behavior by intimidation.
Search systems can be made more usable by incorporating spelling checks (both for user queries and for document terms), by offering synonym expansion, by explicitly recognizing the concept of quality in addition to relevance (update: the year after I wrote this guideline, this was the strategy chosen by Google) , and by presenting results relative to the structure of the site. For example, if the site has a FAQ about a query term, then the FAQ page should be listed on the top of the results page even if other pages have higher relevance scores. Also, hits on a series of pages that belong to the same area of the site should be collapsed into a single reference to that subsite.
On a final note, elementary schools should start teaching search skills. The future of user interfaces is almost certainly going to be dominated by various ways of searching immense information bases and gradually refining the retrieved set. Ideas like query reformulation, relevance feedback, and query-by-example are all important but do not come naturally to users. Search skills are likely to be more useful than most of the computer uses kids are currently taught.