Missing Presumed Non-Existant

When you read a lot of search literature a great deal tends to be made about false positives. These are results that match your search query but not the expectations of the user.

These create noise; they distract from the useful search results and potentially push better matches further down the results page  – and we all know that users don’t tend to stray too far from the top results before giving up. [1]

But in some respects these are a lesser issue than another problem – the positive results  a search fails to bring back.

A false positive is a nuisance, but usually a user is able to identify these bad results and skip over them. They are identifiable issues.

But the failure to return a relevant item is more problematical because a user may not know if a failure to return a result is due to a failing in the search system (say poor tagging or a badly formulated search query) or the item simply doesn’t exist. The behaviour is exactly the same in both cases; the item isn’t in the search results.

The user then has a choice; adjust there search strategy to try and find a method that brings back the hoped for result – however that could turn out to be a lot of wasted effort if the results simply doesn’t exist.

Precision and Recall. Recall is a measure of the number of relevant results returned against the number of relevant results on the document set. In most cases, users will not know how many relevant results are in the document set.

However if a user has a mental picture of the size and breadth of the document set they may be able to make some assumptions about the results they have received.

For example, if I go to my small local bookshop I may not be surprised to not find an obscure academic textbook and so after a quick check in the relevant section abandon my search. However, if I fail to find a popular bestseller in a huge, flagship bookstore I am more likely to question whether I am simply looking in the wrong place.

The problem with websites is that it is frequently difficult to judge the size and breadth of the content they contain. A home page will frequently reduce the content to a manageable number of sectiosn for navigation…