Search log analysis

Discussion articles

  • Better search engine design: beyond algorithms
    "A useful search engine is more than a search algorithm. This article explains how to create a search query analysis tool, a best bets feature, and a basic controlled vocabulary".
    (Peter Van Dijck)

  • How search can help you understand your audience
    A short article on how looking at search logs can help understand the user's vocabulary and issues where search might not return the results they are expecting.
    (Martin Belam)

  • Intranet search reports
    "A range of statistics are typically gathered on intranet usage, but of these, search engine reports are by far the most useful. This briefing explores two key search engine reports that should be implemented on all intranets, and looks at how they can be used to improve the effectiveness of the site".
    (James Robertson - Step Two Designs)

  • Search log analysis and the long trail
    "Addressing the top 500 queries makes good sense because it could improve the user experience for 40% of those who search our sites. But what about all those other queries? There has been quite a lot of discussion recently about the value of the "long tail" in e-commerce and in many other contexts. I wonder if we're missing out on important opportunities by ignoring the long tail of search?"
    (Lou Rosenfeld)

Research papers

  • Understanding web searching and navigation patterns
    "We describe a model for log data of user search sessions obtained from a trail-based search and navigation documentation system. The model elicits interesting patterns that can be used to better understand Web user search and navigation behaviour. Our study shows that such log data reveals interesting patterns beyond the typical statistical query terms analysis".
    (Mazlita Mat-Hassan, Mark Levene - University of London)

Case studies

  • A Day in the Life of BBCi Search
    Since BBCi launched in November 2001, its improved search offering has been collecting data on the way that BBC website users search both the BBC's website, and through its homepage Websearch, the whole wide web. This article shows how the search team analyse their search logs in order to understand user search behaviour.

  • Analysing user behaviour: a case study
    "Data was analysed from the web log files generated from four web-enabled databases hosted by The National Archives, using an in-house word analysis program written in the Perl programming language, using dictionaries of personal and place names built in-house. The initial analysis concentrated on determining the frequency of keywords per search. Keyword frequency analysis revealed that since 2001, users have increased the number of terms they use in a search - there appears to have been a shift from single word searches to two word searches."
    (Chris Kutler, Ray Devaney)

Books and book reviews

Presentations