JCDL 2007 is being held at The Westin Bayshore on Coal Harbor in Vancouver.

The Joint Conference on Digital Libraries 2007 got underway with blue skies overhead as sea planes taxied along Coal Harbor on the edge of Stanley Park in Vancouver, British Columbia. Daniel M. Russell, Uber Tech Lead for Search Quality and User Happiness at Google gave the first conference plenary lecture entitled What are They Thinking? Searching for the Mind of the Searcher on June 20.

While emphasizing that user happiness is well-defined at Google, he likened his role to that of a master parapsychologist–or mind reader. “The information signal is sometimes weak,” said Russell. Poor and incomplete data, multiple meanings for terms, “out of library” information requests, coupled with selective user attention makes it challenging to gauge what users are thinking and what they really want–even for Google. “We walk the valley between low attention and low signal,” Russell explained.

Google has developed a three-level muti-perspective approach to getting inside of users’ search habits. At the lowest level of observation Google has found that results from eye tracking studies don’t necessarily relate to user behavior in logs. Russell cited eye tracking research by Laura Granka, Helene Hembrooke, and Geri Gay at Google and Cornell as he reviewed some of Google’s experiments into determining how users perceive and interact with content on web pages.

Dan Russell, Google

Mid-level observations consist of interviews in places were people do their work which can vary from busy households to quiet offices. In high-level studies millions of observations are tracked where useful adjunct feedback such as, “Advanced search pages are most often described as ‘scary’ by users” are documented.

Ultimately Russell explained that Google users are often surprised by what is returned. “We find that they are rechecking their queries because they can’t believe they are getting the results they are getting—they think there is something wrong with what they asked.”

Russell suggests that users need to have fairly deep knowledge about what sites are possible, and what might be found in a given site to be good Web researchers. This knowledge is particularly significant when performing for two distinct types of search tasks:

1. Looking for direct information such as finding a specific painting by an artist.

2. Locating information such as retrieving complicated information or instructions about something.

Russell believes that most people do not have a mental model of how search works or what they need to “drive the search engine vehicle” in forming successful queries. When subjects are asked if they understand how search works they often reply, “What do you mean by ‘work’?”

A lack of basic understanding of how a search operates puts Google in the situation of “supporting magic.” “To Google” is now a popular culture verb that has started to change conversations. 24/7 search available on hand helds in public places as well as in homes and schools makes winning arguments with friends built on shakey facts far less likely.

Russell called for a new kind of literacy by reviewing four kinds of knowledge and skills that are needed to search effectively while reminding the audience that this boils down to teaching the general public how to perform effective research:

1. Pure engine knowledge—just the basics—enables searches at all levels
2. Information mapping—dictionary keyword frequencies, understanding contents of domains like Wikipedia—speeds searches
3. Domain knowledge—some intrinsic knowledge about what is being searched for like specific medical knowledge

4. Search strategy—knowing when to shift strategies in pursuing a query string.

Educating users in broadly effective models of research, content creation and organization in an evolving and chaotic mediascape is a significant challenge in creating a society that will be able to take advantage of increased access to knowledge.

As part of the nation’s educational cyberinfrastructure the National Science Digital Library can help meet this challenge.