(l-r) Panelists Sherry Hsi, Rob Rothfarb, Mick Khoo, and Mimi Recker

Why do people come to your website? What do they do there, and what could you be doing better? The 2007 NSDL Annual Meeting panel “Beyond Webmetrics” showed how to seek answers to those questions by gathering user data from a variety of sources.

San Francisco’s science museum, The Exploratorium , maintains a website that attracts 28 million unique visitors a year. “The main thing we want to know is what impact we are having on these visitors,” says Rob Rothfarb, director of web development for the museum. “A lot of tools are available, and they often give you conflicting information. You need to use a variety of tools to understand why people come to your site and what they do once they get there.”

The Exploratorium uses “quiet capture” tools such as Google Analytics to understand the geographic areas their visitors come from and the total number of visitors. But these kinds of sources don’t tell you why people come to a site, how they use it, how effectively it met their needs, how it could be improved, and whether or not the user learned any science, says Rothfarb. “Active elicitation” tools such as online questionnaires and requests for information can eventually answer these questions, once enough people participate in them. One Exploratorium page asked people to enter their Zip code, and 73 percent did; another asked for demographic and occupational information from teachers who wanted to download documents from training workshops, and since May 2005 nearly 3,200 responses have been received. “People give us their personal information because they see us as a trusted source that would never sell their name,” says Sherry Hsi, who conducts research and evaluation of the Exploratorium’s science education programs.

One of the Exploratorium’s most popular features was coverage of a total eclipse of the sun in March 2006, streamed live from Turkey in collaboration with NASA. The video, which is still a popular resource, includes a section where users can leave comments about the eclipse. Rothfarb says that most of the comments were thank yous, but many volunteered information about how they used the video in a teaching exercise. Rothfarb also created an event on the site SecondLife.com where about 70 people gathered in a virtual copy of a Turkish ampitheater from 200 A.D., the site where the real event was being filmed. “There was a great moment when the screen showed an ampitheater full of Turkish kids looking at the eclipse through sunglasses, and our SecondLife version gave avatars in the virtual ampitheater a look at the real people in the real ampitheater,” says Rothfarb.

The virtual guests watched the video of the real event on a screen together, and Exploratorium collected an hour’s worth of chat between the guests. Hsi is analyzing those transcripts now to see what the web guests learned. “People were helping each other solve the problems they were having understanding the event,” she says. “We realized that we were turning our visitors into exhibit developers.”

Instructional Architect
The second presenter, Mimi Recker, is the director of Instructional Architect (IA), a simple tool that allows teachers who are not sophisticated online users to post lessons and Internet links on a password-protected web page within the site. IA is much smaller than the Exploratorium, but it also uses webmetrics and more detailed information to evaluate the success of its teacher education workshops. The site had about 32,700 unique visitors between June 7 and November 6, according to Google Analytics, and the majority of that traffic came directly to the site; it was not a referral from a search engine such as Google. This shows that the major way people find IA is through word of mouth, says Recker.

In Google Analytics, the “bounce rate” is the proportion of visitors who leave a site after viewing just one page. Typically, a high bounce rate is seen as a bad thing. But IA’s bounce rate of 59 percent is a sign that the service is working the way it’s supposed to, says Recker. “We surmise that students are logging on and going to their assigned page, then clicking on the link to go to the assigned resource,” she says.

One of Instructional Architect’s main activities is holding workshops where teachers are shown how to use the service. “Teachers typically love the workshops, but we want to know what happens later,” she says. “So we began asking people who had registered at the workshops about their habits six months later. We found that about 10 to 20 percent of them actually had used IA when they got back to work. That sounds depressing, but we actually feel pretty good about it. Changing someone’s behavior is hard.”

Collecting meaningful data can be hard, tooo. Mick Khoo analyzed NSDL’s webmetrics until leaving the Core Integration staff for a new position several months ago. “It was a challenge,” he said, “because there were so many different servers who all provided information in different ways. The key to analyzing web traffic is getting your ducks in a row. “ This means establishing standard tools, definitions, and time scales. NSDL overcame a lack of resources, lack of staff time, lack of expertise, lack of access to servers, and unreliable external web developers, said Khoo; it now gets monthly web traffic reports that are consistent and easy to understand.

Big or small doesn’t matter, said Khoo: a site should collect information that contributes to its objectives. “It all depends on what you’re trying to do and who you’re trying to reach,” he said.