Jinfo: Distance Learning, ifs KnowledgeBank and Usability

Distance Learning, ifs KnowledgeBank and Usability
Jinfo Blog

31st March 2007

Item

Introduction

John McBurnie As access to the Internet becomes less expensive and more widely available, distance learning is becoming an increasingly efficient way of delivering learning programmes to a wider audience, both in terms of geography and non-traditional students. However, when working with a wide, remote audience of distance learners, providing easy-to-use learning tools is vital. This is why quality usability is needed when providing electronic information to distance learners.

The value of information decreases if users cannot find it easily. Just as a traditional library requires a system in which the correct books are easy to find, providing online content also requires a systematic approach.

These are rules ifs School of Finance learned when improving usability for learners while undergoing a redesign in August 2006. Ifs provides financial education via distance learning from GCSE level through to undergraduate degree level. One of the main delivery methods of electronic information is ifs KnowledgeBank, an electronic library and information service. This article discusses the experience of KnowledgeBank, specifically tracking learners' habits and usability testing.

Design and tracking users

As Internet usage and technology have become more advanced, users have become faster in their decision-making. One survey demonstrated that some Web users spend only 30 seconds on a homepage <http://www.useit.com/hotlist/spotlight.html>, suggesting that designers of online information systems have little time to leave an impression on their users.

Since websites first appeared, designers have analysed traffic on their sites. Early websites had simple Web-visit counters, while today, administrators use sophisticated usage-trend software.

KnowledgeBank uses statistical software that allows for examination of complex metrics, such as the most popular route users take to a certain item of content, or the average time users spend on certain pages. Tracking KnowledgeBank in this way has proven a powerful tool because analysing what users are looking at, and what route they are taking to access that content, enables more effective tailoring of the content available.

The ability to look at past user behaviour is the most useful way to predict future behaviour and tailor usability accordingly. For example, tracking user behaviour has revealed:

KnowledgeBank is more popular amongst Bachelor of Science (BSc) students than other students
Our most popular page is the electronic reading list Study Support.

As a result of this analysis, we redesigned static links on the front page so that BSc students are directed straight to the Study Support page, where they are able to access electronic books and journal articles within three clicks. However it was important these links were not construed as misleading 'advertising', hiding other important content on the site. Therefore links to other popular pages were included at the foot of the Study Support pages and other areas were made easily available via a navigation bar. Designers need to appreciate that users have goals, and 'find-ability' is an important criterion.

Bridging the gap

One disadvantage of providing online learning support to distance learners is the disconnect between user and provider. In a traditional library, users and librarians have the ability to have face-to-face discussions. Replicating this kind of relationship is difficult in a virtual distance-learning environment, and users seem more reticent to give feedback about virtual environments compared to brick-and-mortar environments. Usage-trend software can be a powerful tool; however, it becomes more potent when combined with direct feedback from users.

At KnowledgeBank, every opportunity possible is taken to talk directly with students. Students have the option to attend face-to-face workshops with their tutor. KnowledgeBank staff attend these workshops to receive feedback on usability and content. One consistent theme from these views was that students were unsure how the different areas of KnowledgeBank could help them. To address this feedback, the most popular links, with a short piece of text explaining exactly what the link provided, and how they helped students were provided on the homepage. By learning from feedback, as well as thinking through the links and navigation, we provided guidance to help users with their choices.

Initially links were given ambiguous titles such as Resources and Subject Gateways. The (incorrect) assumption underlying the initial design was that users would make the 'right' choice because they had been provided with a link to valuable information. However, designers cannot presume that the average visitor will have the same confidence in navigating websites.

Feedback and statistical analysis revealed that users respond well to a successful first click. Users do not like bouncing back and forth between different links on a page. In the KnowledgeBank context, our research revealed that users want to know if the option offered will help them to complete a specific task, for example passing an assignment.

Usability testing

KnowledgeBank did not go through initial formal usability testing. Colleagues who were not directly involved with the project informally tested it. Due to time constraints the testing only occurred at the end of the design project's life cycle. The testing involved providing five colleagues with a series of tasks. From this, a set of statistics was collated. For example, three out of the five subjects completed 70% of the tasks successfully with an average time taken of 1 minute 27 seconds.

Due to the resource constraints, testing did not affect the usability of KnowledgeBank when it was launched post-redesign. Although some minor problems were identified at the testing stage, there was little that could be done given the time schedule for developing the new design.

We identified that there was a great deal of information on the site in different structures for different groups of users. This posed the biggest challenge for providing a clear path to all information. As a result of testing, our marketing and user manuals were tailored to help the different groups. Importantly, when content is added to KnowledgeBank it is added within the current design parameters, so as to not confuse the disparate groups of users. If this were not the case, then there is the potential to forge ahead with new design decisions, which may bring their own usability problems.

Usability testing cannot reveal all the information needed to make usability perfect, but it is helpful to learn:

How users fulfil their information needs
How website design can be helpful (or not) to users.

If resources had been available, testing of KnowledgeBank would have occurred throughout the design process in a more organic fashion. For effective usability testing, designers ideally receive feedback at different phases of the project, analyse that feedback and then integrate it into the design process. An effective approach is to regard testing results as a body of knowledge to influence usability and design.

Asking the right questions

However, usability testing involves more than simply putting a group of people in front of a computer, and asking them to find a particular piece of information. One key aspect relates to the tasks set to be tested. One way to construct usability tasks is to construct them to see how users complete specific tasks, for example, 'find the link to the Lithuanian Central Bank via the Web Directory'. Another way to construct tasks is to take a more open-ended approach, for example, 'find some information on Baltic Central Banks', to see how users engage with the site to solve problems. If both approaches are used, designers will come to a more comprehensive understanding of how users interact with a website.

One element often overlooked is that the group of people testing will influence the process. Testing using only people who work for the organisation, or using friends and family will not produce a complete overview. Although more expensive, market research and employment agencies are able to provide respondents who will match a certain profile and can be used to 'blind test' the website's usability. When this research is carried out, it is vital that the participants do not know the name of the organisation being tested, or any other details that may affect their judgement.

Designers should remember that it is the website's usability that is being tested, not the subject matter nor indeed their competence. Those carrying out the testing should be made aware that any feedback they can offer is valuable, no matter how trivial it may seem to them. Usability testing should not be seen simply as part of a design process tick-list with the results of the testing filed away and not used. It is crucial that the results of usability testing are analysed and acted upon.

Conclusions

From the experiences of developing the usability of KnowledgeBank we at ifs School of Finance have learned that designers must focus on what user's are trying to accomplish when accessing content online.

Compiling and publishing information is one of the easier aspects of providing information via the Internet for distance learners. Delivering information in a user- friendly way is considerably more challenging. When initially specifying the site's information architecture, designers must look beyond the navigation and links and think more deeply about how the user is finding the information to accomplish their objectives.

Related FreePint links: