Abstract:

The core service of a modern digital library (Library 2.0; see Maness, 2006) is usually the portal for professional literature search. However, nowadays many students and researchers rely on the search by Google, because of the easy handling and the seemingly sufficient quality - even though they are aware of the shortcomings of Google. Thus, it is of special importance for a scientific Library 2.0 to assure a good usability as well as a good quality of their professional search portals. For the assessment of usability there exist several standardized scales, e.g., the System Usability Scale (SUS, Brooke, 1996). One big advantage of the SUS is the general formulation of the items which enables the application of the scale on very diverse software products or internet sites (including search portals). However, for literature search portals, the quality of the listed results of the literature search (hereinafter called “literature list”) is of even higher importance. Thereby, the assessment of the quality of literature lists is often complicated because of the special and partly unique features of the search portals of different libraries.

In this contribution a newly constructed Scale on the Quality of Literature Lists (SQuaLL) is described. The SQuaLL offers a quick and easy way to assess the quality of search portals and the accordingly literature list from the users’ perspective. The SQuaLL was originally developed with respect to the literature search portal EconBiz (www.econbiz.de), but can be easily modified for the assessment of other academic search portals of digital libraries.

The format of the scale was constructed analogous to the SUS, i.e., it comprises ten statements that have to be rated on a 5-point Likert scale. The items are based on an expert query. Each of the items refers to a specific feature of a search portal: Five of these items assessed the “must haves”, i.e., the basic core characteristics that every literature search portal should offer (e.g., a reasonable ranking and filtering). Four items regard to the “nice to have”, i.e., features that are part of most modern search portals because they are of high importance for the users (e.g., availability of full text download). One item refers to a rather special feature of EconBiz, namely the event search. This special item is formulated as such that it can be easily modified with respect to different specifics of other search portals, i.e., instead of “event search” one could asked for “multimedia search”. The global quality-index of the SQuaLL is conceptualized as a sum-score calculated analogous to the SUS. The possible values are between 0 and 100.

The practical use and the parallel test reliability of the SQuaLL were already proven for a controlled offline-sample. For the two parallel forms (A and B) of the SQuaLL we found a good parallel test correlation of r = .777 (p < .001). Additionally, also an online version of the SQuaLL was successfully applied.

To sum up, the SQuaLL offers a global quality-index of scientific literature search portals from the user perspective. The items of the scale can be easily modified regarding divers search portals of multiple scientific domains. The practical and theoretical implications will be discussed.

Keywords:

Library 2.0, digital library, search portals, evaluation, quality check, user perspective, information management.