E - IRS Query and Evaluation

Design, query, and evaluate information retrieval systems.

Introduction and Explication

“The objective of an information retrieval system is to enable users to find relevant information from an organized collection of documents…[and] make the right information available to the right user” (Chowdhury, 2018, p. 93; 98). On a daily basis, the work of the Digital Information Age professional involves searching for and retrieving unstructured bibliographic information that effectively meet user needs. Stored in databases, information is collected, organized, and retrieved in a timely fashion by an information retrieval system in response to queries posited via the system’s user interface (Chowdhury, 2018, p. 94; 98). Information retrieval systems (IRS) are capable of retrieving information of all types and in all formats. Functions include identification and analysis of information sources in the context of a user query; collection, organization, aggregation, and discrimination of those sources; and retrieval and representation of relevant sources matching the query (Chowdhury, 2018, p. 96). Examples of information retrieval systems include online public access catalogs (OPAC) such as WorldCat, online bibliographic research platforms such as EBSCOhost and JSTOR, or general search engines like Google.

Lifecycle System Design

Defined as an iterative and lifecycle-wide process, IRS design is a key responsibility of the library and information science profession (Weedman, 2018, p. 112). Effective design is a user-centered, solution-oriented process which “gives primary emphasis to the convenience of users” (Chowdhury, 2018, p. 102; Weedman, 2018, p. 120). As such, the first step in information retrieval system design is to develop an evidence-based understanding of the users the system will serve, their information problems, and how they will use the system (Weedman, 2018, p. 115). Once this context is established, the system designer develops a system concept and seeks stakeholder feedback on that concept (Weedman, 2018, p. 116). Following confirmation of the concept, the designer creates a mock up of the system interface and functional components, once again seeking stakeholder feedback before proceeding to the prototype implementation phase. A prototype is implemented so as to test for system or user problems (Weedman, 2018, p. 116). Once any immediate issues are resolved, the design is implemented, deployed, used, regularly evaluated, and adjusted as necessary (Weedman, 2018, p. 117). 

The information retrieval system’s search functionality is underpinned by a metadata scheme/data structure which organizes and defines the system’s searchable terms (Weedman, 2021, p. 122). Indexing is the application of a controlled vocabulary to metadata elements (searchable terms) in the database and is necessary for the system to function properly.

Query

IRSs facilitate communication between users seeking relevant information pertaining to a problem and a database pre-supposed to have access to information solutions. However, as Cleveland & Cleveland (2013, p. 91), “an information retrieval system is only as good as its indexing”. Bibliographic control supports users’ intellectual access to information and is maintained by the development of a machine-readable bibliographic record for each item (Reitz, n.d.-a). The bibliographic record contains all descriptive elements assigned by the information professional through subject analysis, description, classification, and indexing (Cleveland & Cleveland, 2013, p. 63). 

To avoid ambiguity in a system’s information aggregation and discrimination processes, information professionals utilize vocabulary control to facilitate subject-specific resource identification (Cleveland & Cleveland, 2013, p. 63). The Library of Congress Subject Heading Authority (LCSH) is an example of a controlled vocabulary. The controlled vocabulary is used for object and concept classification which is the arrangement of information resources hierarchically using designating tags to divide them into topical classes, subclasses, and sub-subclasses based on common or distinguishing characteristics (Cleveland & Cleveland, 2013, p.67; Reitz, n.d.-b). Indexing is an alphabetical list of subject headings through assignment of content and location indicators (Reitz, n.d.-c). It is via the index that users are guided through the search/query process. Metadata is standards-based structured data about a resource that facilitates information exchange and system interoperability (Cleveland & Cleveland, 2013, pp. 78-79). 

IRSs tag user queries with index terms and match those terms to similarly tagged resources (Cleveland & Cleveland, 2013, p. 88-89). The use of Boolean operators is a common strategy for effective query development in IRS (Cleveland & Cleveland, 2013,  pp. 88). While numerous Boolean operators exist, AND, OR, and NOT are most commonly employed by searchers. Regardless of organizational setting, effective information professionals are well-versed in the application of Boolean operators when searching any information retrieval system.

Evaluate

Evaluation of an information retrieval system’s performance and relevance is a critical and iterative process centered around users’ experiences when interacting with the search interface (Tucker, 2021, pp. 349–357). Key aspects of system evaluation include resource findability, usability, and relevance. Appropriateness of retrieved sources is based on the user's judgement of the retrieved items and may be affected by their lack of skill at crafting a functional query, a factor influenced by a user’s social and cultural contexts and practices (Chowdhury, 2018, p. 97). 

Search effectiveness is measured using the concepts of precision, recall, effectiveness, and relevance (Tucker, 2021, p. 353). Information professionals approach the evaluation of search effectiveness from the user’s point of view to ensure that the interface has acceptable visibility (e.g., actionable options), a consistent conceptual model, useful mappings of relationship between actions and outcomes, and system feedback on outcomes (Tucker, 2021, p. 357). Documentation of system audits and search interface evaluations should be maintained for the purposes of transparency, accountability, and future system related decision-making. Further, regular evaluation and comparison of competing systems should be conducted when determining if a different system is more appropriate for an organization’s stakeholders (Weedman, 2018, p. 6). 

Evidence

Evidence 1:    Find your Wine: Beta Prototype Database Design Project Links to an external site.

 

Find your Wine: Beta Prototype Database Design Project is a collective work product co-created by myself and four classmates for Information Retrieval and System Design (INFO 202, F22). The project’s purpose was to design a functional database prototype for consumers to use when searching online for a wine that best meets their needs. This was accomplished by identifying a user information problem, clarifying a purpose for the database, organizing a collection of relevant information objects, researching target users, and crafting searchable field values and rules. My project roles included co-research and co- developing the data structure, object identification and research, co-developing a controlled vocabulary, collaboratively crafting field rules, content creation, database testing/evaluation, and organizing and editing the final report. 

This document represents my mastery of Competency E capacity to implement the phases of IRS development to create a populated and usable database for effective information searching and retrieval.   

Evidence 2:    Metadata Scheme: Women’s Army Corps Digital Repository Links to an external site.

Metadata Scheme: Women’s Army Corps Digital Repository is a locally modified Dublin Core element set created for a collaborative group project in Enterprise Content Management and Digital Preservation (MARA 283, S21). The final goal of the project was to produce a functional trustworthy digital repository in Preservica using WordPress as the search interface. Each project team member was responsible for the identification, description, and uploading of objects into Preservica based on a collectively-researched and agreed upon controlled vocabulary, and classification system. 

Formatted in XML, the scheme facilitates access to viable information resources in the collection that are appropriate to user needs, and which support aggregation, disambiguation, findability, usability, and relevance. Applying a controlled vocabulary and classification system created prior to the creation of this scheme, a machine-readable bibliographic record was generated for each item in the collection so as to facilitate indexing, relationship mapping, linked data, and the identification, organization, retrieval, and representation of collection items that are appropriate, accurate, and usable for the repository’s target audience. This document acts as evidence of my mastery of Competency E as it represents my knowledge and skills in metadata scheme creation and implementation for a functional digital repository. 

Evidence 3:   Site Map Evaluation:  IMDb / Shawshank Redemption Links to an external site.

The sitemap evaluation project, IMDb / Shawshank Redemption was created for a group project in Information Retrieval and System Design (INFO 202, F22). The project’s overarching purpose included the following activities: develop a contextual, user-focused understanding of the mission and purpose of database and associated search interface; streamline the interface to more efficiently satisfy users’ information needs; increase usability and relevance; and draw more users to the site.  Each team member evaluated the sitemap based on its functionality, navigability, self-evidence, redundancy, and organization. These evaluations were discussed and compiled into a set of five specific recommendations. Additionally, the team assessed the user’s sense of its appropriateness, relevance, satisfaction, usability, and sense of engagement with the interface and the information retrieved as a result of searching or browsing. A proposed reworking of the sitemap based on these criteria was presented. 

This project represents my mastery of Competency E by articulating my knowledge and skills related to the comprehensive evaluation of an existing consumer-oriented database sitemap using the criteria required for an effective and efficient information retrieval system. 

Conclusion

In the Digital Information Age, IRSs are ubiquitous and practical systems with which people interact on an everyday basis to retrieve relevant information and satisfy information needs. Designing, testing, using, and evaluating information retrieval systems are fundamental knowledge and skill areas for information professionals. My experience and abilities in this area will continue to inform my professional activities as I move forward into the field of information science. 

References

Chowdhury, G.G. (2018). Basic concepts of information retrieval systems. In V. M Tucker (Ed.), Information retrieval system design: Principles & practice (6.1 ed., pp. 93-117). Academic Pub.  

Cleveland, D. B., & Cleveland, A. D. (2013). Introduction to indexing and abstracting (Fourth, pp. 61–95). Libraries Unlimited.

Reitz, J. M. (n.d.-a). bibliographic record. In ABC CLIO ODLIS. Libraries Unlimited. https://odlis.abc-clio.com/odlis_b.html Links to an external site.

Reitz, J. M. (n.d.-b). classification. In ABC CLIO ODLIS. Libraries Unlimited. https://odlis.abc-clio.com/odlis_c.html Links to an external site.

Reitz, J. M. (n.d.-c). index. In ABC CLIO ODLIS. Libraries Unlimited. https://odlis.abc-clio.com/odlis_i.html Links to an external site.

Tucker, V. M. (2021). Information retrieval system design: Principles & practice (6.1 ed., pp. 349–357). SharedBook Inc.; AcademicPub.

Weedman, J. (2018). Information retrieval: Designing, querying, and evaluation information systems. In K. Haycock & M.-J. . Romaniuk (Eds.), Portable MLIS: Insights from the experts (pp. 171–185). Libraries Unlimited, an imprint of ABC-CLIO, LLC.

Weedman, J. (2021). Designing for search. In Information retrieval system design: Principles & practice (6.1 ed., pp. 119–139). SharedBook Inc.; AcademicPub.