QuickClinical

The QuickClinical (QC) information retrieval system is a new type evidence-access technology that utilises intelligent search filter technology to model typical clinical tasks like ‘diagnosis’ or ‘prescribing’ and ensure that only the most relevant evidence is retrieved. This means clinicians are more likely to search, and when they do search, are more likely to find information which changes their practice.

QC won the Intel Don Walker award in 2006, presented by the Health Informatics Society of Australi. QC is now being commercialised by UNSW.

Contents

---

Background

We know there is a significant gap in current knowledge services. While present generation search technologies have made improvements to evidence accessibility, clinicians still have large unmet information needs, with significant consequences for the public. For example, the greatest proportion (56%) of preventable adverse drug events occurs at the drug ordering stage where access to appropriate information could have avoided the error. Doctors fail to find the information they need because they are unsure what is available, where to look for it, have very limited time available to conduct searches, and when they do search they have poor query formulation skills and often abandon searches because of time pressure. Often the evidence needed is scattered across heterogenous data sources, each with their own unique query and indexing methods, creating further barriers.

Description

The QC user model guides clinicians to first consider the purpose of their search through selection of a profile, and it then asks them to provide specific keywords related to that search task. As a consequence, users are guided through a process that structures their query for them and improves the chances that they will ask a well-formed query and receive an appropriate answer. Figure 1 depicts the QC search interface. On the left hand is a list of search filters that describe typical search tasks and that are customized to the specific information needs of primary care physicians.
Most information sources such as Web sites, on-line texts and databases have their own, proprietary search interface including query language and format for the display of results. Quick Clinical software addresses many of the limitations of other search tools by providing a rule-based mechanism to search only the most relevant of all the available resources, translating and enhancing user queries into the respective query languages of each resource. The underlying design concept within QC is the notion of a Meta-search filter (MSF) which combines the power of meta-search systems and predefined search filter technologies. A MSF might describe which repositories are most appropriate in answering a typical question for researchers in a given discipline, and how best to ask the question within different resources. They can be thought of as encodings of search strategies that capture expert knowledge on where and how to search for answers. MSFs can be designed to support specific user groups, and different tasks and contexts associated with each group. For example, the strategy to search for information on the treatment of a disease for a researcher conducting clinical trials and for a researcher conducting basic science research in a laboratory would probably need to return very different documents sets, probably derived from different resources, reflecting the different skills and needs of these two groups.

Table 1: Search Profiles are collections of specialised filters, individualised for specific sources

Table 1 demonstrates and Example Quick Clinical meta-search filter. The sample “Treatment” profile above describes a set of source-specific search filters, which collectively describe the search strategy believed most likely to retrieve an accurate search result from each resource. The # symbol that appears in the table delimits keyword variables that are to be replaced by user supplied keywords. For example, #1# represents the keyword type “disease,” and QC’s mediator component will substitute the user-provided keywords for ‘Disease’ throughout the profile, prior to sending the query to the individual wrappers for the different sources.

Components of MSF thus include:

• Sources to be searched. Sources can include web repositories, local digital archives of research collections that house ‘grey’ literature not yet available in the biomedical journals, and web versions of academic journals;

• Search filters to be used when querying different sources. Search filters reformulate a user query into the syntax native to a given source, and may add in additional keywords known to improve search quality. For example, if a biomedical scientist selects a ‘diagnosis’ MSF and enters the search term ‘asthma’, when the MSF queries Medline it can add in the additional terms "sensitivity and specificity" [MESH] OR "sensitivity" [WORD] OR "diagnosis" [SH] OR "diagnostic use" [SH] OR "specificity" [WORD]. These terms have been shown to significantly enhance the quality of Medline results, but are unlikely to be known to a typical clinical user. Multiple filters may be constructed for a single source, reflecting different strategies for asking a question.

• Context settings. The display of results can be configured to blend the results obtained from different repositories in a manner most likely to support the intended user. For example, the order in which results from different sources are presented can reflect the likelihood that a source will provide an answer. Order of presentation has been shown in our own research to also influence the way users interpret information, and there is a significant opportunity to develop interfaces that support decision-making by understanding the way digital objects are presented to users. Internally, QC employs a wrapper-mediator architecture. Each repository that is known to the system has a ‘wrapper’ that describes the attributes of the system, such as how searches are to be expressed. Wrappers contain a Document Object Model (DOM), that records how pages are marked up, and what meta-data might be associated with the document. The mediator component takes a query, and re-expresses it as dictated by the MSF and associated wrappers. XML is used for internal communication between components.

QC has had a demonstrated impact on the decision-making behaviors of primary care physicians, the initial target group for the technology, significantly improving the accuracy and timeliness of decision made. QC improves decision accuracy by 20% and using QC is much faster than standard on-line methods (on average 4.5 Vs 6.6 minutes), making it more likely to be uses routinely. In addition to the impact on the quality of clinical decisions, and concomitant reduction in error, the time-savings are also significant. Estimating that 1 minute added to a GP consultation requires an additional 800 GPs to provide the same service levels, saving 2 minutes per consultation represents a significant contribution to primary care service provision, independent of the additional impact of improved clinical decision-making.

QC has undergone multiple stringent evaluations between 2001-6, both in controlled laboratory settings, and in routine use in a primary care setting. Our largest trial involved over 200 general practitioners using QC across Australia. Our trials have measured clinical satisfaction, usage rates, confidence in decision, decision accuracy, time to make decision, and impact on clinical encounter, amongst a host of other clinical variables. We have also evaluated the technical performance of the system, demonstrating its robustness and ability to work under load conditions (full details of the technical evaluation are available as part of a technical description of QC at http://www.jmir.org/2005/5/e52/.) Our controlled laboratory tests have included a randomisation component comparing QC to standard search methods, to assist us in demonstrating a clear cause and effect relationship between system use, and decision making improvements. The studies have shown that:

GPs use the system in the routine clinical setting

In one trial, GPs conducted 1680 searches over four weeks. The use of the system varied over the week and 42% of use occurred in the middle of the week. Seventy nine percent of searches were conducted between 9am-7pm and 62% were initiated in consulting rooms, suggesting that the system integrated into day-to-day workflow and was used during consultations.QC is able to find clinically relevant evidence: GPs reported that in 78% of queries, the results obtained were important or very important to the care of their patient.

Use of QC enable faster and more effective searches

We have conducted laboratory studies to compare the effectiveness of QC against traditional ‘library’ or source-oriented retrieval methods (Westbrook et al., 2004). Three user groups (n=75: 26 hospital doctors, 18 general practitioners and 31 clinical nurse consultants) were randomised to one or other system and asked 8 clinical questions. Clinicians using QC were much faster than those using standard on-line methods (on average 4.5 Vs 6.6 minutes, df=266, p=0.76), and the QC evidence filters resulted in a mean increase of 20% in the proportion of correct answers clinicians gave after searching, compared with unaided search.

Use of QC improves the performance of non-medical clinicians. We have demonstrated that statistically significant differences in the ability to answer clinical questions between clinical nurse consultants and doctors disappear when QC aids both groups in answering clinical questions.

Commercialisation

Quick Clinical (QC) technologies have moved from the academic domain into real world use:UNSW has now twice written commercial licences for the technology, one to a major on-line publisher in the taxation/legal field, and one to a large international Pharmaceutical organization.A pilot implementation is planned for the second half of 2006.Quick Clinical was selected as a finalist in the 2005 Australian Information Industry Association (AIIA) awards, and the 2005 IT secrets competition, sponsored by Austrade.Quick Clinical is currently also under evaluation for use in the UK in conjunction with the National Electronic Library for Health, and by the BMJ publishing group.

References

• Coiera E, Magrabi F, Westbrook J, Kidd M, Day R, Protocol for the Quick Clinical study: a randomised controlled trial to assess the impact of an online evidence retrieval system on decision-making in general practice [ISRCTN03597773], BMC Biomedical Informatics and Decision Making 2006, 6:33 doi:10.1186/1472-6947-6-33 http://www.biomedcentral.com/1472-6947/6/33.
• Coiera E, Walther M, Nguyen K, Lovell NH , An architecture for knowledge-based and federated search of online clinical evidence, Journal of Medical Internet Research. 2005 Oct 24;7(5). http://www.jmir.org/2005/5/e52/.)
• Magrabi F, Coiera EW, Westbrook J, Gosling AS, Vickland V General practitioners' use of online evidence during consultations, International Journal of Medical Informatics, 2005;74(1),1-12.
• Westbrook JI, Coiera EW Gosling AS,. Do online information retrieval systems help experienced clinicians answer clinical questions? J Am Med Inform Assoc 2005; 12: 315-321.
• Westbrook JI, Gosling AS, Coiera EW. The impact of an online evidence system on confidence in decision making in a controlled setting. Medical Decision Making. 2005;25:178-185. (accompanying editorial: W. Hersh, Ubiquitous but Unfinished: Online Information Retrieval Systems, Medical Decision Making, 2005;25:147-8.)