This randomized trial was designed as a pilot study to measure the relevancy of search results using three different interfaces for the PubMed search system. Each participant was randomly assigned to one of 3 interfaces for searching PubMed (Figure 1 ). Protocol A, [14 ] used a PubMed/PICO template designed for use on a wireless handheld device. The PubMed/PICO template prompted the searcher for the PICO elements of the question (patient problem, intervention, comparison and outcome), as well as patient age group and gender. There was also an option to select a publication type. The publication types listed were clinical trial, randomized controlled trial, meta-analysis, review or practice guideline. If no publication type was selected, the search default was to include all five study designs. The PubMed/PICO template with these publication options was designed to favor questions of therapy, the most common type of question asked by clinicians [3 (link)]. Protocol B, [15 ] used a PubMed/PICO/Clinical Queries template that prompted for the PICO and allowed the search to be filtered by type of question and scope of strategy (narrow or broad) or by systematic review. Protocol B incorporated the Clinical Queries filters and allowed the searcher to consider a broader range of question types. Because templates for Protocols A and B were designed for handheld devices, participants assigned to these two protocols were given a handheld device of their choice (either a Palm™ Tungsten C or a HP iPAC Pocket PC™) to use during the study. The Web browser home page for the wireless handheld study devices was preconfigured to connect to the appropriate study interface. Protocol C, PubMed, [2 ] used the standard web-based PubMed system on a PC workstation. Protocol C did not include a PICO template for formulating the search strategy, but the participants had access to Clinical Queries if they chose to use them.
Study subjects were recruited from interns and residents on an inpatient general medicine rotation at an academic medical center in the US. Thirty-one subjects were each given three clinical questions and asked to search PubMed for a set of relevant articles that would provide an answer to the questions. The three study questions were taken from a database of actual clinical questions formulated by residents during general medicine rotations between 2001 and 2002 [3 (link)]. Two of the questions (Q2 and Q3) were related to treatment or therapy, the most common type of question asked and one question (Q1) was related to prognosis. (Table2 .) While the PICO framework was developed specifically for therapy questions [16 (link)], the prognosis question was included because PICO is being taught for all types of questions. Protocol assignments and the 3 clinical questions were placed in a concealed envelope, and participants were asked to select one envelope from a group of identical envelopes. Participants were instructed to search each clinical question, as many times as needed, in order to retrieve a final set of articles that would provide the relevant information needed to make a clinical decision in each case.
The success of the search was measured by comparing the number of relevant citations retrieved to the total number of article retrieved in the final set. The research team identified the relevant articles for each clinical question. The criteria for an article being included as relevant was that it addressed the specific clinical question, including patient, intervention and outcome, and that it was of the best study methodology based on the type of question. For example, a therapy question needed to be answered by a randomized controlled trial, systematic review, or meta-analysis, while a prognosis question required a prospective cohort study. Two researchers conducted PubMed searches for each question. These two and a third researcher selected the relevant articles from the pooled results of the searches. A fourth researcher also reviewed the results and reconciled any disagreement among the other reviewers.
Participants using the handheld devices wrote down their IP address, the time, and the number of citations in the set that best addressed the clinical question. This information was matched with data collected by the project server at NLM and was used to verify the final result set. Participants using the PC workstation were asked to save their final set results to the study account in MyNCBI. Screen captures, a means of saving the image of the search, were made of their complete search history to back up their saved strategies. For all participants, the search terms, date and time of the search, and the Unique Identifiers for the citations retrieved were collected by the system transactions logs and stored on the NLM project server or in MyNCBI. There were no time constraints on any of the participants. The DUMC Institutional Review Board approved the study method and all participants signed an informed consent form.
Study subjects were recruited from interns and residents on an inpatient general medicine rotation at an academic medical center in the US. Thirty-one subjects were each given three clinical questions and asked to search PubMed for a set of relevant articles that would provide an answer to the questions. The three study questions were taken from a database of actual clinical questions formulated by residents during general medicine rotations between 2001 and 2002 [3 (link)]. Two of the questions (Q2 and Q3) were related to treatment or therapy, the most common type of question asked and one question (Q1) was related to prognosis. (Table
The success of the search was measured by comparing the number of relevant citations retrieved to the total number of article retrieved in the final set. The research team identified the relevant articles for each clinical question. The criteria for an article being included as relevant was that it addressed the specific clinical question, including patient, intervention and outcome, and that it was of the best study methodology based on the type of question. For example, a therapy question needed to be answered by a randomized controlled trial, systematic review, or meta-analysis, while a prognosis question required a prospective cohort study. Two researchers conducted PubMed searches for each question. These two and a third researcher selected the relevant articles from the pooled results of the searches. A fourth researcher also reviewed the results and reconciled any disagreement among the other reviewers.
Participants using the handheld devices wrote down their IP address, the time, and the number of citations in the set that best addressed the clinical question. This information was matched with data collected by the project server at NLM and was used to verify the final result set. Participants using the PC workstation were asked to save their final set results to the study account in MyNCBI. Screen captures, a means of saving the image of the search, were made of their complete search history to back up their saved strategies. For all participants, the search terms, date and time of the search, and the Unique Identifiers for the citations retrieved were collected by the system transactions logs and stored on the NLM project server or in MyNCBI. There were no time constraints on any of the participants. The DUMC Institutional Review Board approved the study method and all participants signed an informed consent form.
Free full text: Click here
