| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
From the Department of Anesthesiology, Kingston General Hospital, Queen’s University, Kingston, Ontario, Canada.
Address correspondence to: Dr. Elizabeth G. VanDenKerkhof, Department of Anesthesiology, Kingston General Hospital, 76 Stuart St., Kingston, Ontario K7L 2V7, Canada. Phone: 613-549-6666, ext. 3964; Fax: 613-548-1375; Email: ev5{at}post.queensu.ca
 |
Abstract |
|---|
 TOP Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Methods: Using a historical control and a "time and motion" study design, comparisons were made on acute pain assessment time and comprehensiveness when patient assessments are documented on a PDA vs the current paper-based method.
Results: The study physician (a PDA-user) reported feeling comfortable with the assessment software after five patient assessments. PDA assessments were more likely to contain documentation regarding pain and side effects (e.g., nausea, pruritus, hypotension) than the paper assessments. The median time of the "assessment only" component of the patient encounter was 53 sec longer using the PDA compared to paper (P < 0.00), however, the median "total encounter" (chart review, assessment, documentation) time was 74 sec shorter using the PDA vs paper (P < 0.00).
Discussion: The findings of this preliminary study suggest that the PDA is a reliable tool that meets the data management requirements within an APMS setting. This study found that patient assessments documented using acute pain software developed for use on a PDA were as efficient and content-rich as paper assessments. The PDA may even enhance the efficiency of the patient assessment process through the provision of more comprehensive digital data for research, clinical, and administrative needs.
|
Introduction |
|---|
|
TOPAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
|
Methods |
|---|
|
TOPAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Approval to complete this study was given by the Queen’s University Health Sciences Research Ethics Board. Patient assessments were performed in the routine manner and the study physician did not alter patient care provided by the APMS. A thorough review was performed of the current flow of patients through the APMS at Kingston General Hospital, followed by an adaptation of the flow to include electronic assessments.13 Based on these findings, plus the findings of our national survey of academic acute pain services,14 a portable assessment record was developed using PDAs on the Palm Operating System (Palm Computing® Platform, Palm Inc., CA, USA).
A before/after study was designed using historical controls to compare the PM assessments with the PDA assessments. This approach was selected over a randomized parallel design in order to blind the study physician to the contents of the APMS software. No studies were identified in the literature that assessed the impact on patient care of the use of PDAs to document patient assessments (i.e., do physicians document differently when using a PDA?). Given that clinicians may alter their documentation patterns, it was believed that this pilot study should be carried out separate from regular patient rounds. All of the information that was available during APMS rounds was made available to the study physician during both the PM and the PDA assessments; however, the data collected by the study physician were only made available for study purposes and did not become part of the official patient chart. One physician (third year anesthesiology resident) who had spent one month with the APMS as part of his training was recruited to test the tool. The rationale for having only one physician participate in the study was that this was the first test of the PDA with the assessment software in the clinical setting, therefore we believed it was prudent to test it outside of patient rounds before introducing it into patient rounds in a study involving the APMS physicians. The alternate solution was to have a research assistant carry out the study, however, it was believed that this would have resulted in an even less accurate depiction of the point-of-care technology as a clinical assessment tool.
The study was conducted over a three-week period in February/March 2001. The control group (PM) was recruited in the first half of the study period (February 19–27, 2001) and the treatment group (PDA) was recruited in the second half of the study period (February 29–March 9, 2001). For both groups the study physician spent the initiation period (two days) becoming familiar with the data collection process and the timing of assessments. Any issues that arose during that time were addressed before the formal data collection process began. After the initiation period for each arm, five days were spent recruiting patients and recording assessments. For the PM arm the study physician had access to the complete patient chart but documented on a blank chart that included a doctor’s order sheet and a progress note sheet. The study physician was asked to record his assessment and orders as if he were performing and recording a routine APMS assessment. The APMS software was not downloaded to the study physician’s PDA until completion of the PM arm of the study. The study physician spent the first two days of the PDA arm of the study becoming familiar with the software followed by five days of data collection. After all study assessments were completed, the PM assessments were entered into an Excel spreadsheet (Appendix available at www.cja-jca.org). In both the PM and the PDA arms, free text not captured by the APMS variables, was captured under the open-ended variable called "Doctors Notes". Patient confidentiality on the PDA was addressed through the use of a login identification number and a password which the study physician was required to enter before patient data populated the PDA screen.
The APMS software included standard pain scoring systems (e.g., numeric rating scale), as well as an extensive list of potential drug-related side effects (Appendix available at www.cja-jca.org). Items could be selected by checking tick boxes or items from a drop-down menu. The study physician was not forced to follow a specific order, but could move freely between screens without having to make a selection. A depiction of a standard screen can be found in Figure 1
. The study physician was asked to record the findings of the assessment on the PDA, either in front of the patient or outside the patient’s room.
|
Patients recruitment was standardized for both the PM and PDA arms of the study. The majority of patients are routinely admitted to the APMS by a consultation process initiated in the operating room between the surgeon and the anesthesiologist. At this time an APMS consult request is completed and the patient is entered onto the APMS list by the anesthesiologist. Alternately the patient may be entered onto the APMS list in the postanesthesia care unit by either the anesthesiologist or APMS nurse. This process flags the patient’s record on the HIS indicating that the patient has been admitted to the APMS. Each morning the APMS nurse or anesthesiologist generates a paper list of patients on service from the HIS. For the purpose of this study, patients on the PM arm were identified for the study from this paper list generated from the HIS. The same mechanism was used for the PDA arm, except rather than print a paper list, an interface was developed between the HIS and the APMS server. Patient’s names were transferred directly from the HIS to the APMS server and then downloaded via synchronization to the PDA.
PM and PDA assessment content and time were compared. The frequency of documentation of pain and side effect variables for the two methods were determined. Comparisons were made between the median and mean time (in seconds) required to perform the patient assessment, record the assessment, and the total encounter time. Tests for normality were performed using the Kolmogorov-Smirnov test. The data were found to be not normally distributed, hence, the Mann-Whitney test was used to test for differences in the median assessment, recording, and total encounter time for PDA vs paper. Repeated measures analysis was not used, as the relationship between the various components of the total patient encounter time was not relevant to this analysis. In order to account for dependency between outcomes, the analysis was performed twice, on all patient assessments and on the first assessment only for each patient. Results were similar on data including only the first assessment per patient and on data including all assessments per patient; therefore, all assessments per patient were included and treated independently in the final analysis. Sample size was calculated using an alpha of 0.05 and a power of 80% with an effect size of 60 sec and standard deviation of 84 sec. Based on these requirements, it was determined that at least 32 patients would be required per arm. The goal was to perform 100 PM assessments and 100 PDA assessments.
|
Results |
|---|
|
TOPAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The age and sex characteristics did not differ between the two arms. There were slightly more males than females in both arms of the study and the average age in each arm was approximately 60 yr. Table I lists the percent of times the ten most frequently documented side effects were included in the paper record, and the corresponding percent of times the side effects were documented on the PDA. All ten side effects were recorded for virtually all the PDA assessments, while on the PM arm the percent of times the side effects were recorded ranged from 100% for hypotension down to 5% for headache. No significant difference was found between documentation of the numeric rating scale (NRS) score for paper vs PDA (Table II), however, the nature of pain was much more likely to be described on the PDA vs PM arm. Aspects of the patient assessment times are reported in Table III. Assessment time only, recording time only, assessment plus recording time, and total patient encounter time are reported. There were statistically significant differences between the PDA and paper arms in median assessment time for all time-points in the assessment process. The median time to complete the assessment only was 53 sec longer for the PDA arm compared to paper arm (P = 0.00), however, when recording was included, the median time was 61 sec shorter in the PDA arm compared to the paper arm (P = 0.00). The median PDA time remained significantly shorter than the median paper time when the total encounter was considered (227 vs 301 sec, P = 0.00). Figure 2 depicts the median total encounter time by day of data collection for each arm when all patient assessments were included in the analysis and when only the first assessment per patient was included. Encounter time was slightly higher on day one of the PDA arm; however, the difference in encounter time between the PDA and the PM was consistent over days two to five. Encounter times did not vary significantly between the first patient assessment and all patient assessments.
|
|
|
|
|
Discussion |
|---|
|
TOPAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
In reviewing the discrepancy in content between the two arms, several factors must be addressed. Information may have been forgotten on the PM arm because recording took place outside of the patient’s room. Lack of documentation on the PM arm may also be indicative of a negative result or a side effect thought to be irrelevant to the clinical situation. For example, if a patient does not have a headache the clinician may not record this information unless it is deemed to be relevant to the situation (e.g., epidural or spinal anesthetic). Therefore, without prompting by the physician, a side effect such as headache or pruritus may not be captured unless the patient offers the information. Patients are not necessarily versed in side effects associated with therapy (e.g., pruritus, headaches). Such symptoms may not be viewed by patients as being associated with pain and therefore may not be reported to the APMS. Prompts on the PDA may encourage physicians to ask patients about specific side effects associated with therapy, thereby resulting in a more comprehensive approach to patient care. In spite of the discrepancy in documentation content, the primary opioid-related side effects found in the literature14,15 - hypotension, pruritus and nausea - were also the most frequently documented side effects in both arms of the study. Finally, a paper check list may well have resulted in the same amount of data being collected in the PM arm, however, the purpose of this study was to compare the use of a PDA to the current "gold standard", which in most institutions is not in the form of a checklist, but rather a blank assessment record. A paper checklist would also not result in data that is readily available, with the capacity to be queried, for ongoing clinical, quality assurance, administrative and research purposes.
The use of the NRS to document level of pain was consistently reported in both arms of the study. The use of this measure to document pain levels is supported by the literature.14,16,17 In patients reporting a NRS score greater than 0, the study physician elaborated on the characterization and location of the reported pain in the majority of assessments on the PDA arm, but not on the PM arm. The documentation of detailed pain information on the PDA arm was likely due to the prompts provided by the PDA. This impact on documentation content is an important finding, as the identification and characterization of pain is necessary in order to distinguish between expected surgical pain (visceral or somatic) and other potential postoperative complications (e.g., acute neuropathic pain).
The various aspects of the patient encounter presented in Table III
indicate that while recording on the patient’s chart occurred after the patient assessment in the PM, recording on the PDA occurred during the assessment. This may also account for the discrepancy in comprehensiveness of documentation between the two arms. Documentation on the PDA may also be influenced by physician practices. Practice patterns, including documentation, may differ for physicians-in-training compared to physicians with extensive experience. While the study physician used the PDA as a guide to his assessment, our early impression from experienced physicians is that they view electronic data capture as a data entry tool. This in combination with the tendency by many physicians to document on the patient’s chart outside of the patient’s room may result in experienced physicians documenting differently on the PDA than our findings suggest. In particular, when recording occurs after the assessment, outside of the patient’s room, the assessment prompts provided on the PDA may not be fully utilized. Unlike the current paper record, the PDA may increase assessment documentation content to a certain extent by reminding physicians to document information they observed or obtained from the patient. However, prompts for data the physician did not collect are less likely to be utilized, as it would require that the physician return to the patient’s room to obtain the information.
One might expect total patient encounter time to decrease as the study physician became facile in using the PDA assessment tool, however this did not occur as illustrated in Figure 2. The consistent nature of the difference in time between the two methods indicates that the historical nature of the study did not adversely affect the findings. The study physician had a history of PDA use, but he had no prior experience with the assessment software, therefore, as expected, the total encounter time was slightly longer on the first day of the PDA arm, however, on subsequent days the time remained relatively consistent.
The findings of this study suggest that the use of PDAs at the point-of-care is equivalent to paper and may even enhance the patient assessment process, while providing more comprehensive data for research, clinical, and administrative needs. The limited adoption of these real-time systems often occurs because of the lack of infrastructure required to support these systems in most hospitals.18 However, with the introduction of PDAs and wireless technology into the health care setting, a viable opportunity exists to bring real-time information to the patient’s side, while collecting valuable patient information in an electronic format.3,4,12,19,20 A report by Harris Interactive indicates that approximately 26% of physicians now use handheld computers in their practice and they predict that this number may increase to 50% by 2005.21
Study strengths include the use of a single investigator with experience in both acute pain patient assessments and in the use of PDAs. While it could be argued that some of the differences between groups could be attributed to increased efficiency in performing patient assessments and in the use of the PDA, it is believed that the differences between groups are largely a reflection of real differences in the two methods. The study occurred over a very short period of time (three weeks); therefore issues such as changes in physician practices and the evolution of technology did not affect the findings. Due to the paucity of data in the literature assessing the impact of technology on health care and clinical decision-making at the bedside, it was deemed relevant to conduct the study outside of formal patient rounds. The study physician simulated patient rounds, but decisions he made regarding patient care were not carried out. In situations where he felt that the information he obtained should be acted upon, he notified the APMS physician. All other aspects of the patient encounter were consistent with routine APMS rounds. The study took place within a clinical setting and all of the data normally available to the APMS were available to the study physician. A further strength of the study is that a time and motion study was carried out in a consistent manner for both arms using a PDA with stopwatch software.
Study limitations include the non-randomized nature of the study. However, it was felt that the disadvantage of not using a randomized design was outweighed by the advantage of the study physician not being influenced by the contents of the acute pain software when performing PM assessments. An additional limitation is that only one physician, who was early in his specialty training, participated in the study. As such, the results can only be generalized to physicians in similar circumstances. The study physician was a previous PDA owner. Non-PDA owners may perform differently with respect to both timing and comprehensiveness of data collection. The study physician found the tool to be very useful as a guide for the assessment. It is expected that experienced physicians are less likely to be impressed by the usefulness of the tool as a guide to their assessments. Assessments were performed after APMS rounds, resulting in potentially shorter than usual assessment times, however, this bias existed in both arms of the study. A further limitation to the study was the study physician was responsible for timing his own assessments, which may have affected timing. Again this limitation was present in both arms and any impact on timing was likely experienced in the PDA arm. The handling of two PDAs (the PDA timer and the assessment PDA) may have been awkward, resulting in increased assessment time on the PDA arm. A final limitation of the study was the loss of the first ten PDA patients during data synchronization. Due to over-recruitment of patients to the study (40 PDA patients), this loss resulted in a sample size of 30 rather than the 32 required to achieve a significant result with a 60 sec difference between arms. Given that the difference in time between arms for the total patient encounter was greater than 60 sec plus the fact that the final analysis included all patient assessments rather than only one assessment per patient, the study had ample power to detect a significant result.
The findings of this study suggest that the PDA is a reliable tool that meets the data management requirements within an APMS setting. Since the ultimate goal in developing point-of-care systems is to have access to near real-time information to assist in evidence-based clinical decision-making at the bedside, hand held technology linked with the HIS appears to be a viable option in an APMS setting. Research and development efforts play an important role in the adoption of these technologies in a clinical environment.22 A follow-up clinical trial has recently been completed comparing consultant anesthesiologist’s assessment time and comprehensiveness using PDAs vs paper on APMS rounds. An additional study is underway looking at the feasibility of having patients complete their self-administered pre-admission questionnaire using a computerized interface. Future trials will investigate the effectiveness and practicality of the hand held in concert with wireless computer technologies in the APMS environment, and the implementation and integration of an electronic preoperative consultation record and an intraoperative automated record keeper.
|
Acknowledgments |
|---|
|
Footnotes |
|---|
Revision received January 8, 2003. Accepted for publication September 30, 2002.
|
References |
|---|
|
TOPAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
2 Shortliffe EH. Computer-Based Medical Consultations: MYCIN. New York, NY: Elsevier/North; 1976.
3 Snyder-Halpern R. Assessing health care setting readiness for point of care computerized clinical decision support system innovations. Outcomes Manag Nurs Pract 1999; 3: 118–27.[Medline]
4 Lapinsky SE, Weshler J, Mehta S, Varkul M, Hallett D, Stewart TE. Handheld computers in critical care. Crit Care 2001; 5: 227–31.[Medline]
5 Ebell MH, Barry HC. InfoRetriever: rapid access to evidence-based information on a hand-held computer. MD Comput 1998; 15: 289, 292–7.[Medline]
6 Wofford MM, Secan R, Herman C, Moran WP, Wofford JL. Clinical documentation: the handheld computer as a survival tool. MD Comput 1998; 15: 352–4, 356, 358.[Medline]
7 Robinson D. Hand-held computers in clinical audit. Sr Nurse 1993; 13: 14–7.
8 Curl M, Robinson D. Hand-held computers in clinical audit: a comparison with established paper and pencil methods. Int J Health Care Qual Assur 1994; 7: 16–20.
9 Robinson D. Data capture using hand-held computers. J Psychiatr Ment Health Nurs 1994; 1: 126–7.[Medline]
10 Lal SO, Smith FW, Davis JP, et al. Palm computer demonstrates a fast and accurate means of burn data collection. J Burn Care Rehabil 2000; 21: 559–61.[Medline]
11 Nahm R, Poston I. Measurement of the effects of an integrated, point-of-care computer system on quality of nursing documentation and patient satisfaction. Comput Nurs 2000; 18: 220–9.[Medline]
12 Young PM, Leung RM, Ho LM, McGhee SM. An evaluation of the use of hand-held computers for bedside nursing care. Int J Med Inf 2001; 62: 189–93.[Medline]
13 Goldstein DH, VanDenKerkhof EG, Rimmer MJ. New media. Can J Anesth 2002; 49: 749–54.
14 Goldstein DH, VanDenKerkhof EG. Organization of acute pain services in Canadian teaching hospitals. Can J Anesth 2001; 48: 17A (abstract).
15 Canadian Pharmacists Association, Compendium of Pharmaceuticals and Specialties. 35 ed. Ottawa: Canadian Pharmacists Association; 2002.
16 Goldstein DH, VanDenKerkhof EG, Sherlock R, Sherlock J, Harper S. How an audit of epidural patients in a community hospital setting resulted in the development of a formal acute pain management service. Pain Res Manage 2001; 6: 16–20.
17 Rawal N. 10 years of acute pain services–achievements and challenges. Reg Anesth Pain Med 1999; 24: 68–73.[Medline]
18 Drazen EL, Metzger JB, Schneider MK. Patient Care Information Systems: Successful Design and Implementation. New York, NY: Springer-Verlag; 1995.
19 Smith CM. Is wireless technology at hand for healthcare? J AHIMA 2001; 72: 22, 24
20 Larkin M. Can handheld computers improve the quality of care? Lancet 2001; 358: 1438.[Medline]
21 Taylor H, Leitman R. Physicians’ use of handheld personal computing devices increases from 15% in 1999 to 26% in 2001. Health Care News 2001; 1: 1–4.
22 Sittig DF, Jimison HB, Hazlehurst BL, et al. Techniques for identifying the applicability of new information management technologies in the clinical setting: an example focusing on handheld computers. Proc AMIA Symp 2000; 804–8.
This article has been cited by other articles:
 |
|
 |
|
  C. E. Kuziemsky, J. H. Weber-Jahnke, F. Lau, and G. M. Downing An Interdisciplinary Computer-based Information Tool for Palliative Severe Pain Management J. Am. Med. Inform. Assoc., May 1, 2008; 15(3): 374 - 382. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 F. Yu, T. K. Houston, M. N. Ray, D. Q. Garner, and E. S. Berner Patterns of Use of Handheld Clinical Decision Support Tools in the Clinical Setting Med Decis Making, December 1, 2007; 27(6): 744 - 753. [Abstract] [PDF]
|
|
|
|
 |
|
 E. G. VanDenKerkhof, D. H. Goldstein, W. C. Blaine, and M. J. Rimmer A Comparison of Paper with Electronic Patient-Completed Questionnaires in a Preoperative Clinic Anesth. Analg., October 1, 2005; 101(4): 1075 - 1080. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T K Kvien, P Mowinckel, T Heiberg, K L Dammann, O Dale, G J Aanerud, T N Alme, and T Uhlig Performance of health status measures with a pen based personal digital assistant Ann Rheum Dis, October 1, 2005; 64(10): 1480 - 1484. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Poissant, J. Pereira, R. Tamblyn, and Y. Kawasumi The Impact of Electronic Health Records on Time Efficiency of Physicians and Nurses: A Systematic Review J. Am. Med. Inform. Assoc., September 1, 2005; 12(5): 505 - 516. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. A. MISSINOU, C. H. O. OLOLA, S. ISSIFOU, P.-B. MATSIEGUI, A. A. ADEGNIKA, S. BORRMANN, D. WYPIJ, T. E. TAYLOR, and P. G. KREMSNER SHORT REPORT: PILOTING PAPERLESS DATA ENTRY FOR CLINICAL RESEARCH IN AFRICA Am J Trop Med Hyg, March 1, 2005; 72(3): 301 - 303. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
B. A. Finegan Access Denied; Care Impaired: The Benefits of Having Online Medical Information Available at the Point-of-Care Anesth. Analg., November 1, 2004; 99(5): 1450 - 1452. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Con PDA's: are we there yet? Can J Anesth, January 1, 2004; 51(1): 91 - 91. [Full Text]
|
|
|
|
|
|
E. VanDenKerkhof, D. H. Goldstein, J. Lane, M. Rimmer, and J. Van Dijk REPLY Can J Anesth, January 1, 2004; 51(1): 91 - 92. [Full Text]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
