The AKAMAI Evaluation Initiative: Evaluating the Impact of
Telemedicine on Health Care Delivery (Phase I)

V. Garshnek, Ph.D.* and L.H. Hassell, COL, MC, USA**

*Pacific Region Program Office, Tripler Regional Medical Center, HI
**AMEDD Studies Branch, AMEDD Center and School, Fort Sam Houston, TX

Abstract: Telemedicine is the use of modern telecommunications and information technologies for clinical care of individuals at a distance. It has the potential to profoundly alter the medical landscape world-wide. One effort in particular, Project AKAMAI, is contributing to this vision by pioneering the implementation of telemedicine in the Pacific. AKAMAI is a Federally funded program designed to augment health care delivery to DoD and beneficiaries in the Pacific using advanced telecommunications technology. The objective is to provide health care policy-makers with scientific and technical information. AKAMAI, led by Tripler Regional Medical Center (TRMC), Hawaii, consists of 27 individual projects and studies. Among these is a research study evaluating telemedicine in a limited setting (consultation between Hickam Primary Care Clinic, Hawaii, and TRMC in the areas of Dermatology and Orthopedics). The study consists of four sub projects in the areas of Clinical Outcomes, Patient/Provider Satisfaction, Organizational Impact, and Cost/Benefit, with the intent of providing information on the influence of telemedicine insertion into a military health care system. The evaluation study will be conducted in three phases: Phase I (Infrastructure design/instrument and procedure development); Phase II (Experiment Verification Test (EVT) or system/procedure simulation) and Phase III (Experiment/Data Analysis). Phase I is currently in progress.

Introduction: What is Project AKAMAI?

Medicine is an infinite frontier - as infinite as the mysteries of the human body. We are living in an extraordinary time for technology is rapidly changing the face of health care practice, access, and education ushering our planet into a new health care age for the new Millennium. Through telecommunications and information technologies, medicine can extend its reach regardless of physical distance through two-way transmission of information between places of lesser and greater medical capability and expertise. This capacity, known as telemedicine, has the potential to profoundly alter the medical landscape world-wide. One effort in particular, known as Project AKAMAI, is contributing to the fulfillment this vision by pioneering the implementation of telemedicine in the Pacific.

"AKAMAI" is a Hawaiian word for clever or smart. It embraces the spirit of using new or improved tools and ideas to do things better. We are becoming "akamai" in the way we practice medicine with new technologies and methods that are expanding medicine's reach and effectiveness with seemingly endless possibilities. Thus, the name "Project AKAMAI" is appropriate for a Federally funded program, led by Tripler Regional Medical Center (TRMC), HI, designed to augment health care delivery to DoD and other beneficiaries in the Pacific using advanced telecommunications technology. The new capability may allow more patients to receive definitive treatment at their local health care facility and thus, avoid the requirement for aeromedical evacuation.

The vision of Project AKAMAI is (a) to enhance the health care delivery system by linking Federal agencies with advanced telecommunications capabilities that enable patient information to be transmitted digitally between health care providers in local and remote geographic locations and (b) to improve military medical readiness in the Pacific Basin Region by projecting the medical specialty expertise of TRMC, Hawaii, to remote sites through the use of telemedicine. A major objective of Project AKAMAI is to provide healthcare policy-makers with scientific and technical information to assist them in the decision to employ this telecommunications technology within the scope of capabilities of all DoD medical treatment facilities.

Telemedicine Evaluation Study

AKAMAI consists of 27 individual projects and studies divided into the broad areas of operations (teleradiology, multimedia systems, needs assessments), evaluation, and emerging technology exploration. Of these, one is a scientifically peer-reviewed and approved research study entitled, "A Pilot Project to Evaluate the Impact of Telemedicine on Health Care Delivery." Its primary focus is to evaluate telemedicine in a limited military setting between Tripler Regional Medical Center and Hickam Primary Care Clinic, HI. The evaluation study was launched due to the fact that despite the simplicity and apparent logic of the telemedicine concept and three decades of experience, we are still unable to show that telemedicine improves access, reduces cost, or affects the quality of healthcare.

The evaluation study consists of four sub-projects, each with a principal investigator and co-investigators representing full research studies in their own right, yet highly interdependent to yield an integrated "big picture" of the influence of telemedicine insertion into the military health care system. The four sub projects address the following research aims: to investigate if telemedicine consultation between medical treatment facilities (a) impacts the clinical outcome of patients compared to usual care; (b) affects patient and provider satisfaction compared to usual care and (c) reduces cost in relation to benefits compared to usual care; and (d) to investigate the organizational changes that occur when telemedicine is implemented in a medical treatment facility.

The general protocol employs a randomized design to compare patient and process outcomes of two methods of consultation (telemedicine and usual care) from a primary care outpatient clinic (at Hickam AFB, HI) to a tertiary care medical center (TRMC). The sample population for this study is active duty and retired military and their dependents requiring consultation from a primary care to a tertiary care facility and their health care providers. For the study, we made the a priori assumption that technology is currently available to transmit diagnostic quality media elements necessary for clinical decision making. We also established "boundaries" which include the following:

- Only routine consultations will be evaluated
- Store and forward technology will be used
- Focus on population health care delivery
- Physician to physician consultation
- Only Dermatology and Orthopedics specialties will be utilized for data collection

The Evaluation Study will be conducted in three phases over a two-year period:

Phase I: Infrastructure design, formally known as the Tripler Telemedicine Prototype Project (T2P2) -- In this part of the study, we are designing and building the infrastructure to capture clinical information from a patient at a remote site and electronically transfer this information to TRMC for consultation. Also during this phase, survey instruments and data-gathering procedures will be developed.
Phase II: Experiment Verification Test (EVT) -- During this phase, a system/procedure simulation at TRMC will take place and instruments/methods refined based on the simulation outcomes.
Phase III: Experiment/Analysis -- During this phase, data will be gathered over a 90-day period. TRMC Dermatology and Orthopedics Services will be asked to see routine consults from the Hickam Primary Care Clinic.

As of this writing, Phase I is currently in progress. The following describes the background, current status, and plans of the four sub project areas.

Project #1: The Effect of Telemedicine Implementation on Clinical Outcomes in a Limited Military Setting -- Principal Investigator: L.H. Hassell, COL, MC, USA (AMEDD Center and School, TX); Co-Investigators: V. Garshnek, Ph.D. (TRMC, HI); M. Nadeau, LTC, MC, USAF (Hickam AFB, HI); G. Underwood, M.D. (TRMC, HI)

Few well-designed studies exist evaluating the influence of telemedicine on health care delivery. In addition, of the few studies existing, it is very difficult to generalize the findings to the military health care system and patient population. Consequently, there is clearly a need for well designed research on (a) outcomes in telemedicine in general and (b) outcomes in telemedicine in the military health care system. The objective of this particular study is to provide a scientific evaluation of clinical outcomes resulting from insertion of telemedicine into a primary care clinic that routinely refers patients to a tertiary medical center for specialty care (specifically, comparing the impact of telemedicine with usual care on key processes and outcome measures). The central hypothesis states that the clinical outcomes of patients are not adversely affected when telemedicine is used as a substitute for usual care in the military health care delivery system.

The study uses store and forward telemedicine technology (internet-based), a method that appends textual information about a clinical problem with graphic data files of physical exam findings and diagnostic results. Clinical protocols (consult "worksheets") have been developed for 12 specialty areas. These clinical protocols were developed jointly by Tri-Service primary and specialty physicians on Oahu. Each protocol defines the essential history and physical examination information, lab, and radiograph results, and clinical images required for a consultant to make a remote diagnosis and recommend treatment for a distinct clinical problem.

The process envisioned consists of the following: The health care provider accesses the server using an off-the-shelf web browser and selects a clinical consultation protocol to initiate a consult. The server will assemble information from a number of sources on the hospital local area network. This consult will then be "mailed electronically" over the internet to medical specialists for analysis and recommendation. The system will allow the health care provider to choose the time the electronic consult is reviewed. This method overcomes the distinct limitations caused by interactive video systems in the past. Importantly, the remote physician will initiate, and the consultant answer, the consult from their desktop personal computer. We will pilot the server as part of the AKAMAI Evaluation Initiative in 1998.

Project #2: Designing a Satisfaction Tool for Telemedicine Services-- Principal Investigator: S. DeRuvo, LTC, NC, USA (Landstuhl RMC, Germany); Co-Investigator: E. Hill, LTC, NC, USA (TRMC,HI)

The purpose of this study is to develop, pilot, and revise as needed, an instrument for evaluating patient and provider satisfaction with the delivery of health care through telemedicine. While there is a great deal of literature related to telemedicine, there are few studies adequately assessing satisfaction of either patient or provider. As this particular study will be conducted as an ongoing project during the initial implementation of telemedicine in the region, feedback from both patients and providers can be used to make changes in the system as it is being developed. In addition, investigators will develop a concept analysis of telemedicine, to be used in future applications related to this study.

Initial surveys will be piloted, revised as needed, and then final surveys developed and tested. Once the systems for telemedicine consultation are up and running for at least 2 to 3 months, focus groups evaluating patient and provider satisfaction will be conducted. Prior to this, other focus groups will be conducted off-site to give some initial groundwork and an idea of what should be done in setting up telemedicine at TRMC, as well as some interesting comparisons of how the civilian, mainland systems compare to the system that will be established and used by the military here on Oahu, and later in the Pacific Region. Arrangements are currently underway to conduct the preliminary focus groups.

Project #3: Assessing Organizational Impact of the Implementation of a Telemedicine System -- Principal Investigator: R. Doktor, Ph.D.; Co-Investigator: D. Bangert, Ph.D. (University of Hawaii College of Business Administration)

The Organizational Impact study seeks to assess the impact on the organization which occurs as a consequence of the introduction of telemedicine technology, and concomitantly seeks to identify and assess the organizational factors which, if effectively prepositioned, will minimize resistance and maximize acceptance and utilization of telemedicine technology. The study team has chosen to focus upon variables at the individual group and organizational level. Sensitivity analysis will be performed in order to identify variables which have the greatest potential impact in insuring successful implementation of future telemedicine program introductions. Thus far, on the individual level, the research team has identified an inventory which measures cognitive perceptions of the caregivers job characteristics. On the group level, the team has identified the medical decision making process as the variable to be measured and has adopted qualitative research methodology of ethnography as the operative tool. Thus, through observation and interview, the research team will study medical decision making which may occur. On the macro or organizational level, the team has chosen an inventory assessing organizational learning as the measuring instrument and plan to administer it with the same regimen as the job characteristics inventory. The variables measured by the organizational learning inventory, give an indication of changes in the organization's culture as a consequence of the introduction of telemedicine. Currently, the team is finalizing the evaluation instruments and preparing procedures to test these instruments during the EVT phase of this study.

Project #4: Assessing Cost-Effectiveness in a Telemedicine System -- Principal Investigator: E. Mais, Ph.D. (University of Hawaii College of Business Administration)

An analysis of the cost-benefit/effectiveness that occurs within a medical care system when new technology is introduced has not been adequately addressed. Virtually no evidence exists concerning the economic viability of telemedicine. Therefore, a strong need exists for concrete, scientific evidence on (a) the cost-effectiveness of telemedicine in general and (b) the cost-effectiveness of telemedicine in the military health care system. The aim of this study is to investigate if telemedicine consultation between medical treatment facilities reduces cost in relation to benefits compared to usual care. The economic analysis of this project will proceed in five parts. Part 1 will identify the important and relevant costs and consequences for the two alternatives specified (telemedicine vs. usual care). Part 2 will accurately measure the identified costs and consequences in appropriate physical units. Part 3 will value the measured costs and consequences in a credible and valid manner. Part 4 will comprise a sensitivity analysis of the costs and consequences. Part 5 will present the results using decision indices that will summarize the findings of the study. The point of view will be that of the patient and the health care provider and will include both direct and indirect costs and consequences. The analysis will follow closely the research design of Project 1 (Clinical Outcomes) so as to appropriately capture the economic differences between consultation to a tertiary care medical center by telemedicine technology versus usual care. Currently, refinement of an initial cost/benefit model is underway based on a high-level process flow diagram for the telemedicine consults and will continue refinement through the EVT phase.

Summary/Conclusion

This paper addresses the background, development, and status of the AKAMAI Telemedicine Evaluation pilot study. Currently the study is in the first phase of activity, concentrating on development of infrastructure, procedures, and instruments. It is our sincere hope that our methodology and final results will be of value and applicable to other telemedicine research activities in the future.