Abstract: This paper will describe the new field of Virtual Arts Therapy, defining some of the key concepts and discussing the results of our studies involving this use of computer technology in healing settings. Examples will focus on implementations emphasizing audio multimedia.

Introduction

The computer revolution has provided musicians and artists with new tools for use in their creative process. Music synthesis, manuscript printing, digital painting . . . these are but some of the applications that have benefited from advancements in the microprocessor industry. Musicians and artists working in the therapy professions have begun to migrate these tools into their work as well. In the Music Therapy department at Immaculata College, we have developed and explored the use of virtual settings in a therapeutic context. These have included adaptation of hardware and metaphors which will allow and enhance communication with disabled or affected individuals.

As this work is both equipment and labor intensive, we found it necessary and fruitful to establish research relationships with a variety of organizations and individuals. In this report we discuss our work performed in conjunction with scientists and engineers at the University of Pennsylvania and at Bell Laboratories, with clinicians at various health care facilities, in particular, the Albert Einstein Medical Center, as well as with various artists and educators. The included illustrations are taken from large exhibitions at the Franklin Institute Science Museum, and the American Museum of Natural History, where we demonstrated our virtual therapy settings to the general public.

Background

The arts therapies involve the performing and graphics arts (music, dance, movement, drama, painting, photography, and so on) in a systematic, controlled, therapeutic environment, supervised by board-certified arts therapists, often in conjunction with medical practitioners (physicians, psychiatrists, psychologists, nurses, and others). These therapies, which often appear to untrained observers as "play sessions," are actually respected methods of communication -- a palette upon which clients and care-givers may reveal expressive, non-verbal ideas with detail and precision, for further analysis. Art has long been recognized as a healing tool, as noted even in Biblical times: "David would take his harp and play upon it. Then relief would come to Saul; he would feel better, and the evil spirit would leave him." [1] The modern application of music as a therapy began in the post-W.W.II era, when Veterans' Administration hospitals first used this form of treatment successfully with trauma sufferers.[NJ95]

Virtual Arts TherapyTM [2] (VAT) combines traditional arts therapy techniques with computer technology for the goal of alleviating symptoms, enhancing creativity, inducing relaxation or other mood states, and promoting client well-being. Certified arts therapists and health care providers in a VAT environment work in conjunction with engineers, computer scientists and other technicians to develop and monitor the application of virtual arts technologies to patients. Therapy can occur at a medical office site or, through the use of Internet, World Wide Web and other communication methods, it can also be provided in a supervised fashion at home or even at work. One advantage of VAT is that group and individual participation is no longer restricted to local access -- it can occur remotely, with patients and therapists separated by great distances. Another advantage of VAT is that immediate self-therapeutization can be provided when and where it is needed. Furthermore, recording of therapy sessions for analysis, experimentation, or repetitive administration can occur with a level of precision previously unattainable using traditional methods.

In order to better understand this technology, it is helpful to segment virtual arts applications into three categories, described as follows:

VAT systems can run the gamut from simple personal computer based systems, to elaborate NASA-style interactive interfaces.[BEG90] The therapist need not spend years becoming a computer expert -- it is possible with scripting tools to learn how to create animated productions, complete with graphics and audio, in only a few hours. VAT technologies are accessible to persons with a variety of disabilities -- they have been successfully used, for example, with psychiatric inpatients, the elderly, and non-verbal cerebral palsy sufferers.

Computer technology is not new to the arts therapies -- biofeedback, speech synthesis and other tools have long been applied to these settings.[GIL90] What makes these therapies virtual is the fact that the new technology permits a scope of experiences which creates an alternative reality for the clients. What we perceive as actual reality is just that which our sensory system has filtered and provided to our neurological processes for evaluation and interpretation. A vivid red to one person may be a dark gray to the color-blind. Deutsch has developed specific sound patterns which produce different effects on various listeners -- some hear a phrase ascending in pitch, while others listen to the same presentation but describe it as descending, for example.[DEU95] All reality is therefore virtual, in some sense. What is "real" then, is that which is presented in a "believable" fashion. Expressive participation, using imagery and sound, encourages a sense of other-worldliness. The technologies provide broader opportunities for clients to expand their imaginative skills, and the environments formed by these techniques bring a refreshing newness to the act of art creation.

In VAT therapy, variables are adjustable by the clinician. Clients and therapists, in a music setting, would be able to improvise with the rhythm that is being simulated, then shift to melodic improvisation with rhythmic elements, and then to dynamic elements (such as speed and volume) with rhythmic and melodic dimensions, and so forth. Artistic behaviors which have been latent or dormant in clients may more accurately be brought forth and developed during these improvisation sessions. Arts therapy often evokes deep-seated memories -- one vivid illustration of this appears in the research by Sachs with postencephalitic patients who, when they were "awakened" through drug therapy, were discovered to relate most strongly to music that had likely been heard near the time of onset of their illness.[SAC90] This "familiar" music was used as a launch-pad for subsequent communication. In the same way, positive "triggers" discovered in a VAT setting, even such things as a simple pitch or rhythm or musical phrase, can be recorded and re-introduced as appropriate to the therapy process.

Our Research

In our exploratory models, we have discovered that the VAT client offers a challenge to the system designer, often because of their special needs. Some patients have a limited range of motion, or they may be frightened or overly encumbered by head-mounted displays or other immersive devices. Successful techniques have involved our use of motion sensors, such as the Lightning [3] and Soundbeam [4] MIDI controllers. The Soundbeam module is a gestural device, requiring no hand-held equipment. The client simply moves through a region in space, activating the interface. Experiments at Immaculata College with a blind woman enabled her to expand and smooth her gestures, gaining a sense of freedom and fluidity of movement that her disability typically constrains (due to fear of collision with objects). Lightning uses a pair of wands to explore an imaginary grid in the air -- triggering sounds, images or whatever happens to be interfaced to the system. Research by Reilly [REI95] with severe psychiatric inpatients at the Albert Einstein Medical Center, demonstrated that this tool could be used with a tonal space to indicate or confirm diagnoses of mania versus depression. Clients who were depressed were found to restrict their use of the tonal space to a few notes, and even when encouraged to move to other portions of the sound grid, they would typically wander back to the small space they had selected. Manic clients, conversely, would wildly play many notes, and often go off-grid, where no sounds would be activated. Strong correlations were found for the manic and depressive disorders in terms of the variety of notes used, the amount of notes played, and the time spent outside of the grid. The advantage of these MIDI devices is that they allow expression in an intuitive fashion, using simple body movements that can be learned in a few minutes, thus permitting improvisational development (see Figure 1).

Other VAT experiments have included the use of the Jack® [5] Human Factors Modeling System from the University of Pennsylvania. This system is primarily intended for modeling and animation of human movement, but has been extended to permit virtual environment creation and interaction. Researchers Mercuri, Bhatnagar and Stokes have created SoundWorld, an interactive setting where musical instruments can be played (Figure 2). An early electronic synthesizer, the Theremin, was recreated -- this instrument permits control of sounds by volume and pitch, by waving one's hands over capacitance-related antennas -- hence, the virtual simulation could be activated using Soundbeam, Lightning or various non-gestural input mechanisms. A virtual marionette was also designed in the Jack system for use in "Punch and Judy"-style play acting.

The adaptation of metaphors as communication settings has been explored by various researchers. Seligman, Edmark and Laporte at Bell Laboratories have created the Metaphorium -- a place where real-world analogies can be used to facilitate unconventional web communications. They use the metaphors of a subway ride, the message in a bottle, sky-writing, and traces in the sand to enable intentional and random transactions among participants. Earlier work by Seligman, Mercuri and Edmark [SEL95] involved the development of virtual meeting rooms, interactive environments which enable multimedia services to be provided to users who rendezvous in settings as simple as a conference call, or as complex as a computer-generated 3D graphical and audio interface. This system incorporates sophisticated techniques such as self-modifying and animated intelligent objects, constrained viewers, and context-sensitive annotation. Participants in these meeting rooms can, for example, write on a document that is viewed by all present, they can hear sounds in directional orientations corresponding to the virtual setting, and they can use equipment such as CD players, VCRs, tape editors, computers, fax machines and so on. This system makes particular use of the holophonic sound service, a Convolvotron-based system [6] which provides audio assurance cues, indicating activities and events. Sampled sounds and simulated acoustics provide a sense of realism to the setting, enhancing attention of participants.

The concept of virtual metaphors were also used by Mosher in his Collaborationation (Figure 3), an interactive kiosk which explores the themes of community art, music performance, and social discourse. [MOS92] Here various aspects of the neighborhood -- the garage in which music is created, the porch on which conversations take place, the walls on which murals are painted -- are used to enable communications between participants of all ages. Senior citizens can view the virtual community from comfortable porches, where passers-by will feel welcome to stop and chat. Youngsters can play in rock bands, and all residents (who may be in actuality separated by vast distances) can work together on artworks that represent the feelings and emotions of the present moment. Facilitators can help guide the participants in developing themes and ideas relevant to the group. Mosher, Mercuri and Bhatnagar also created Fears (Figure 4), an interactive display based on artwork created by young teenagers in a comics class at the Community School of Music and Arts. Students were asked to sketch their fears -- which depicted such images as the death of family and friends, old women with long hair, snails, frightful creatures of the unconscious, grappling with chickens, and masked wrestlers -- these were then digitized and enhanced with 3D graphics and appropriate audio tracks. This concept could be used with individuals in conjunction with other therapies, thus creating an artistic portfolio that records and displays ongoing thoughts and feelings through the healing process.

Treating aging populations of adults is one of the most challenging and necessary tasks for therapists. As the population ages and accompanying problems of Alzheimer's and Parkinson's diseases infiltrate the society, so will health care services need to seek innovative ways of sustaining and maintaining the quality of life for these individuals. Rigidity of muscles, as well as gradual deterioration, weakness and slowness of movements, tremors, flexion of the head, trunk, and limbs, are physical manifestations of the aging process which limit the use of traditional arts therapy approaches. In addition to these problems, such patients often appear to have mask-like faces which manifest a lack of emotional response. This is not due to the fact that these persons are incapable of responding emotionally, but that their physical responses do not match their emotional ones due to the hardening of brain tissue or other areas of the nervous system which prevent them from reacting overtly. Thus, they respond emotionally from "within," although they wish to respond from "without." One can imagine the sense of frustration and depression which overcomes these clients during such experiences. Through VAT technology, these physical barriers can be partially overcome. Patients can evoke feelings through the use of a virtual face prosthetic, they can sing, and dance, and paint pictures using tools appropriate to their needs within an expanded time frame which allows them to develop and perform artistic creations.

The virtual setting induces a dream-like state similar to therapeutic structured visualization and hypnosis. Most people have had the experience of the "falling dream," where one believes that they are dropping over an endless cliff, and one feels the panic that ensues in trying to awake from this somnolent virtuality. Many others have also had the sensation of the "flying dream," where they believe that they can float in order to move from location to location -- this is typically a more pleasant feeling. Persons having long-term exposure to virtual environments describe their experiences in manners similar to those of the dream sensations. It is not yet determined whether VAT techniques could be successfully combined with subliminal audio or video suggestions in order to enhance the therapy process, particularly with individuals who are otherwise resistant to traditional hypnosis and visualization techniques.

Risks

The VAT setting is not without a variety of associated risks, each of which must be assessed and weighed as to their impact on the therapeutic process. The interaction between patients and therapists in a VAT session must be carefully monitored -- participants in a group setting must not be allowed to produce behaviors which could be perceived as harmful to the other members of the environment. Adequate controls must be provided to the therapist, or to the system, by the VAT designers, to allow for blocking of an individual's actions when deemed necessary. The therapist must be available to provide or insert the essential "reality checks" into the experience. Who shall have access to the sessions or their recorded forms also needs to be examined. The Internet and Web, although popular, are also highly insecure and should be used judiciously.

Other risks include the effects of long-term exposure to electromagnetics, along with hazards of computer use, including carpal tunnel syndrome, headaches, and other physical adverse reactions. Simulator sickness is a well-documented phenomenon, where disparities in displayed versus physically experienced motion can trigger vertigo or nausea.[MCC90] The use of geometric or pulsating displays have been found to induce epileptic reactions even in individuals who have never had any prior incidents of seizure. VAT designers must be advised to test their systems on a significantly large population prior to release into general use, in order to better gauge whether anomalous effects are likely to occur.

Ultimately, the therapist's goal should be to reunite people with the world, not to lure them out of it. The intention should not be to create Nintendo couch-potatoes in the name of therapy. It should be recognized, though, that the virtual world may be so attractive that a person may not wish to rejoin the society. Take, for example, the elderly person using a face-prosthetic to express emotions -- for such an individual, there may be some disappointment or even withdrawal symptoms experienced when reentering the real world where severe physical limitations are reimposed. This may just involve a momentary disorientation, like changing from a car with manual transmission to one with automatic, but therapists will need to work with these clients to see that they are able to make the transition between the virtual and the real world without difficulty. Those clients who wish to escape from reality through the use of hallucinogenic substances, or who are inwardly withdrawn (such as the autistic) by nature, might be highly attracted to VAT technology. Here they could "escape" without the adverse consequences associated with their disorder -- these patients must be heavily monitored to insure that the VAT does not become a mechanism that reinforces rather than reduces negative behaviors.

Conclusions

It appears, then, that despite these minor reservations, VAT provides a rich tool for therapists in their work. Methods for applying these techniques to the healing setting have only begun to be explored. The continued development of faster and more user-friendly equipment, along with new communication metaphors, will allow virtual environments to be more closely matched with client requirements. Cooperation among computer/engineering and health-care practitioners will enable systems to be produced that assist the therapeutic process in ways which could not be possible using traditional methods. Remote transactions will permit greater patient outreach. The arts therapy community, and even more importantly, the people they assist, all stand to benefit greatly from the enhanced setting that this virtual technology provides.

Acknowledgements

The authors wish to acknowledge the following individuals for their contributions and insights to the work presented in this paper: Norman I. Badler, Ranjit Bhatnagar and Brian Stokes of the University of Pennsylvania, Joseph Reilly of the Albert Einstein Medical Center, Dorée Duncan Seligman of Lucent Technologies, Michael Mosher of the Community School of Music and Art, and Samuel Provenzano of Keystone Studios.

Biographies

Rebecca Mercuri is completing a Ph.D. dissertation on simulated directional hearing at the University of Pennsylvania. She is also an Assistant Professor in Computer Science at The College of New Jersey, and a member of the graduate faculty at Immaculata College. Rebecca holds various degrees in Music, Computer Science and Engineering, and serves on the boards of the Association for Computing Machinery (Princeton section), the Audio Engineering Society (Philadelphia chapter), and the Delaware Valley Acoustical Society of America.

Sr. Jean Anthony Gileno is the chair of the Music Therapy Department at Immaculata College. She is a board-certified music therapist and holds a Ph.D. from Temple University, where her research involved investigation of musical achievement in neurologically impaired and normal children. Sr. Jean has collaborated with Ms. Mercuri on a number of projects involving Virtual Art Therapies -- these have included exhibits at the Franklin Institute Science Museum in Philadelphia, and the Museum of Natural History in New York. Sister has served on the board of the American Association for Music Therapy, and is also affiliated with the National Association for Music Therapy and the International Society for Arts and Medicine.

Figures

1. Lightning -- demonstration by music therapist Joseph Reilly and volunteer Roe Konopka at the American Museum of Natural History.

2. SoundWorld -- virtual band created by Ranjit Bhatnagar, Rebecca Mercuri, Angela Lai and Brian Stokes.

3. Collaborationation -- interactive kiosk by artist Mike Mosher.

4. Fears -- 3D display created by Mike Mosher, Stephanie Chen, Ivan Evans, Dan Raim, Alisa Shyr, with audio and graphical enhancements by Ranjit Bhatnagar, Rebecca Mercuri, and Samuel Provenzano.

Notes

[1] Samuel I, 16:23.

[2] Virtual Arts TherapyTM is a trademark of Notable Software, Philadelphia, PA.

[3] Lightning is a product of Buchla and Associates, Albany, CA.

[4] Soundbeam is a product of The Soundbeam Project, Norwich, England.

[5] Jack® is a registered trademark of the University of Pennsylvania, Philadelphia, PA.

[6] Convolvotron is a product of Crystal River Engineering, Fremont, CA.

References

[BEG90] Begault, Durand R., and Wenzel, Elizabeth M., "Techniques and Applications for Binaural Sound Manipulation in Human-Machine Interfaces," NASA Technical Memorandum 102279, Ames Research Center, August 1990.

[DEU95] Deutsch, Diana, "Musical Illusions and Paradoxes," Philomel, 1995.

[GIL90] Gileno, Sr. Jean Anthony, "Nursing, Healing, and Biofeedback in Music Therapy," Immaculata College, 1990.

[MCC90] McCauley, M., et al, "The Effects of Simulator Visual-Motion Asynchrony on Simulator Induced Sickness," AIAA Flight Simulation Technologies Conference, September 1990.

[MOS92] Mosher, Mike, "The Neighborhood, The Garage, The Porch: Healing Spaces of Home," First Conference on Cyberspace in Music Therapy, 1992 World Congress on Arts and Medicine, New York, NY, February 29, 1992.

[NJ95] State of New Jersey, Senate Resolution No. 79, introduced by Senator Codey, May 15, 1995.

[REI95] Reilly, Joseph F., LIGHTNING Strikes: A Correlational Study of Gesturo-musical Responses of Inpatients with Acute Manic or Depressive Symptomatology using the LIGHTNING module, Immaculata College, December 1995.

[SAC90] Sachs, Oliver W., Awakenings, Harper, NY, 1990.

[SEL95] Seligman, Dorée Duncan, Mercuri, Rebecca T., and Edmark, John T., "Providing Assurances in a Multimedia Interactive Environment," in the Conference on Human Factors in Computing Systems, CHI '95, Denver, CO, May 7-11, 1995.