Module 4: Assessment

 

Introduction


Tele-assessment requires prerequisite knowledge above and beyond providing tele-therapy services. Clinicians should be intimately comfortable and knowledgeable in 1) their clinical skills, 2) tele-behavioral services broadly, 3) face-to-face, traditional assessment skills and knowledge, and 4) tele-assessment skills and knowledge. This module will walk clinicians through some of the major points regarding the provision of tele-assessment services.

 

Tele-Assessment and Ethical Considerations


Tele-assessment is a multifaceted clinical service, and can look different depending on the set up of the clinic and the client population’s needs, resources, and accessibility. For example, tele-assessment can be a minor area of a larger assessment process, such as providing online rating scales or meeting on a tele-conferencing platform for interviews. Tele-assessment can also be the major component of testing, such as when a clinician administers standardized tests when the client is at home or is in a third-party location. These major forms of tele-assessment can also include testing aids or professionals who can be an assistant to the clinician and provide services to the client, such as providing them with materials or orienting the camera for optimal viewing by the clinician.

Ethical Components in Delivering Tele-Assessment

The following ethical components are taken from Bilder et al. (2020), the Joint Task Force for the Development of Telepsychology Guidelines for Psychologists (2013), McCord et al. (2020), Postal et al. (2021), Wright et al. (2020), and Wright & Raiford (2021).

Clinician’s Competency
  • Clinicians need to be fully trained, comfortable, and competent in administering, evaluating data, and evaluating limitations of the tele-assessment used.
  • If not competent, the clinician risks not aligning with the APA Ethics Code 2.01 Boundaries of Competence, 2.03 Maintaining Competence, and 9.07 Assessment by Unqualified Persons.
Tele-Assessment as an Appropriate Service for Clients
  • Certain client characteristics may make tele-assessment a poor fit. Some questions to ask include: 
    • Is the client high risk? Due to being physically removed from the client, clinicians should consider how burdensome it would be to make contact with a client who demonstrates risk, should an emergency happen.
    • What is the client’s level of literacy and comfortability with technology, and what is their level of access to technology? If a client feels uncomfortable using technology, their scores may not accurately reflect their potential and clinicians may inadvertently be assessing technological anxiety. Furthermore, if clients have spotty WiFi, the testing results may not be accurate.
    • Is the client accepting of tele-assessments methodologies? The client may not be comfortable with this relatively new area of service, and may question the validity of the results. 
  • The answers to these questions may contraindicate the use of this particular service. Clinicians should talk with their supervisor about client characteristics that might interfere or impact the use of tele-assessment.
Do No Harm Using Tele-Assessments
  • APA Ethics Code 3.04 details Avoiding Harm. There are several ways in which clinicians can inadvertently do harm to their clients in tele-assessment. Below are a few considerations and how to get around them.
    • Determine if the results are a valid representation of the client’s functioning, traits, and abilities. Could the results be skewed as a product of the environment or tele-health system?
    • Ensure online privacy and security- clinicians can protect their clients in two ways for this item; 1) choose and use a secure telehealth platform, taking steps to ensure security on the clinician’s side, and 2) inform clients about how to increase privacy and security on their end
    • Establish awareness that tele-assessment is suboptimal and further or repeated testing may be needed- providing this information at the beginning can set realistic expectations for clients to not be emotionally upset at the end of the process.
Informed Consent
  • Informed Consent is explicitly in the APA Ethics Code twice (3.04 and 9.03), and is quintessential to tele-assessment since it is a novel clinical service being provided. 
  • Ensure that the client knows and understands the tele-assessment process, factors that may impede the assessment, and limitations of the process 
  • Limitations must be explicitly stated to the client, such as  not being able to see the client’s entire body, or not being able to administer certain tests, measures, or tasks, and performance based tasks being less precise.

How to Set up the Session


Setting up the testing session is crucial to control the environment and ensure validity with the test results. Adapting tips from Galpin, Sikka, King, Horvath, Shipman, and the AAMC Telehealth Advisory Committee (2021), here are some considerations before, during, and after the testing sessions.

  • Set up approved telehealth software for service delivery, data collection, and communication
    • Training clinic directors likely have already thought about this and choose an appropriate software platform, but trainees can consider how to explain to clients what makes the platform secure, and how users can inadvertently limit privacy and confidentiality
  • Provide resources to client on the use of technology needed for telehealth services 
    • One helpful suggestion is to set up a case management session with the client before the start of any clinical service. Case management sessions can be 10-20 minutes to check that the client’s internet connect is sufficient, explain the informed consent and policies of the clinic, and describe issues of privacy and confidentiality as related to tele-health
  • Ensure your virtual and physical space is professional, confidential, and free from distractions
    • To protect your client’s confidentiality, counselors can orient themselves on a wall so that there will not be people walking in from behind. Counselors should also consider minimal trinkets and a solid background without distracting patterns
    • Utilize lighting and camera angle to ensure client can see you 
  • Make sure the client is located in the state in which you are able to provide services, and that they plan to be in the state for all of the testing. If this is not possible for them, tele-assessment may not be appropriate. Confirm their location beforehand and look up emergency resources such as the nearest hospital or emergency number, in case any medical or mental health emergency arises.
  • Clearly describe the purpose and objectives of telehealth service 
    • Example
      • Purpose: To continue therapy services even with traveling barriers
      • Objectives: Increase attention span in 30 minute interval 
  • Explain relevant risk, benefits, and use between in person or telehealth services
    • Risks
      • Ex) Confidentiality concerns
    • Benefits
      • Ex) Does not require travel or flexible scheduling
    • Use
      • Ex) An online software system rather than sitting face to face in person 
  • Use the client’s personal environment to inform evaluation, assessment, and treatment plan decisions. For example, if someone is using a shared family computer in the living room, they need to be aware of their lack of privacy and potential interference from others in the space.
  • Obtain emergency contact information from the client if an emergency arises. For example, contact information from another person in the house or nearby in case there is a medical or mental health emergency. This can be another responsible adult, a neighbor, significant other, or close family member. File this information in the electronic medical record system for safe keeping.
  • Clinicians should communicate with their supervisor about the case using approved technology and ensure that data collection or assessment tools are appropriately uploaded. This could include intake notes, protocols, and raw scores into online scoring software.
  • To maintain security, make sure electronic items are encrypted and/or password protected, and physical items are stored in a secure location.

 

Telehealth Microskills


Clinical microskills are necessary for providing clinical services, but are even more essential in providing tele-assessment. Some skills may need to be modified for the digital platform to build good rapport with your clients and make them feel at ease. The following is from Galpin et al. (2021), McCord et al. (2020), and Myers & Turvey (2013).

 

Body Language and Voice

In face-to-face clinical work, clinicians attempt to convey empathy and connection through use of eye contact, body positioning, and facial expressions. These may become lost and not translate well through an online platform. Here are some tips to alter them and convey that same warm you want:

Eye Contact

Make sure that you are looking at the camera of your computer to convey eye contact. Sometimes it is helpful to put a sticker or eyes by the camera to remind yourself to look there. Another tip can be to sit slightly more away from your camera than normal, so that there is more “room” for your gaze to move without it appearing as breaking eye contact.

Body Positioning

Make sure you take up the majority of space in your video box, and show your upper part of your body. You may need to adjust the camera so that it cuts off the very top of your hairline or head so that clients can see your torso. This can also help with modeling for the clients how you would like them to set up their own camera and video themselves.

Facial expressions

Seem interested in what they are saying and try to mirror their reactions. Because you will see yourself as well, consider covering that box up if it is distracting to you. You should adjust the lighting in the room to avoid unsightly shadows or backlighting on your face. Appropriate lighting of your face will allow for your facial expressions to be clearly seen.

Tone

Make sure to have a warm tone of voice similar to what you may use in an in-person session. It is easy for us to sometimes speak louder when on tele-platforms, so be sure to modulate your voice. Check the quality of your microphone. If the microphone causes your voice to come across as muffled or unclear, the tone of your voice will not be heard.

Flexibility and Establishing Rapport

  • Technology issues can arise at any time, so be sure that there are multiple options for communication, in case one fails. For example, if video calling is not possible, do you have the capabilities to switch to a phone call?
  • Establishing rapport includes reassuring that tele-assessment is a valid and reliable service delivery model. Reiterating the purpose of tele-health delivery can help ease the client’s concerns.
  • Clinicians can also establish rapport by responding to any changes to the client’s body language and emotions, and offering them choices when appropriate.

 

Testing Materials


Each test or measure comes with certain materials that either need to be presented to the client or the client needs to interact with to complete the task. Choosing materials wisely and distributing them to the client are two major considerations in tele-assessment.

 

Choosing Testing Materials

In traditional assessment, clinicians consider what the most appropriate test or measure is for the client based on their age, suspected disorder, and physical capability (Riverside Insights, n.d.). For tele-assessment, with a wide variety of options, there may be certain measures that are more appropriate for tele-assessment. For example, there are certain manipulatives or response booklets that would be required for standardized administration that may complicate the assessment process. As well, there may be certain manipulatives that would be better given with a professional monitoring, as opposed to with a client who is at home testing (Riverside Insights, n.d.). Think wisely and consult your supervisor about which tests may be most appropriate and still give you the test results you need. 

 

Distributing Relevant Testing Materials to the Client

Once test materials have been chosen, they need to be delivered or distributed to the client. Larger test companies such as Pearson and Riverside Insights have created digital materials for the more popular tests that can be shared with the client on screen. 

If there are physical materials such as manipulatives or response booklets, the clinician needs to ensure that 1) the materials arrive to the client in a secure manner, 2) the client does not look at or use them before or after they are supposed to, and 3) the client delivers the materials back to the clinician in a secure manner. To achieve this, a procedure for delivering materials if the client is conducting a tele-assessment at their home (Riverside Insights, n.d.; Wright et al., 2020).

First, clinicians should mail the materials to the client in two envelopes, such that when they open the initial envelope, the materials are still secure in a second one. Clinicians should tell the client to not open the materials until they are instructed to on camera. In the mailing package, clinicians are recommended to include an empty envelope that is already addressed and stamped, so that once the client has completed their testing, they can place the materials back in a new secured envelope and mail it to the clinician. Having the client open the materials, as well as seal the used materials, on camera allows the clinician to reasonably assume that the test materials were secure. 

 

Virtual Assessment


The administration of tele-assessment requires preparation before and after the testing session, to focus on aspects such as the clinician’s competence in assessment and technology, the physical environment of the clinician and client, and flexibility when problems arise. These points are adapted from Pearson (2020).

  • Practice administering assessments in a virtual format- novice clinicians may feel uncomfortable regarding the process of tele-assessment, so practice is key to work out nerves and obtain some muscle memory regarding juggling the many digital and electronic materials.
  • Teach any additional people who will be assisting in the assessment- other professionals or even parents may be pseudo-assistants for a tele-assessment session. Make sure these assistants understand their role in the assessment process and explicitly describe the boundaries to their ability to help. For example, a parent helping too much may skew the test results if they inadvertently provide an answer to a testing question.
  • Troubleshoot problems that may arise- Preparation ahead of the testing session may include brainstorming what could go wrong, and planning for those circumstances. What will you do if there is an internet connection issue? What if there is an administration error on your part? Talking through these scenarios with your supervisor may be helpful.
  • Make sure digital materials are viewable to the client- ensure that the client can adequately see the materials being presented and troubleshoot as needed. For example, does the clinician need to zoom into the materials, or does the client need to enlarge their viewer?
  • Only share necessary information with the client- the clinician may be using their computer to display information, so make sure there are not other applications or documents open that could be inadvertently viewed.
  • Follow the procedures for tele-assessment- certain tests and measures may have specific protocols or guidelines for administering tele-assessment, so make sure you are following the instructions as needed to ensure a valid assessment.
  • Ensure that the data collected matches the assessment- reviewing the test results, make sure that the results “make sense” in the context of the tele-assessment. If a score seems low or unusual, think back or go back to that time and consider if there were distractions or technology issues during that time. Explaining divergent findings in the context of tele-assessment can contextualize results.
  • Verify that data is stored in a secure location- much like how the clinician makes sure the client secures their testing materials, clinicians should also make sure any testing materials they have are secure. If the administration occurred in the clinician’s home or outside of the clinic, test materials should be stored in a secure location and transported as soon as possible back to the clinic.
  • Write any additional observations notes as needed- we hold so much in our mind, and observations can easily be forgotten over a period of time. Take some time out of your schedule after the testing session to review the session and write down any observations you may remember about the session.

 

 

Supervisor and Training Clinic Considerations


For supervisors or other psychologists in a training clinic, additional considerations may be appropriate for guiding trainees in tele-assessment.

  • Ethically consider what assessments trainees are competent in administering. Competency includes both face-to-face administration and tele-assessment administration.
  • Make sure trainees are well versed in talking about the benefits and limits of tele-assessment. Providing talking points or scripts may be helpful.
  • Trainees should be familiar with the protocols of the clinic regarding tele-assessment scoring and testing files. 
  • Supervisors may consider “testing” or having students “check out” their administration before working with clients. This allows the supervisor to be confident that the trainee is comfortable and confident administering over tele-assessment.
  • Direct observation is recommended for at least the first few times of administering tele-assessment measures. This will ensure valid and standardized administration, and the supervisor can be there if there are any initial problems. 
  • To encourage critical thinking, supervisors should incorporate multiple discussions of culture throughout the testing process. Specific timepoints can be at the initial intake, assessment planning, behavioral observations, case conceptualization, and report writing (including conceptualization and recommendations).

 

 

Diversity Considerations


Assessment and tele-assessment should always be considered with diversity and multicultural components. Here are a few considerations:

Historical Awareness and Cultural Considerations for Assessment (Dadlani et al., 2012; Guthrie, 2004; Wright & Raiford, 2021)

Many assessments were created to utilize overt racism and distinguish majority and minority groups. This is most obvious in the fact that the tests were normed on the knowledge and values of the majority groups. While test companies and the larger assessment research field appear to be more aware of this history and attempt to have more representative norm groups and research, this work is slow. Clinicians should keep this in mind and choose measures that are representative of and appropriate for their clients.

Culture is embedded in each test to a certain extent, with certain tests more culturally loaded (usually crystalized/comprehensive knowledge tests). When choosing a measure, clinicians can think about the client’s background and history with the normed group of the assessment, as well as their language proficiency. Examining measures can include looking into the language, wording, interpretations, and administration of the test. For example, consider a child who is a recent immigrant and an English language learner. Certain tests would be inappropriate because the child has not been exposed to certain cultural information, may struggle with language proficiency, and the results may not be tapping into the intended domains. 

Case conceptualization is when clinicians take all information and create a narrative to explain the diagnosis. Cultural components can easily influence data, so clinicians should be careful to not pathologize cultural phenomena. 

Clinician’s cultural awareness should include how clinicians view culture in the context of behavioral observations, case conceptualization, diagnosing, and recommendations. Self-reflection is the first step to cultural awareness, and how personal/professional biases may impact work with clients and conceptualization. Next, utilizing supervision will help the clinician find blindspots or gaps to understanding how identity and culture manifest in the testing. Finally, it is vitally important for a clinician to be open to talking about culture throughout testing with the client, starting with asking about it in intake questionnaires and interviews. This models to the client these conversations are welcomed and warranted in the assessment.

Tele-Assessment Cultural Considerations (APA, 2017; Dadlani et al., 2012; Van Dorn et al., 2020; Yun et al., 2010)

Specifically to tele-assessment, cultural considerations include accessibility, resources, and literacy.

  • Client’s socioeconomic status may prohibit access to certain locations and materials. For example, does your client have their own computer and a secure, stable form of internet? Do they have a private space where they will not be disturbed? For adults, do they have access and resources for childcare so they are not disturbed while taking the tele-assessment?
  • Socio-economic status is also often intertwined with race and ethnicity, so in addition to SES, clinicians can also consider how these accessibility issues relate to race and ethnicity.
  • Along the lines of accessibility, the level of technology literacy is another cultural component to consider. If clients do not have access to a personal computer or other technology, they may feel uncomfortable doing assessment digitally. As well, older clients who are not well versed in technology may feel similar. This discomfort with technology may make these clients inappropriate for tele-assessment services.

 

Feedback


Providing clients with the information about their testing and diagnosing in the feedback session can have certain considerations for tele-assessment. First, we describe the general feedback session information, and then some tips on tele-assessment feedback.

 

General Feedback Session (Dombrowski, 2020; Gass & Brown, 1992)

A general feedback session may include these components:

  • Preparing for the session in advance
  • Discussing the evaluation process and purpose of testing
  • Explaining the test results
  • Concluding the assessment process and providing diagnoses
  • Detailing the recommendations
  • Answering questions and addressing concerns

Aspects of Tele-Assessment Feedback (Lewandowski et al., 2020; Wright & Raiford, 2021)

  • Have knowledge of and be prepared to detail the technology that was used. A debrief on how the tele-assessment was done may once again ease the client’s anxiety and make them feel like the assessment and results were valid
  • Be prepared to implement screen sharing- although you are not engaging in formal tele-assessment testing, you may need to share your screen at points 
  • Prepare to share and provide virtual handouts- these can include the normal curve distribution, a summary of the results, or preliminary recommendations
    • If the tele-platform allows, the clinician can also provide the client with specific website URL hyperlinks to give them instant access to some of the recommendations, such as to psychoeducational sources or other provider referrals.
  • Check in on the client throughout the session for their emotional state and any questions that may arise. When a client becomes emotional, implement microskills and psychoeducation.
  • Be transparent about the novel circumstances in which the assessment was conducted, alterations made to the assessment due to circumstance, and how this could have implications on data interpretation.

 

Case Study Walkthrough


The following presentation will focus on a case study walkthrough related to tele-assessment. Throughout the video, there will be questions presented and you are asked to pause the video and consider how you may answer these questions.

Video Example- Building Rapport in Intake and Feedback Sessions

The following video is an overview of an intake session and feedback session, all focused on building and maintaining rapport. The video stops at times to summarize what happened and allow you to reflect on what you saw or other things you can consider.

For the intake session, the video demonstrates a trainee talking with a client for the first time. It shows some basic conversations regarding how to introduce telehealth, how to talk about privacy and confidentiality, how to navigate emergency procedures, and talk about consent forms. 

In the feedback session, the video shows the trainee walking through some of the results with a normal curve being displayed. The video also demonstrates conversations that can occur if a client obtains a diagnosis that they weren’t expecting.

References


American Psychological Association. (2017). Ethical principles of psychologists and code of conduct (2002, amended effective June 1, 2010, and January 1, 2017). Retrieved from https://www.apa.org/ethics/code/.

American Psychological Association. (2017). Ethnic and Racial Minorities & Socioeconomic Status. Retrieved from https://www.apa.org/pi/ses/resources/publications/minorities.

Bilder, R. M., Postal, K .S., Barisa, M., Aase, D. M., Cullum, C. M., Gillaspy, S. R., Harder, L., … & Woodhouse, J. (2020). Inter organizational practice committee recommendations/guidance for teleneuropsychology in response to the COVID-19 pandemic. Archives of Clinical Neuropsychology, 35, 647-659. doi: 10.1093/arclin/acaa046

Dadlani, M. B., Overtree, C., & Perry-Jenkins, M. (2012). Culture at the center: A reformulation of diagnostic assessment. Professional Psychology: Research and Practice, 43(3), 175. doi: 10.1037/a0028152.supp

Dombrowski, S. C. (Ed.). (2020). Psychoeducational assessment and report writing. Springer Nature. doi: 10.1007/978-3-030-44641-3_3

Ford, D. Y. (2004). Intelligence testing and cultural diversity: Concerns, cautions, and considerations. National Research Center on the Gifted and Talented.

Galpin, K., Sikka, N., King, S. L., Horvath, K. A., Shipman, S. A., & AAMC Telehealth Advisory Committee (2021). Expert Consensus: Telehealth Skills for Health Care Professionals. Telemedicine Journal and e-Health: The Official Journal of the American Telemedicine Association, 27(7), 820–824. 

Gass, C. S., & Brown, M. C. (1992). Neuropsychological test feedback to patients with brain dysfunction. Psychological Assessment, 4(3), 272. doi: 10.1037/1040-3590.4.3.272

Guthrie, R. V. (2004). Even the rat was white: A historical view of psychology (2nd edition). Pearson Education.

Postal, K. S., Bilder, R. M., Lanca, M., Aase, D. M., Barisa, M., Holland, A. A., Lacritz, L., … & Salinas, C. (2021). InterOrganizational practice committee guidance/recommendation for models of care during the novel coronavirus pandemic. The Clinical Neuropsychologist, 35(1), 81–98. doi: 10.1080/13854046.2020.1801847

Joint Task Force for the Development of Telepsychology Guidelines for Psychologists (2013). Guidelines for the practice of telepsychology. American Psychologist, 68(9), 791-800. doi: 10.1037/a0035001

Lewandowski, R., Stratton, A., Gupta, T. S., & Cooper, M. (2020). Twelve tips for OSCE-style Tele-assessment. MedEdPublish, 9. doi: 10.15694/mep.2020.000168.1

McCord, C., Bernard, P., Walsh, M., Rosner, C., & Console, K. (2020). A consolidated model for telepsychology practice. Journal of Clinical Psychology, 76(6), 1060-1082. doi: 10.1002/jclp.22954

Myers, K., & Turvey, C. L. (Eds.). (2013). Telemental health: Clinical, technical, and administrative foundations for evidence-based practice. Elsevier.

Pearson (2020). Staying connected through telepractice. Retrieved October 22, 2021, from https://www.pearsonassessments.com/professional-assessments/digital-solutions/telepractice/about.html. 

Reynolds C. R., Altmann R. A., & Allen D. N. (2021). The Problem of Bias in Psychological Assessment. In: Mastering Modern Psychological Testing. Springer, Cham. doi: 10.1007/978-3-030-59455-8_15

Riverside Insights (n.d.). Tips for remote assessment using Riverside Insights clinical products. Retrieved from https://riversideinsights.com/home.

Van Dorn, A., Cooney, R. E., & Sabin, M. L. (2020). COVID-19 exacerbating inequalities in the US. Lancet (London, England), 395(10232), 1243. doi: 10.1016/S0140-6736(20)30893-X

Wright, A. J., Mihura, J. L., Pade, H., & McCord, D. (2020). Guidance on psychological tele-assessment during the COVID-19 crisis. Retrieved from https://www.apaservices.org/practice/reimbursement/health-codes/testing/tele-assessment-covid-19 

Wright, A. J., & Raiford, S. E. (2021). Essentials of psychological tele-assessment. John Wiley & Sons.

Yun, K., Lurie, N., & Hyde, P. S. (2010). Moving mental health into the disaster-preparedness spotlight. New England Journal of Medicine, 363(13), 1193-1195. doi: 10.1056/NEJMp1008304.