Speeding Up the VB-MAPP

Streamlining Administration, Scoring, Reporting and Goal Generation with Online Tools

Speeding up the VB-MAPP 2

Abstract

The VB-MAPP assessment is frequently used by behavior analysts in their endeavors to teach valuable communication skills to children with Autism Spectrum and related disorders. Tools as comprehensive as the VB-MAPP can, however, pose challenges in implementation. In particular, the VB-MAPP is noted to take several hours to complete, and not all of this time is reimbursed. With online tools, there are opportunities to facilitate the use of assessments such as the VB-MAPP.

CentralReach, a cloud-based practice management and clinical documentation and data software solution, conducted a survey of nearly 2000 VB-MAPP users to understand how they use VB-MAPP today, the challenges they face, and their

perspectives on leveraging online technology to support administration, scoring, reporting, and goal development.

Our Challenge: An Incredible, but Time-Consuming Tool

The Verbal Behavior - Milestones Assessment & Placement Plan (VB-MAPP; Sundberg, 2008) is one of the the most informative tools on the market for assessing the verbal behavior skills of children with Autism Spectrum and related disorders (Gould et. al., 2011). Based on Skinner’s Verbal Behavior (1957), the assessment builds upon a solid research base supporting the use of a behavior analytic approach to language as functional communication. Supporting evidence of reliability and validity can be found in several studies (c.f., Esch, et. al., 2010; Dixon, et. al., 2015).

Speeding up the VB-MAPP centralreach.com3

The VB-MAPP is divided into five components: 1. Milestones assessment2. Barriers assessment3. Transition assessment4. Task analysis and skills tracking5. Placement and IEP goals

The assessment components are intended to be used in a repeated fashion, both to obtain baseline levels and to track progress. The final two components are used to aid in forming interventions (Sundberg, 2008).

With so many features to offer, it is no surprise that the VB-MAPP has been widely embraced by behavior analysts. However, it can be time consuming to administer and requires a labor-intensive goal development process. In a field already characterized by hours of unreimbursed activities (cf., Sundberg, 2016), assessments as thorough as this one can certainly benefit from a technology boost. This is important for several reasons:

1. When designed appropriately, tech applications can reduce the time required to complete various components of assessment tasks.

2. Opportunities for administration and scoring errors are decreased.

3. Using technology to organize and streamline assessment activities can decrease clinician frustration and increase job satisfaction.

“In a field already characterized by hours of unreimbursed activities, assessments as thorough as this one can certainly benefit from a technology boost.”

Research on the use of technology to improve assessment processes in healthcare and education has demonstrated that, with appropriate design and effective user training, time savings can be found by reducing administrative burden (Muehlhausen et. al., 2015); improving the ease of administration and scoring processes (Vallejo et. al., 2007); and reducing time spent in validation processes for missing or erroneously reported data. (Zbrozek et. al., 2013).

Technology does more than save us the time and frustration of cumbersome and duplicative data entry. It also helps to substantially reduce opportunity for scoring calculation and conversion errors. Additionally, algorithms can be created to ensure all possible comparisons are made, thereby enhancing the thoroughness of the report. Algorithms are available to handle common scoring problems such as missing

Speeding up the VB-MAPP 4

families. On the other hand, it is time consuming to administer and report upon. Having no linked curriculum, it also creates additional work in the form of creating goals and teaching plans. Current support tools are available on the market to guide administration, scoring, and reporting; but our survey of nearly 2,000 VB-MAPP users indicated concerns about the adaptability and efficiency of existing tools, and a continued reliance on paper-and-pencil scoring.

CentralReach’s VB-MAPP Survey Results

Administration & Scoring According to Chart 1, the majority of respondents use time-consuming manual processes such as paper-and-pencil or multi-step spreadsheet-to-document input. Fifty-nine percent of the respondents indicated administration and scoring tasks take between three to six hours to complete.

data; parameters can be set to block data entry errors such as invalid values (Zbrozek et. al., 2013). Moreover, basic problems that arise from the use of paper and pencil are eliminated entirely (e.g., illegible handwriting, entering values in the wrong place, etc.; Vallejo et. al., 2007).

“Technology does more than save us the time and frustration of cumbersome and duplicative data entry. It also helps to substantially reduce opportunity for scoring calculation and conversion errors.”

Studies of job satisfaction and burnout in mental health fields have shown a strong correlation with factors such as large caseloads, time pressures, and inadequate resources (Morse, et. al., 2012). Likewise, support has been found for the idea that, given appropriate design and effective end-user training, technology enhances both job effectiveness and satisfaction (Selway & Pilkington, 2002). Having more work than time is a demotivating factor for the conscientious employees that organizations seek to keep, and technology can help balance the load.

The VB-MAPP is the type of tool that brings forth a dilemma for conscientious behavior analysts. On the one hand, it provides a wealth of information for making a real difference in the lives of clients and their

Chart 1

Speeding up the VB-MAPP centralreach.com5

Report Writing When it comes to report writing, 79% of respondents indicated that the task takes one to four hours, with 87% of all respondents using self-created templates or

writing reports from scratch each time. As seen in Charts 4 and 5, ease and affordability continue to be important factors in choosing a method of report writing. The ability to customize is also important to VB-MAPP users.

Chart 2

Chart 3

Chart 4

Chart 5

Respondents were also asked about the features that were most important to them in selecting administration and scoring methods. Charts 2 and 3 clearly show that time and ease of use are of paramount importance.

Speeding up the VB-MAPP 6

Goal Development An additional challenge relates to the aforementioned labor-intensive goal development process. There are currently no commercially available tools that link goals directly to VB-MAPP items. Although, a significant portion of survey respondents indicated they use templated goals, most endorsed starting from scratch for each client (see Chart 6). 80% of survey respondents indicated spending one to four hours on goal development alone. Again, time, affordability, and ease of customization were the leading factors in satisfaction with goal development tools (see Charts 7 and 8).

The complexity of the administration, scoring, reporting, and goal development processes inherent in existing options for VB-MAPP use is costly. While survey respondents reported spending at least six hours start-to-finish, many state reimbursement plans have rules that will limit reimbursement to $400 or less per assessment (c.f., Florida Agency for Health Care Administration, 2017; Colorado Department of Health Care Policy and Financing, 2017).

When compared to the per-unit rates a provider can make utilizing other, less informative assessments, use of the VB-MAPP is considerably disincentivized.

Chart 6

Chart 7

Chart 8

Speeding up the VB-MAPP centralreach.com7

The CentralReach VB-MAPP does all of the following

• Guides you through administration.

• Gives scoring hints.

• Calculates summary scores.

• Plots the scores.

• Automatically generates a report.

i. This report is customizable.

ii. Sections of the report you will typically want to customize are highlighted, so you can find it easily.

• Automatically creates goals based on the VB-MAPP scores.

iii. Pulls zeros and half point scores for you

iv. These goals are fully customizable

v. You can activate the ones you want, and save others for later.

Speeding up the VB-MAPP centralreach.com8

The CentralReach Solution

As the opportunity to streamline the VB-MAPP tasks presented itself, CentralReach listened to survey respondents and rose to the challenge. In partnership with the author of the VB-MAPP himself, Mark L. Sundberg, CentralReach developed a complete electronic version of the VB-MAPP, designed to save clinicians time and money while making the powerful tool less daunting and more accessible.

With CentralReach’s digital VB-MAPP, identifying client skill deficits has never been faster and more efficient! You will be able to pair the great, electronic power of VB-MAPP with the other wonderful clinical tools CentralReach has to offer (electronic data collection, advanced goal analysis, progress reporting), all on ONE cloud-based platform.

As our survey results verify, clinicians sometimes worry that goals yielded by goal banks will lack the level of individualization to adequately address their client’s needs. CentralReach cares about this, too. For that reason, each goal can be customized at all levels. When a goal is suggested, it is pre-populated with commonly used metrics. Using a linear progression, clinicians can modify each goal element, including (but not limited to):

1. The behavior to change

2. The behavior definition

3. The direction of change

4. The goal rate

5. The subgoal increments

6. The instructional settings

7. The prompt types and levels

Additionally, clinicians can add their own goals if the goal bank does not yield exactly what is wanted.

Speeding up the VB-MAPP 9

Benefits of the VB-MAPP on CentralReach: Behavior Analyst ScenarioBecause CentralReach offers an all-in-one client information platform for behavior analysts, incorporating assessments such as the VB-MAPP provides several advantages to clinicians. First, it eliminates time lost to shuffling between different record formats. Consider this typical experience:

Nita works as a Behavior Analyst in a multi-disciplinary clinic for children with Autism Spectrum Disorders. The clinical psychologist in the practice recently diagnosed 4-year-old Jamal and referred him to Nita for intervention to build communication skills. Upon receiving the referral, Nita also received a paper copy of the psychologist’s report (which included diagnosis, test results, and broad goal recommendations), and the intake papers completed by Jamal’s mother prior to testing.

After reading this, she wanted to speak to Jamal’s mother prior to his arrival to ensure she had reinforcing items for him. Contact information was available in the clinic’s practice management software program. After speaking with Jamal’s mother, she wanted to record the identified reinforcers

in a place where she could maintain and update the list frequently. She created a Google Doc to do this.

On the day of Jamal’s initial visit, Nita used the practice management software to create a note for billing purposes. As she conducted her interview with Jamal’s mother, she placed more extensive notes into a google doc, which she later converted into a formatted report to share later with Jamal’s interventionists. To record behavioral observations on Jamal, she used an app she likes on her iPad. As she administered the VB-MAPP over the next several sessions, she utilized the paper protocol, keeping several different running lists of tacts, mands, etc., observed both spontaneously and with prompting in the small amount of space available for doing so. When she ran out of space on the protocol, she began attaching post-it notes to the protocol to track his word lists.

When the VB-MAPP administration was complete, she consolidated all of the lists into another Google Doc and filled out an Excel spreadsheet to record his levels and code his chart. She then wrote goals based on the information gathered in the VB-MAPP assessment, the interview with his mother, and the recommendations made by the

Speeding up the VB-MAPP centralreach.com10

psychologist. In so doing, she switched between two Google Docs, three paper files, an iPad app, and the clinic’s practice management software platform.

If Nita had been using CentralReach, she could have opened Jamal’s file and seen his intake forms and psychological report. At the click of a button, she would have found his mother’s phone number. Using a custom contact form, she could have collected and recorded information about Jamal’s reinforcers as she spoke with his mother on the phone. At the first appointment, she could have entered any new information collected through interview into a note, while simultaneously generating the information needed for billing insurance. She also could have collected real-time data while observing Jamal. When it was time to administer the VB-MAPP, she would have remained on the CentralReach platform for administration, then allowed the system to automatically generate scores, charts, and reports. She would have then chosen from a list of system-recommended goals, customized them for Jamal, and added others from scratch if she wanted. In so doing, she would have never needed to leave the CentralReach system.

Conclusion

CentralReach strives to be the best all-inclusive platform for behavior analytic service providers and interdisciplinary practices. By streamlining multi-step processes with a system robust enough to automate frequently used rote processes, yet flexible enough to allow users to customize how those processes are handled, CentralReach saves you time and money and allows your practice the freedom to focus on growth. Our success in cutting down the time and effort required to set up program books that reflect best practices have readied us for applying the same skill set to saving you time and effort administering, scoring, charting, reporting, and establishing goals from your VB-MAPP assessments.

Interestedin Learning More?

Email Us

Speeding up the VB-MAPP 11

References

Colorado Department of Health Care Policy and Financing. (2017). Pediatric behavioral therapies. Available: https://www.colorado.gov/pacific/hcpf/pediatric-behavioral-therapies.

Dixon, M. R., Belisle, J., Stanley, C., Rowsey, K., Daar, J. H., & Szekely, S (2015). Toward a behavior analysis of complex language for children with autism: Evaluating the relationship between PEAK and the VB-MAPP. Journal of Developmental and Physical Disabilities. 27, 223-233.

Esch, B. E., LaLonde, K. B., & Esch, J. W. (2010). Speech and language assessment: A verbal behavior analysis. The Journal of Speech-Language Pathology and Applied Behavior Analysis, 5, 166-191.

Florida Agency for Health Care Administration. (2017). Community behavioral health services fee schedule. Available: https://ahca.myflorida.com/medicaid/review/Reimbursement/Community_Behavioral_Health_Services.pdf.

Gould, E., Dixon, D. R., Najdowski, A. C., Smith, M. N., & Tarbox, J (2011). A review of assessments for determining the content of early intensive behavioral intervention programs for autism spectrum disorders. Research in Autism Spectrum Disorders, 5(3), 990-1002.

Morse, G., Salyers, M. P., Rollins, A. L., Monroe-DeVita, M., & Pfahler, C. (2012). Burnout in Mental Health Services: A Review of the Problem and Its Remediation. Administration and Policy in Mental Health, 39(5), 341–352. http://doi.org/10.1007/s10488-011-0352-1

Muehlhausen, W., Doll, H., Quadri, N., Fordham, B., O’Donohoe, P., Dogar, N., & Wild, D. J. (2015). Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health and Quality of Life Outcomes, 13, 167. http://doi.org/10.1186/s12955-015-0362-x

Selwood, I. & Pilkington, R. (2005). Teacher Workload: Using ICT to Release Time to Teach. Educational Review, 57 (2), 163-174.

Skinner, B. F. (1957). Verbal behavior. New York: Appleton-Century-Crofts.

Sundberg, D. (2016). The high cost of stress in the ABA workplace. Bsci21 blog. Available: http://www.bsci21.org/the-high-cost-of-stress-in-the-aba-workplace/

Sundberg, M. L., & Michael, J. (2001). The value of Skinner’s analysis of verbal behavior for teaching children with autism. Behavior Modification, 25, 698-724.

Sundberg, M. (2008). The Verbal Behavior Milestones Assessment and Placement Program (2nd ed). Concord, CA: AVB Press.

Vallejo, M. A., Jordán, C. M., Díaz, M. I., Comeche, M. I., & Ortega, J. (2007). Psychological Assessment via the Internet: A Reliability and Validity Study of Online (vs Paper-and-Pencil) Versions of the General Health Questionnaire-28 (GHQ-28) and the Symptoms Check-List-90-Revised (SCL-90-R). Journal of Medical Internet Research, 9(1), e2. http://doi.org/10.2196/jmir.9.1.e2

Zbrozek A, Hebert J, Gogates G, et al. Validation of electronic systems to collect patient-reported outcome (PRO) data - Recommendations for clinical trial teams: Report of the ISPOR ePRO systems validation good research practices task force. Value Health 2013:16:480-9.