36
Accessibility Metrics Ted Drake and Sagar Barbhaya Intuit Accessibility CSUN 2017 Accessible version: http://www.last- child.com/a11y-data-metrics/

Accessibility metrics Accessibility Data Metrics and Reporting – Industry Best Practices

Embed Size (px)

Citation preview

Accessibility MetricsTed Drake and Sagar Barbhaya Intuit Accessibility CSUN 2017 Accessible version: http://www.last-child.com/a11y-data-metrics/

What Do You Need?Write it on a post it note

Three Stages

1. Accessibility Management Goals

2. State of the Industry

3. Solutions to fill gaps

Current State• Manual Testing and Evaluations

• Continual improvements

• Documentation

• Community Outreach

• Stakeholders

• Centralized Accessibility Leadership

What we have• Product database with evaluations, scores,

stakeholders, and documents

• Bug Tracking

• Scattered Employee Resources

• Centralized documentation

• Non-integrated automation

What we need• Dashboards to surface data and trends

• Connections between platforms

• Automation and unit testing for developers

• Shared test results

• Connected resources

• Support channel integration

Future Goal• Measurable lifecycle from tests/evaluation to

production

• Deep consumer feedback integration

• Diverse workforce with support channels

• Developer ownership of testing and solutions

• Product ownership of process and success

State of the Industry

How are we doing? - Product Manager

What to Measure?

Product Quality Metrics

• Issues by Priority – 77%

• Issues by WCAG category – 54% • last time product had a manual evaluation – 46%

• Closed Captioning support – 38% • Number of accessibility bugs opened per month/

year – 31%

Priority Levels

• Impact on Customer

• Template / Module / Unique

• Density of Issues

• High Traffic, Key Flows

Test to Ticket flow

Company Metrics•AODA compliance – 55% •Number of media inquiries about product/company

accessibility – 45% •Conference talks by employees on accessibility – 45%

•Product VPAT Coverage – 45% •Compliance towards 21st Century Communications Act

(CVAA) – 36%

Compliance and Legal

Human Resources

• Section 503: Self-Identification

• Accommodations Requests

• Ergonomic Evaluations

• Service/Perk Requests

Customer Metrics•Usability participants with a disability – 63%

•Feedback via customer support – 50%

•Usability activities, including user personas,

with disabilities – 38%

•Customer support calls - 38%

•Tickets filed by customer support – 38%

Net Promoter Score

Storage and Sharing

QuickBase

Spreadsheets

Pa11y Testing + Dashboard

Google Docs

Cloud Storage• DropBox

• Box

• Amazon Web Services

• Google Drive

• Slack

WorldSpace

Solutions

Bug Tracking

JIRA

JIRA Queries

• labels = accessibility

• labels=accessibility and status=Open

• labels=accessibility and status = blocked

• labels=accessibility and status=Closed and resolved > “-4w”

DOMO

Integrated Testing

• Developer testing: Tenon, aXe, Nemo

• Unit tests include accessibility

• Export test results to JIRA

• Tracking tool usage

Finding Outliers

• Extremely critical to prioritize

• Outliers need to be identified and fixed first

• Example - Turbo Tax in season production bugs

Customer Support Logs

• Search for Deaf and HOH customer requests: [ LEVEL:SENTENCE TYPE:KEYWORD {deaf, hearing}] [ LEVEL:SENTENCE TYPE:AND_KEYWORD {capability, impaired, challenged, hard}]

• “Accessibility” topic within Salesforce for reports

• Microsoft: Dedicated Support Channel

What Did You Need?

Questions?