1

Review of Government Agency Surveys and Interviews

Terry Weaver

Joy Gatewood-Fulton

2

The Accessibility Forum: Interviews

Interviews with Federal Agencies regarding the Procurement Challenges of Section 508: acquiring accessible Electronic and Information Technology.

3

Section 508 - Interviews with Government Procurement (1 of 2)

• Nine agencies interviewed: from Depts. of HHS, Commerce, Treasury, and Defense.

• Purpose: to gather best practices in procuring accessible E&IT and to determine if agencies need further guidance in implementing Section 508.

4

Section 508 - Interviews with Government Procurement (2 of 2)

• Interviews were Confidential, with data presented in an aggregate form only.

• Range– Section 508 applicability to a purchase

– VPAT’s

– What 508 information do they require from vendors

– Comments

– Other

5

Implementing Section 508 - Question One

How do you determine whether or not Section 508 applies to a particular procurement?

– 44% Section 508 applies to all E&IT procurements

– 56% We have tools and policies to advise of appropriate determinations and when it is not applicable

6

Implementing Section 508 - Question Two

How do you document the results of your determination (whether or not Section 508 was applicable)?

– 22% - No formal documentation process

– 33% - An in-house tool leads users to the appropriate form or checklist that is used to document the determination and decision, including any exemption

– 56% - Other

Other includes: Reports, Templates, Procurement Requests, VPAT’s ,or other vendor supplied information

7

Implementing Section 508 - Question Three

How do you determine which parts of the 508 standard apply to a purchase?

– 44% - The requiring official determines which applies

– 56% - Other:

• Guidance from the organization's Section 508 committee

• Often it's a matter of interpretation and is ultimately the Contracting Officer's call

• The standard is applied equally across the board

• The in-house tool guides the user. Weighted standards are even possible (one provision can be more heavily weighted than another can)

• We rely on the vendor to provide us that information

8

Implementing Section 508 - Question Four

What process or procedure do you follow to determine whether or not vendor products, deliverables, and services meet the requirements of Section 508?

– 33% - Vendors are required to self certify that products meet Section 508

– 22% - Market research which may include VPAT's

– 44% - A combination of both

9

Implementing Section 508 - Question Five

What Information do you require of Offerors / Vendors?

– 33% - Self-Certification Language

– 33% - VPAT/template

– 22% - Vendor must complete our template

– 11% - Vendor must offer information about how their products meet Section 508 standards somewhere on their website, in brochures or handouts, in a standard template or VPAT format

10

Implementing Section 508 - Question Six

How often do you use the Voluntary Product Accessibility Template (VPAT) in your assessment of a vendor’s products, deliverables, and services?

– 44% - Never

– 22% - Always

– 33% - Sometimes, when available

11

Implementing Section 508 - Question Six-A

Is the VPAT the primary source of information?

– 44% - N/A - never use

– 44% - Yes

– 11% - No

12

Implementing Section 508 - Question Seven

What other sources of information do you use?

– 44% - Market research

– 33% - N/A

– 11% - We also may test the product

– 11% - Our own template

13

Implementing Section 508 - Question Eight

If the VPAT is used, we would like to follow up with some questions on how well it is working for you.

What do you like about it?

– 44% - N/A

– 33% - Everything - easy to read and all the information is on one page

– 22% - Nothing

14

Implementing Section 508 - Question Nine (1 of 2)

What do you dislike about it? (VPAT)

– 44% - N/A

– 22% - Nothing

– 56% - Other

15

Implementing Section 508 - Question Nine (2 of 2)

Comments included:– Too complex

– Ambiguous

– Vendor simply states "meets" to every criteria - we ask the vendor to customize a VPAT that states how their product or service meets the criteria

– Language is too technical for most requiring officials

– People get confused and don't know what parts apply

– Vendors often complete the VPAT’s incorrectly

16

Implementing Section 508 - Question Ten

The next few questions deal with your opinion of the quality of information provided by the VPAT.

Have you had to go back to the vendor with questions about the information provided?– 33% - Yes

– 22% - No

– 44% - N/A

17

Implementing Section 508 - Question Eleven

How many times? (Did you have to go back to the vendor regarding the VPAT)

– 11% - Don’t Know

– 22% - Many

– 67% - N/A

18

Implementing Section 508 - Question Twelve

The next few questions deal with your opinion of the quality of information provided by the VPAT.

How difficult was the VPAT to use? (On a scale 1 to 5 where 1 is not difficult at all and 5 is extremely difficult)– 33% - Not Difficult

– 11% - Difficult

– 56% - N/A

19

Implementing Section 508 - Question Thirteen

The next few questions deal with your opinion of the quality of information provided by the VPAT.

Did the VPAT save you time?

– 33% - Yes

– 11% - Not

– 56% - N/A

20

Implementing Section 508 - Question Fourteen

How much confidence do you have in the accuracy of the information provided by the VPAT? (On a scale of 1 to 5 where 1 is little or no confidence and 5 is full confidence)– 56% - N/A

– 22% - Full Confidence

– 11% - Fairly Confident

– 11% - No opinion

21

Implementing Section 508 - Question Fifteen

Do you perform any technical evaluations of the products, deliverables, and services themselves to determine 508 compliance?

– 56% - No

– 44% - Yes

22

Implementing Section 508 - Question Sixteen

After a product is delivered, are its 508 compliance claims verified?

– 78% - No

– 11% - Yes

– 11% - Don’t Know

23

Implementing Section 508 - Question Seventeen

Does your agency conduct 508 training?

– 100% - Yes

24

Implementing Section 508 - Question Eighteen

To who is the training provided (who is the principal audience)?

– 56% - All employees

– 11% - Contract Force

– 11% - Program Officers

– 11% - ROs, COs, COTRs, Legal Staff

– 11% - ROs, COs, COTRs

25

Implementing Section 508 - Question Nineteen (1 of 2)

Do you think there is any area where additional training is required?

– 44% - No

– 56% - Yes

Comments:– Yes, eventually we would like to spread out to the

program offices.

– Yes, more should be done for general employees since anyone can be an RO at any time. May explore doing this as part of the new employee orientation.

26

Implementing Section 508 - Question Nineteen (2 of 2)

More Comments– Make this training ongoing and part of a regular

awareness programs.

– Many RO’s need training tailored to the products they are responsible to buy for the agency. The current training tends to handle the subject too broadly without enough hands-on examples. Small and disadvantaged businesses as well as value added and reseller vendors don’t understand 508.

– Future surveys will help us determine this.

27

Implementing Section 508 - Question Twenty

What could raise your level of confidence in what you’re buying meets 508 requirements?

Another way to ask this question might be “What information or other resources are lacking or would be helpful to you in determining whether or not an offeror can meet the requirements of Section 508?”

44% - Some sort of logo, icon or other labeling program to confirm that the product is compliant would help. Like the UL laboratories icon on electrical products. If the products were certified as Section 508 compliant by an impartial 3rd party testing lab.

28

Implementing Section 508 - Question Twenty-one

Do you have any examples of particularly good vendor responses regarding 508 requirements?

(Could you provide us with a copy of anything that have been or can be sanitized and would represent good examples)?

– 33% - No

– 22% - Dell

– 11% - We have an database of customized VPAT’s, but they were proprietary to each purchase

– 11% - Oracle

– 11% - Acrobat and Dell have good VPATs

– 11% - Ricoh, Canon, and Sharp

29

Implementing Section 508 - Question Twenty-two

Have you made any purchases where the award decision has hinged on 508 requirements?

If yes, what kinds of products and services were purchased?

– 78% - No

– 22% - Don’t Know

– 11% - YesWe recently needed to buy thousands of printers and one vendor said that they could not provide printers that were compliant. We approached another vendor who provided printers that are compliant. We have had similar experiences with copying machines.

30

Implementing Section 508 - Question Twenty-three

What is your biggest challenge in doing a 508 compliant purchase? (What would make your job easier in dealing with Section 508)?

– 22% - Lack of dedicated resources (money and staff) to do justice to the 508 requirements

– 22% - Simply having a bigger pool of 508 compliant products

– 22% - What would be really helpful would be a seal of approval from an agency or independent body as to how well a product or service met Section 508 requirements. The example of the Green Star Energy logo used by EPA comes to mind.

31

Agencies Best Practices Pages (1 of 2)

• DOE:http://www.eren.doe.gov/websitestandards/508.html#B

• NOAA:http://www.nws.noaa.gov/sec508/

• ARNET/FAR:http://www.arnet.gov/far/far_faqframe.html

32

Agencies Best Practices Pages (2 of 2)

• GSA - the definitive guide and “buy accessible” portalhttp://www.section508.gov

• Usability.govThe recent recipient of a Hammer awardhttp://www.usability.gov

33

The Accessibility Forum: Interviews

• Interviews with Federal Agencies regarding the Procurement Challenges of Section 508: acquiring accessible Electronic and Information Technology

• Interviews conducted over August - September 2002

• Interviews conducted by Forum staff and GSA Staff