Design of accessibility framework for mobile devices

Summary of this paper

• Considerations

• Technical issues for Accessibility Mobile Framework.

• Existing technologies.

• APIs.

• Tools and implementations of existing Java Virtual Machines.

• Open Accessibility Framework methodology, its features and its concepts.

• Illustrates the concepts of the accessibility mobile frameworks, their role and the interaction between them.

• Presents a view map schema describing each layer.

• It analyses three different implementation approaches for the development of the Accessibility Mobile Framework.

Accessibility framework

• Definition

• “an abstraction in which common code providing generic functionality can be selectively overridden or specialized by user code providing specific functionality” [http://en.wikipedia.org/wiki/Software_framework]

• A framework is a special case of software libraries that they are reusable abstractions of code wrapped in a well-defined API.

• AEGIS provides the accessibility framework for mobile devices:

• open source product,

• suite of (standalone) applications,

• API(s),

• custom User Interface components.

Building the accessibility framework

The view map (1/3)

• The architecture of the built-in accessibility framework for mobile phones will be dedicated to JME platform.

• The figure depicts the division of the JME software layers in lower and higher layers.

• Different tools and approaches for building and porting processes will be used for each of them.

The view map (2/3) - Lower layers

• Porting the CLDC and CDC HotSpot Implementation Virtual Machine to a new platform.

• Consider the “port” of several important issues such as CPU and Operating System.

• A porting effort for JVM and configuration will require building and modifying the source code.

The view map (3/3) - Higher layers

• The “Application Layer”.

• Stands on top of existing platforms.

• Avoid incompatibility and portability problems among different device technologies (CPUs and Oss).

• Group of totally independent applications which reside on top of existing platforms .

Framework’s architecture

• Research on:

• Existing Accessibility APIs.

• Java Accessibility API.

• ATK Accessibility Toolkit from Gnome's GTK+ project.

• Android Accessibility API.

• BlackBerry Accessibility API .

• iPhone Accessibility API.

• Their sample applications.

Architecture's components (1/2)

• Accessible application

• The application containing accessible features, e.g. Accessible TextField.

• Accessibility Bus

• The Event (Server) bus which receives notifications/events from accessible applications and distributes them to assistive technology applications.

• Assistive Technology (AT) applications

• Applications which serve the corresponding output to end-users, e.g. screen readers.

Architecture's components (2/2)

• Sequence diagram

TCP/IP approach

• The Accessibility Bus is the Server side.

• Both accessible application and assistive application are thought as

clients.

• Assistive Technology (A.T.) has to be registered by informing that it is interested

in listening notifications/events.

• AT establishes a socket connection with Accessibility Event (Server) Bus.

• Accessible application establishes also a socket connection with the Accessibility Bus.

• A message (notifications regarding accessible events) is sent to the Accessibility Bus.

• Accessibility Event (Server) receives a collection of accessible messages and

distributes them to the corresponding registered A.T. applications.

• The socket connection between these parts is taking place “localhost” (locally)

for the CLDC and CDC devices.

Listener approach

• An interface (AccessibleListener) will be implemented providing at least one method

• It has to be implemented by the assistive application.

• The Accessibility Bus (Server) holds a list with all registered listeners and forwards the messages that are available in the “message stack” to the corresponding assistive technology application.

• An A.T. application will implement the AccessibleListener interface in order to listen for incoming messages.

• The accessible application will contain a method which will be responsible to send notifications to Event Bus.

• The interaction parts are the same to TCP/IP’s approach.

JNI and KNI: the native approach

• A Java virtual machine commonly needs access to various native functions.

• Both JNI and KNI are implementation level APIs that are targeted primary at Java Virtual machine developers.

• JNI is visible to CDC Java programmer and KNI is not visible to CLDC Java programmer in the aspects of accessing methods.

• Providing accessibility features in native level, a further development and modifications has to take place within the Java Virtual Machine (JVM).

Notifications/Messages (1/2)

• The A.T. application will be notified when accessible information is available.

• The notification about the change may contain the following information:

• name of the custom UI component,

• type of event, for example, the contents of the custom UI component changing or the custom UI component getting focus,

• value of the custom UI component before the event,

• value of the custom UI component after the event.

Notifications/Messages (2/2)

• The following changes to a UI component can trigger a notification to an AT application

[AOF mythology]:

• a change to the position of a cursor,

• a change to the name,

• a change to the text,

• a change in a child component,

• a change in the state,

• a change to the numeric value.

Rules to support accessibility

• If a component does not display a short string, a descriptive name for it has to

be specified.

• A tool tip for each component has to be set whenever it makes sense.

• If a tool tip for a component can not be provided, a description alternatively can

be provided that assistive technologies can give the user.

• Specify keyboard alternatives wherever possible and keep in mind that keyboard

alternatives varies by components.

• Assign textual description to all Images and Icons objects in your application.

• In a bunch of components that form a logical group, try to put them into one

container.

• Any custom component is created, it should support accessibility.

Conclusions

• Focus on higher layer

• Accessible application, Accessibility Event(Server) bus and AT application will be built and run on top of existing Operating Systems and (Java) Virtual Machines.

• Tested over existing OS and JVMs using existing tools (mobile emulators)

• Listener approach assumes

• all components (accessible application, A.T. application and the Event (Sever) bus) have to be wrapped together in the same Project-Application.

• TCP/IP approach assumes

• each component as independent application.

• TCP/IP approach compared with the other two is the most flexible, openness and portable solution and it can be applied easily to existing assistive application technologies and also to existing device models in the market.

• The assistive application vendors can extend existing technologies with the minimum possible cost and effort in comparison with the other two solutions.

• The final version of AEGIS Accessibility Mobile Framework can be exploited and published as a Java

Specification Request (JSR) following the specific Java Community Process standards.