FROM SIMULATION TO VR BASED SIMULATION AND TESTING

114-16 July 2005 International Journal of Systems Science

01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

FROM SIMULATION

TO VR BASED SIMULATION AND

TESTING

FROM SIMULATION

TO VR BASED SIMULATION AND

TESTING


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Dr. Pavel G. [email protected]

Tel: 269-276-3284

Emiliya [email protected]

Tel: 269-329-3019


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Interactive Computer simulation system

RealWorld

Computer Interface

Work interactively

Manufacturability

and assembly tolerance

restrictions

Designer

Computer

Input

Output

Keyboard

Part A

Part B

Manufacturing and Assemblytolerance simulation

Part A

Part B

Data flow

Nominal Shape

Properties

Properties

Tolerance

Tolerance

Nominal Shape

Statistical Tolerance

Statistical Tolerance

Manufacturing tolerance simulation

Manufacturing tolerance simulation

Manufacturing and Assembly Data base


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

VR simulation


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality Simulation

•Virtual reality simulation/prototyping provides complete description: not only of the geometrical, but also the functional capabilities of the future product and process before the actual product is delivered.

•Allows external user to test and give comments for product improvement

•Virtual reality technology allows instant examination of the accuracy of the design concept early in the development cycle before even the physical model is produced.

• VR system can be used for training of the personal before the real process is delivered.

•The initial cost of virtual product is much lower than the real product

• VRA Implementation is based on real process


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality simulation has matured

Virtual Reality simulation has matured from high-end supercomputers and special designed interface devices to mainstream daily computing.

Electronic Visualization Laboratory's (EVL) development of virtual reality (VR) devices, software libraries/toolkits and applications for collaborative exploration of data over national and global high-speed networks - called "tele-immersion."

CAVE became a symbol of high-end VR, high equipment cost and development lead to limited application to small group of research labs.

Washington University Virtual assembly project VADE has some advance feature like CAD data import/export, natural hand interaction, and physical based simulation.

Problem: High-end computer requirements and difficult to use interface has not lead to wide acceptance of this advance VR system.

Why? Cost factor, User Interface, VR simulation design and modeling complexity


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Product (beta) testing in software engineering

•Software designer: extensive testing of their design product not only within their company, but also by getting feedback from the end-user.

•So called beta testing is well-established practice among almost all software companies before they release new software to find bugs, test usability problems, and judge about needed features, new or existing.

•Alpha version or alpha release is the first release of a computer program or other product, likely to be very unstable but useful for demonstrating internally and to select customers. It is also called "preview” or “technical preview''

•A beta version, intermediate step of the full development cycle, is not fully debugged or fully functional but satisfies a majority of the requirements.

•When a beta is released to the general public it often becomes used almost as widely as the finished product


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

The beta testing for simulation purpose can be used well in advance of the

developing of the hardware and software products to employed different options and

engineering solution that lead to design optimization before the real products is

ready.

For example:

Companies: to understand where technology is heading, which products appear

promising, and--equally important--which software isn't worthy of deployment.

Physical science: phenomenological and conceptual model evidence to help them

develop robust and valid ideas. To represent their predictions regarding the

outcomes of experiments (either hands-on or virtual).

Why there is need for product simulation and Beta testing (BT)


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

VHDL

VHDL: originally developed at the behest of the US Department of Defense in order to document the behavior of the ASICs that supplier companies were including in equipment.

Because of this general-purpose nature, it is possible to use VHDL to write a testbench that verifies the functionality of the design, using files on the host computer to define stimulus, interacts with the user, and compares results with those expected.

Design simulation can include motion simulation and FEA to test different products from aircraft to micro-miniature switches.

For example, according to CAE engineer from Big Three automakers, they cut the cost at least in half using design simulation, optical switcher supplier SiWave Inc. cuts empirical testing by an order of magnitude. Using virtual prototyping, John Deer reduces number of prototypes and saves time; from one to six months for each prototype


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

While 3-D visualization is used to fly trough and play role for design product description and acceptance, it is still only mere 3-D visual presentation with limit interaction or animation missing the way real machines, objects, human and environmental operate together.

VR-based applications are used to enable personnel to control and optimize the efficiency of manufacturing systems at different stages of the systems life cycle in real-time.

Designers are able to integrate new core concepts within the design process.

Usability is a therefore a major issue. The interface provides functions for machine operation, inspection, failure detection and diagnosis as well as cooperative team training

Virtual reality system uses natural human interaction for control of the design products. It works in a matter similar to Beta testing and VHDL simulation, it allows not only testing of a new product, better interaction, and experience by user while a the same time allowing feedback of data to the designer.

What is missing from Virtual reality simulation


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Product design testing framework

Design Product

(DP)

CAD present.

Design Product

(DP)

CAD present.

VR representation

of Design Product

VR representation

of Design Product

VR beta testing of DP

Manuf. simulation


Manuf. simulation


Assembly simulation


Assembly simulation


testing simulation


testing simulation

VR beta testing of DP for

Sales simulation

VR beta testing of DP for

Sales simulation


Marketing simulation


Marketing simulation

Factory

Based

Beta

Testing


Maintenance simulation


Maintenance simulation

Change DP toVR Simulation

Multi-modal VR representation of DP

User

Based

Beta

Testing

Feedback

Problems& needs

Feedback problems and needs

Mobile wearable computer

Automatic log of VR simulation

Automatic log of VR simulation


End-user simulation


End-user simulation


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Using Virtual Reality Simulation Trough Product Lifecycle

For each application case, specific and visual and peripheral interface can be build and a feedback module to log information back to the designed can be added.

While preserving the main VR beta testing capabilities it adds new dimension to the way user interacts.

Multi-modal VR representation

Multi-modal VR representation

CAVE HMD

Desktop Tablet

Mobile PDA


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality?

Virtual Reality, ”VR”

– Describes the ”sensation” of being immersed in a virtual environment

– Heavily ”hyped” by media: ”entering cyberspace,” etc.

– Virtual Reality does not have to be identical to ”reality”, not even come close

InteractiveInteractive

Real timeReal time

RealismRealism

IntuitiveIntuitive

• 3-D-The ”reality” simulated in a VR system is presented in three dimensions to resemble the real world

• Graphics-Most VR systems are based on computer generated imagery

Open GL, DirectX

• Real-timeImages generated by the

computer are ideally updated in

real-time, about 30/60 times per

second (30/60 Hz).

Also, the response from the

simulation to the user’s actions is

immediate.


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

From interactive PC work to immerse environment

The designer works interactively with the computer to design parts in accordance with manufacture and assembly restriction and tolerance requirements. When those requirements are met redesign in accordance with manufacturing and assembly tolerances are applied instead.


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

New level of Interactive Visual Simulation

• Interactive -User interaction (hands grabbing, movement, press of a button, point-and-click) affects the simulation

• Visual-The simulation is presented to the user as a sequence of 3Dimages

• Simulation-3D Models of existing or imaginative environments are simulated using a computer


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual reality application in industry

• Visualization

– Visualize kinematics

– Virtual plants

• Interaction

– Ease of assembly

• Evaluation

– Maintenance

– Virtual assembly


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Assembly and machining path restriction

•Move the holder (tool) position near the part •Grasp the part (tool)•Transfer the part (tool) to the current assembly (part)•Mate the part with the subassembly (machine)•Retract the holder

Assembly/machining path restriction

DimensionalAssemblyrestriction

Dimensional

hand (tool)

hand

restriction

Ass

embl

yre

stri

ctio

n

Assembly (machining)restriction

restriction

restriction

Assembly

Exampleassembly / machining

path restriction

A new challenge is the requirements deriving from assembly (machining) path restriction. The Assembly and machining path to be considered is a so called basic motion planing problem. It is used to check geometrical feasibility of the planned operation. As the motions are with small amplitude, we can restrict our

attention to the workplace as a small volume around the assembly.


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Creating a Virtual Reality application

Once the mechanical parts system is designed in CAD software the geometrical data is imported to VR package EON Reality. For the complicated geometry with polygon reduction of the irrelevant parts is done before importing in order to keep performance of the simulation enough high.

Depending of the assembly requirements additional tools and machinery like screw driver, wrenches, torque wrenches, push tools, press machine, crane machine and controllers are developed and set. Further room interior: walls, floor, working tables is created and imported to VR package. New 3D GUI user-friendly interface is designed to be comfortable for ordinary user, not accustomed to PC environment. Later setups, behavior model, and the sequence flow and control functions of the assembly system is programmed.

Application

Buildsimulations Integration

ModelingDevelopcustom

components

Existingsystems

PeripheralsVisuals

CAD DataApplicationknowledge


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Manufacturing and Assembly – CAD/CAM

First problem is that CAD data is very big, so perforce is very slow so frame rate is so slow that even without assembly interaction with an object, the user has to wait. The other problem is that CAD software are designed for other type of usage, mainly drafting, but when it come to interface, working space and immerse filling it cannot fulfill those needs.

Another issue is that that VR peripherals are not designed for and does not work with CAD systems. Certainly some CAD companies have tried to include assembly function in the software, but virtual reality presents can not be achieved in most of the cases. The design shall be evaluated in 3-D environmental to achieve an immerse filing of the of the model, workspace and user movements to allow interaction when the simulation run in real time speed.

Why not only CAD/CAM?


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Creating VR simulation (1)

CAD Designed model shall be modified for virtual assembly:• Data reduction - As the virtual assembly simulation is dynamic process, using motion, animations, etc., huge data model will cause performance degradation. • Immerse feeling of the model and workspace -Virtual working space shall be designed in the matter that is close to the real situation to carry out an immerse feeling• Peripherals-3d tracing sensors, gloves for new task used in virtual assembly such as grasping and caring objects, tracking the movement of the assembled parts and the user during assembly process• HDM, Stereo Glasses-use of Head Mounted Display or stereo Glasses to achieve an immerse filing in Tree-dimensional environmental.


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Creating VR simulation (2)

GUI in 3D-Space User Interface (SUI), user position set, forward button, cancel, redo etc. - New easy to use SUI shall be designed, including gesture mechanisms closely resembling real gesture movement of the operator.

Feedback Force- The role for the feedback force is to implement touch feeling and physical quantities such as friction, dumping, and collision detection between the user and virtual objects.

Without such physical quantities, simulation is basically orientation to visual perception, that is not enough for person to control and use VR simulation, moreover without such feelings user can get disoriented and lost in VR space.

Keeping VA version update and synchronization in virtual enterprise -As the virtual assembly design works in condition of the concurrency, special attention shall be paid to VA data version update and synchronization.


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

The concept for building Virtual manufacturing and assembly

• Other very important issue is the user interface interaction with VR environmental in the natural manner. No special knowledge or computer skills are required as the work within the immerse space VR assembly environmental with gloves, HMD and tracking sensors offer an easy non interrupting style of work.

The system developed has several important differences from usually developed VR based simulation:

• It is implemented based on the real assembly practice

• Achieves high speed frame rate on ordinary personal computer.


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality simulation example-assembly schema

Assembly between parts shall meet constraint such as axis alignment, surface matting and alignment. Constraint are implemented in the assembly simulation to secure and help the assembly operation during the assembly, as the is no possible for him to fulfill those constraints in the real assembly.

• Assembly ordered tree Assembly precedence order constraint shall be implemented in the designed assembly simulation structure to prevent confusion or inaccurate assembly result.• Collision detection prevents part interfere with other part.

• Physical constraint are simulated by assembly constraint (connection strength should be simulated)

Assembly hierarchy Ordered tree

Collision detection hand to part between parts

Hand before collision

Ghost handShow force

Behavior Model and relations


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality simulation hardware and simulation

Peripherals-3D tracing sensors, gloves for new task HDM, Stereo Glasses-use of Head Mounted Display or stereo Glasses to achieve an immerse filing in Tree-dimensional environmental.

Simulation consideration CAD Data import and reduction Immerse feeling of the model and workspace

3D Space User Interface (SUI), New easy to use and intuitive SUI shall be designed, including gesture mechanisms closely resembling real gesture movement of the operator. Force Feedback - Version update Keeping VA version update and synchronization in virtual enterprise


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality simulation - hardware and software • One of the benefits of using of the ordinary business PC reduces the cost, but more important is the possibility to use available peripherals to set up a complete system available even for medium size research laboratory. The system we use to develop and run the simulation, consist of PC Pentium 400, 128 RAM.

For complete immersion HMD Virtual research V8 (also Sony Glasstron-Stereo HMD, Polhemus Fastrack Tracking System, Fake Space pinch gloves. We run EON Virtual reality software that supports most of the peripherals. Key parts of this VR software are: EON studio the main engine, Data Exchange –Most geometry was imported using data exchange from CAD software IGES, STEP and Pro/E file format. The rest of the models were develop in Multigen, 3DSMax and Autocad and imported.

Behavior Model and relations

Assembly hierarchy Ordered tree


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual assembly example

•Constraint imposed to the sequence of the assembly does not allow improper order of assembly.

• Special 3D-SUI was developed. As a result system work using common human gestures, no special computer skills are required.

• Gloves are used with FakeSpace and Fastrack were used to interact VR simulation in natural, manner. Each part was assembled to other using hands to hold and move them.

•The hands, head, body movements correspond 1:1 to the real movement


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

• Gloves are used with FakeSpace and Fastrack were used to interact VR simulation in natural, manner. Each part was assembled to other using hands to hold and move them.

• The hands, head, body movements correspond 1:1 to the real movement

Virtual assembly example (2)


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality simulation systems

From simple Virtual assembly simulation to virtual automation plant system


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality Application


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality Application (2)


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Virtual Reality Application (3)


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

SERVER

CLIENT CLIENT

CLIENT CLIENT

SERVER

CLIENT CLIENT

CLIENT CLIENT

EngineeringWorkstation OS1

NotebookOS2

PC DesktopOS2

PC DesktopOS3

PC DesktopOS4


NotebookOS2

PC DesktopOS2

PC DesktopOS3

PC DesktopOS4

•A new computing network method peer to peer (P2P) that provides an opportunity for direct design collaboration between two clients without the use of a central computer.

There are different activities within the product realization process, such as design, marketing, sales, etc., all of which having different objectives and needs. Collaboration involves feedback from these activities into the product realization process.

•Connections between computers are established and eliminated on demand.

Collaboration work using Peer to Peer network


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

P2P network consists of both wireless and wire-line Internet and network connections.

The method of working by marketing and sales is not often the same as designers, whereas designers typically sit on the computer to generate the product design on the computer, the marketing and sales staff are typically in the field. Marketing and sales staff usually carry mobile devices, like personal digital assistant (PDA), wireless notebook computer, cellular phone, because the task they perform are generally not those performed by design engineers. For example, the objectives of the marketing activity may include selection of color, material

P2P collaboration (2)

for aesthetic features of the product, whereas the objectives of the assembly designer are to ensure that no two components are intersecting in 3 dimensional (3D) space

Open on demandConnection


PDANotebookOS2

Webpad

PC desktopOS3

CellularInternetPhone Opened P2P

Connection

Open on demandConnection


PDANotebookOS2

Webpad

PC desktopOS3

CellularInternetPhone Opened P2P

Connection


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

We described framework and applications of Virtual reality simulation trough product lifecycle.

The proposed Virtual reality test system allows complete geometrical representation and simulation of the functions of designed products.

These products can be tested to provide feedback, and design changes can be done on demand.

Further this VR system can be applied at any step of production realization process as well at the end-user site.

No special knowledge or computer skills are required to be learned by end user for interaction and control.

Conclusion


01001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001000100010010010100011110100010101001010010100100100100101000100100100101001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100110100100100100101000100100

010010010100011110100010101001010010100100100100101001000100100100101000111101000101010010100101001001001001010010001000100100101000111101000101010010100101001001001001010001001001001010001111010001010100101001010010010010010100100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100101100101100101

0100100101000111101000101010010100101001001001001010010001001001001010001111010001010100101001010010010010010100100010001001001010001111010001010100101001010010010010010100010010010010100011110100010101001010010100100100100101001001001010001111010001010100101001010010010010010100100010010010010100011110100010101001010010100100100100101001101001001001001010001001

Questions?

Documents

FROM SIMULATION TO VR BASED SIMULATION AND TESTING