Two Kinds of Novel Multi-user Immersive Display Systems

Two Kinds of Novel Multi-user Immersive Display Systems

Dongdong Guan1,2 Chenglei Yang1* Weisi Sun1 Yuan Wei1 Wei Gai1,2

Yulong Bian1,2 Juan Liu1,2 Qianhui Sun1 Siwei Zhao1 Xiangxu Meng1,2

1Shandong University

Jinan, China

chl_yang, [email protected]

2Engineering Research Center of Digital Media

Technology, Ministry of Education

Jinan, China

[email protected]

ABSTRACT

Stereoscopic display is a standard display mode for virtual

reality environments. Typical 3D projection provides only a

single stereoscopic video stream; thus co-located users

cannot correctly perceive the virtual scene based on their

own position and view. Several works devoted to developing

multi-user stereoscopic display, but the number of users is

very limited or the technical implementation is complicated.

In this paper we put forward two flexible and simple

projection-based multi-user stereoscopic display systems.

The first one, named TPA, is based on a triple-projector array

and provides a 120Hz active stereo for three users. Two

TPAs can be combined to form a six-user system. The second

one, named DPA, is a dual-projector and easy-implemented

system providing individual stereoscopic video stream for

two to six users. Finally, a co-located multi-user virtual

fireman simulation training system and a virtual tennis

simulation system were created to verify the effectiveness of

our systems.

Author Keywords

Multi-users stereoscopic display; virtual reality; co-located

collaboration.

ACM Classification Keywords

H.5.1. Information interfaces and presentation: Multimedia

Information Systems—Artificial, augmented, and virtual

realities

INTRODUCTION Stereoscopic display is very popular in cinema and virtual

reality environments providing the audience with immersive

experience. This technology is implemented by displaying

two-channel videos of left and right eyes on the same screen

such that users can gain stereoscopic visions with active

stereos or polarizers.

At present, typical 3D televisions and 3D projectors provide

only a single view for all audiences; thus in co-located

collaborative virtual environments, they offer only a correct

perspective view for a single user, causing other co-located

users’ views more or less distorted. [12] analyzes the

influence caused by viewpoint distortion of group users in a

single-view CAVE environment; [10,15] endeavor to display

average view image to compensate for the distortion, but fail

to offer a correct perspective view for each user. On the other

hand, co-located users cannot roam in virtual environments

independently, but only follow a leader user. Thus multi-user

display is essential to improve reality experience and

flexibility in co-located collaborative virtual environments.

Compared with 3D television and Head Mount Display

(HMD), projection-based stereoscopic display can support

large-scale display and allow users move freely in physical

spaces. Multi-user projection can provide a correct

perspective view for each user and improve their sense of

reality experience and immersion. This technology has been

applied in fields such as virtual surgery [2], urban design [3],

and games [17].

Although most of the current projection-based multi-use

display systems can only serve two users [1,2,4,14], the

C1×6 System in [8] can serve six users with six projectors

working concurrently by modifying the projectors (removing

color wheels, adopting a fixed color filter, etc.). The C1×6

System owns the largest user capacity, but the technique

realization and the modification made to the projector (such

as removing color wheels and image calibration) are

complex and expensive, and the construction is not flexible

for two to five users. Furthermore, each adjusted projector in

a C1×6 System cannot be used as a color projector alone

anymore.

Therefore, in this paper, we put forward two projection-

based multi-user stereoscopic display systems, which can be

easily implemented. They are also flexible in building multi-

user collaborative VR systems and conducting related

human-computer interactive research. The first system is

named TPA (Triple-Projector Array), and the second system

is named DPA (Dual-Projector Array). TPA is based on a

triple-projector array, which provides a 120Hz active stereo

for three users, and two TPAs can be combined as a six-user

system. DPA is a dual-projector and easy-implemented

Permission to make digital or hard copies of all or part of this work for

personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies

bear this notice and the full citation on the first page. Copyrights for

components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to

post on servers or to redistribute to lists, requires prior specific permission

and/or a fee. Request permissions from [email protected]. CHI 2018, April 21–26, 2018, Montreal, QC, Canada

© 2018 Association for Computing Machinery.

ACM ISBN 978-1-4503-5620-6/18/04…$15.00

https://doi.org/10.1145/3173574.3174173

CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada

Paper 599 Page 1

system providing individual stereoscopic video streams for

two to six users. In summary, this paper makes the following

contributions:

(1) We propose TPA, which is a projection-based multi-

user stereoscopic display system with three projectors.

TPA can implement stereoscopic projection for three users.

Compared with the C1×6 System, TPA only needs a simple

modification in the color wheel of the projector. It is easier

build while possessing lower cost and higher refresh rate at

the same time. Additionally, two groups of three-user

arrays can be extended to six users combined with the

optical polarization, resulting in more flexibility for users.

Furthermore, each adjusted projector in a TPA can still be

used as a color projector alone afterwards.

(2) We also present DPA, which is another projection-

based multi-user stereoscopic display system with two

projectors. DPA does not require modification to the

internal structure of the projector. With the help of

dedicated shutter glasses we designed in this paper, it can

flexibly provide individual stereoscopic video streams for

2 to 6 users. Its implementation is easier and its cost is

lower than those of the C1×6 System and TPA, so that it

can provide more convenient environment construction for

co-located collaborative virtual environments, although its

refresh rate may be reduced when the number of users

increases.

(3) One force-to-zero driving strategy is put forward to

design a kind of dedicated shutter glasses that can keep the

open/close sequence of the LC shutter in sync with the

projection displaying sequence, to achieve users’

independent viewing. It can reduce the switching time of

LCD, thus avoiding crosstalk at high refresh rates.

(4) Two application examples are developed for verifying

the effectiveness of our work proposed in this paper. One

is the multi-user virtual fireman simulation training system

based on DPA, and the other is the multi-user virtual tennis

simulation system based on TPA.

The rest of the paper is organized as follows: the first is a

brief review of related work; the second is a realization of

TPA; the third is a presentation of DPA; the fourth is a

demonstration of the two application examples; the fifth is

discussion and the final is conclusion.

RELATED WORK There are three main ways to achieve stereoscopic display:

lenticular, optical polarization, and active stereo, based on

which multi-user display systems could be developed. When

a multi-user stereoscopic display system serves n users, there

are totally 2n-way videos displaying on the same screen area

concurrently: n-way for left eyes and n-way for right eyes.

The key issue is to enable users to get their own image from

overlapping images on the same screen.

[7,11,13] employ lenticular or parallax barrier displays to

implement a two-viewpoint display system; [9] employs

LCD 3D screen combined with active stereo to provide

haptic interaction for two users. A drawback of these systems

lies in that users are limited to a relatively small area in these

LCD screen-based systems.

Several works are devoted to projection-based two-user

display systems. [1] proposes a system that employs a 144Hz

CRT Projector to realize active stereoscopic displays for two

users, which results in 36Hz per eye per user. Combining

optical polarization and two active 3D projectors, Barco et al.

[2] develops the “Virtual Surgery Table” for two users. This

is also applied to a large display wall by Riege et al. [14]. A

similar technology is used in [4] to design a CAVE system

in co-located collaborative interaction research for two users.

A common weakness of these systems is that they can serve

only two users.

The C1×6 System in [8] uses the active stereo projection in

combination with optical polarization to realize a

stereoscopic display function for six users. It employs 6

projectors to construct a projection array and display

individual stereo video streams for 6 users. With three

projectors for each of the left and right eyes, the

corresponding left or right video is displayed respectively,

which is distinguished by a polarizer. The popular single-

chip DLP projectors are adopted to construct a projection

array. The color wheel of the projector is removed to adjust

a 120Hz color projector to a 360Hz monochromic projector

and three projectors work in parallel to improve the standard

120Hz image refresh rate to 360Hz; thus 120Hz image

refresh rate is ensured for each user. Moreover, dedicated

shutter glasses are designed and toggled in sync with the

display sequence; therefore each user can watch an exclusive

stereoscopic video stream.

System type Array

projector number

User capacity

Monocular refresh rate

of each user

Frequency of shutter

glasses

Stereoscopic combination

mode

Whether to modify projector structure

Whether to modify projector

circuit

Whether to modify

glasses circuit

Agrawala[1] 1 2 36Hz 144Hz N N Y

Riege[14] 2 2 60Hz 120Hz User N N N

C1×6

System[3] 6 6 60Hz 360Hz

Left and

right eyes Y Y Y

TPA 3/6 3/6 60Hz 360Hz User Y N Y

DPA 2 2-6 =<60 Hz 120Hz Left and

right eyes N N Y

Table 1. Comparison among different systems.


Paper 599 Page 2

Figure 1. TPA: overview of triple-projector array system.

[3] develops a collaborative urban design system for six

users based on the C1×6 System. However, the removal of

the color wheel causes the temperature of the projector to rise;

thus extra cooling module is needed, and the engineering

complexity is increased. Moreover, the adjusted projectors

cannot be used as normal ones. The driven strategy of

dedicated shutter glasses is also complex.

Table 1 lists the performance and implementation of the

existing multi-user projection systems and our proposed

systems.

TPA: TRIPLE-PROJECTOR ARRAY SYSTEM

TPA in this research uses three commercial single-chip DLP

active 3D projectors to construct a projection array, allowing

120Hz active stereoscopic for three users. Dedicated shutter

glasses are also designed to ensure that each user can watch

their own videos. Figure 1 shows the construction and

operation of TPA.

Triple-projector Array

With the increase of the number of users, the image refresh

rate of the projector must be improved, and based on the

working mode of commercial single-chip DLP projectors,

three projectors can work in parallel to triple the rate. For a

single-chip DLP projector, the RGB bit plane are projected

respectively synchronizing with the rotating motion of the

color wheel. When the red section of the color wheel turns in

front of the lamp, the red bit plane is projected. The same is

true for the green and blue bit planes. The colors are thus

displayed sequentially at a sufficiently high rate that the

observer sees a composite “full color” image. From this we

can see that the projection process of one full color image

can be divided into three monochromatic intervals. Therefore,

one 120Hz color projector can be regarded as one 360Hz

monochromatic projector, and three 120Hz projectors can

work concurrently to realize the 360Hz full color refresh rate.

During the assembly of the three projectors, their color

wheels of the C1×6 System are removed with R-G-B filters

respectively in front of the projector lens instead, so that each

of the three projectors respectively is adjusted to a

monochromatic projector to display a primary bit plane of

the images. However, as the color wheel is removed, which

causes temperature rise of the projector, extra cooling

module is needed, so the complexity of engineering is

increased. In contrast, TPA changes the color sequence of the

color wheel, and through GPU-based image mixing, the full-

color images of the three users can be composited in each

monochromatic interval respectively. Since the color wheel

is not removed, there is no rise in temperature and so no more

cooling module is needed.

The modification of the color wheel is shown in Figure 2: the

first projector retains its original RGB sequence; the color

wheel of the second projector is modified to GBR sequence;

and the third modified to BRG sequence.

(a) (b) (c)

Figure 2. Modification of the sequence of color wheels. (a)

Original R-G-B sequence of projector 1;(b) G-B-R sequence of

projector 2;(c) B-R-G sequence of projector 3.

After replacing color wheel, the primary components of the

three projectors in each monochromatic interval is shown in

Figure 3c. In the first monochromatic interval, the three

projectors respectively display R-G-B components of User 1;

in the second interval, they respectively display G-B-R of

User 2; and in the third interval, they respectively display B-

R-G of User 3. Therefore, three images of the three users are

composited in order in each interval. Figure 3 compares the

composite mode of the color image on normal projectors, the

C1×6 System and TPA.

Real-time Image-mixing and Geometric Alignment Based on GPU

To fit the display mode of the triple-user projection array, the

three independent video images need to be mixed to form

triple-user stereoscopic video stream format. Different from

the classical mode, each projector displays a mixed image

consisting of three primary components extracted from three

video images respectively. Whatever the arrangement of the


Paper 599 Page 3

projector array is, it only can reach a rough image registration,

so image deformation for three projectors is necessary to

achieve a precise per-pixel alignment. The processing flow

of image data by TPA, including GPU-based image mixing

and alignment processing is shown in Figure 4.

(a)

(b)

(c)

Figure 3. Comparison of the color image composite modes on

(a) normal projectors, (b) C1×6 system and (c) TPA.

Similar to the alignment processing in the C1×6 System, this

research also adopts camera-to-projector calibration

algorithm based on Gray code patterns [5]. After identifying

common projection on the screen, a look-up table is

generated for each projector based on the calibration results,

and deformation processing is conducted to realize the image

registration of three projectors. Because the mixing process

and the look-up tables are fixed, GPU acceleration can be

adopted to ensure real-time rendering and displaying.

TPA conducts real-time rendering of virtual scene with

Unity3D engine. It renders left and right eye image of three

users in each frame respectively at a 60FPS frame rate. Each

three left or right images are composited into one tripled-size

image, which is output by graphics card, after image mixing

and deformation processing. It is distributed averagely to

three projectors due to the screen splicing function of the

graphics card.

Dedicated Shutter Glasses

Dedicated shutter glasses should be designed to keep the

open/close sequence of the LC shutter in sync with the

projection displaying sequence, to achieve users’

independent viewing. Classical commercial shutter glasses

are designed for single view, with the shutter glasses toggling

per 1/120 second, so the user can perceive left-eye image and

right-eye image with corresponding eye. The interval of open

(white) state and close (black) state is equal as 1/120

second≈8.33ms. To match the display format of the triple-

projection array, the dedicated shutter glasses also toggle per

1/120 second, but the opening interval is reduced to 1/3 of

the previous, meaning that they only open in their own

monochromatic interval, and close in other two intervals to

avoid crosstalk. The open/close timing of classical shutter

glasses and our shutter glasses are shown in Figure 5.

Figure 4. The processing flow of image data by TPA.


Paper 599 Page 4

Figure 5. Open/close timing of dedicated shutter glasses.

As the opening time is reduced to 1/360s≈2.78ms, LC

glasses need to open/close at a fast switch speed, which

regular LC shutters fail. Due to the property of liquid crystal

molecule, LC shutter has asymmetrical switch time, with

short closing time (from white to black) less than 0.1ms and

longer opening time (from black to white) of around 2.5ms.

During the opening period, LC transmittance gradually

increases in curved shape. This opening time, compared with

display period of 360Hz, is obviously too long. If LC shutter

opens in sync with the corresponding monochromic interval,

longer opening time result in insufficient brightness, and if

LC glasses open in advance, crosstalk may be caused. Thus

the important issue is to reduce LC opening time in designing

dedicated shutter glasses.

C1×6 uses two layers of differently configured regular LC

shutters, and reduces its opening time through asynchronous

open/close driving, while this method is complex to realize,

and the transmittance decreases due to shutter overlap. This

research puts forward an easier driving strategy which we

call force-to-zero driving, to reduce the opening time of LC

shutter.

Typically, LC shutter is driven by H-bridge circuit. H- bridge

has three-state output: +15V, 0V and -15V. When driving

voltage is 0V, LC is in open (white) state, when driving

voltage turns +15V or -15V, LC glasses is in close (black)

state, and when H-bridge changes from ±15V to 0V, LC

glasses need about 2.5ms to fully open (white). Similar to the

overdriving strategy of LCD screen in [6,16], when driving

voltage reduces from +15V to 0V, a short-time - 15V pull-

down interval is inserted to speed up its process back to 0V.

The effectiveness of force-to-zero strategy depends on the

duration of the pull-down interval. If the duration is

insufficient or too long, its opening time cannot be reduced,

and based on experiments, when the duration of pull-down

interval is 10-15μs, the opening time of LC glasses can be

reduced to about 0.2ms, being able to satisfy the display

speed requirement of 360Hz or higher. Equally, when

driving voltage changes from -15V to 0V, a pull-up interval

should also be added. Force-to-zero strategy can be

completed by embedded single chip machine in shutter

glasses, and it does not increase the circuit complexity

considerably. Figure 6 compares LC driving with the

overdriving zero crossing driving in this research.

(a)

(b)

Figure 6. Comparison between typical H-Bridge LC driving

and Force-to-zero driving strategy of LC glasses, (a) Typical

H-Bridge LC Driving, (b) Force-to-Zero LC Driving.

To realize the synchronized display sequence between

open/close timing of LCD shutter glasses and the projector

array, the build-in TI DLP Link mode inside the projector is

adopted. When the projector displays one frame of image, it

projects a high-brightness optical pulse of 20μs as frame

synchronization signal, which is received by shutter glasses

through photodiode. As the time of optical pulse is rather

short, human eyes cannot perceive it.

Triple-projector array and dedicated shutter glasses can offer

individual 120Hz active stereoscopic display for three users.

As the commercial 3D projector is used, the modification

process of projection array is much easier than C1×6, only

necessary modification in color wheel of the projector and

no modification in control circuit. The C1×6 System needs


Paper 599 Page 5

projection array composed by six projectors to work

simultaneously, while in this research, three projectors as one

group may be used to realize active 3D projection for three

users, and two groups of three-user array can be extended to

six users combining with optical polarization, which is more

flexible to use.

DPA: DUAL-PROJECTOR ARRAY SYSTEM

DPA employs dual projector as a projection array to provide

correct perspective view for 2 to 6 users. It can provide

independent 120Hz 3D display for 2 users, and for more than

2 users, refresh rate of each user is scaled down, but it is easy

for system implementation, without modification to the

projector, and only dedicated 3D shutter glasses are needed.

Two projectors respectively display the images of users’ left

and right eyes, which is distinguished by optical polarizer.

The standard 120Hz projectors display the 3D video of each

user in order. When the user number is n, projection is

divided into n intervals, Projector 1 displays according to the

sequence of “User 1 left image – User 2 left image - … - User

n left image”, and Projector 2 displays according to the

sequence of “User 1 right image – User 2 right image - … -

User n right image”. In each interval, the screen

demonstrates the left image and right image of a certain user

at the same time, and the two LC shutters open in the

meantime to form stereoscopic vision. In other intervals, they

close at the same time to avoid crosstalk. When the user

number is more than 2, the single- eye refresh rate of each

user is scaled down, that is 40Hz for 3 users, 30Hz for 4 users,

24Hz for 5 users and 20Hz for 6 users.

Unity3D engine is employed to perform real-time rendering.

When the user number is n, n pairs of virtual cameras need

to be set, respectively corresponding to n left eyes and n right

eyes. To ensure the left-eye and right-eye image shown

simultaneously on the screen, the left and right camera

rendering result of each user needs to be set as a double-size

stereo image side by side, and the image is displayed by two

projectors due to the screen splicing function of the graphics

card.

The two projectors in DPA is divided based on left and right

eyes, whose advantage is that it can distribute video refresh

rate to each user. When the Unity3D engine operates at a

standard frame rate of 60Hz, the double-size stereo images

of two users should be rendered and output in each game

frame. Taking three users as an example, the rendering

sequence of n pairs of virtual cameras is as follows: In the

first frame, the stereo cameras of User 1 and User 2 conduct

rendering; in the second frame, the cameras of User 3 and

User 1 conduct rendering; and in the third frame, the cameras

of User 2 and User 3 conduct rendering, thus to form an

average and circular rendering sequence. If two projectors

are divided based on user number, resolution may be

distributed averagely only when user number is even, and for

3 or 5 users it cannot be distributed. Figure 7 shows virtual

camera rendering sequence and shutter open/close sequence

of each user.

As DPA adopts the mode of multi-user video projection in

order, and uses dedicated shutter to realize independent

watching, the user number can be extended to 6, but with an

image flicker caused by a 20 Hz refresh rate. The experiment

show that 3 or 4 is the optical number for this system.

Figure 7. Rendering sequence of virtual camera pairs and

corresponding open/close timing of shutter glasses.

APPLICATION EXAMPLES

This section introduces two application examples, with one

being a co-located multi-user virtual fireman simulation

training system employing DPA and the other a co-located

virtual tennis simulation system utilizing TPA.

Virtual Fireman Training System

A co-located virtual fireman training simulation system has

been developed (See Figure8 (a)). As three firemen typically

form a basic action group in real firefighting actions, three-

user mode is chosen for co-located collaborative fireman

training.

Here, DPA is selected to construct the system since 40Hz

refresh rate for each user can satisfy the requirement of

image quality for virtual roaming and extinguishing in the

fireman training system. It is constructed by two Benq

MS524 projectors connected by a NVIDIA Quadro P4000

graphics card. Obviously, the construction of DPA is simple

and has lower cost.

The Unity3D game engine is employed as the VR platform

to build the architectural environment, and particle effect is

used to simulate flame and smog to implement virtual fire

scene. The users watch the virtual fire environment from the

first perspective and three sets of 3D cameras are set in the

virtual scene, offering independent stereoscopic videos

rendering for three users and displaying in order. Three users

can watch their own video by the dedicated shutter glasses as

shown in Figure 8(a).

A virtual squirt gun has been designed as the interactive tool

for fireman users, which is easy for a user to control the move

path and the extinguishing action. Its outline is generated by

3D printing based on the actual structure. A rocker switch is

set on the virtual squirt gun to control the avatar’s movement

in a virtual scene, and a 9-axis inertial sensor is put inside the


Paper 599 Page 6

squirt gun to gather the hand direction of the user, such that

the fireman user can identify the gun direction and set the

ejection button to eject virtual water. The virtual gun, by

means of Bluetooth, uploads the user’s movement control,

aiming direction, and ejection information to the server.

Based on the information, the server controls the movement

and fire control of three virtual avatars in a virtual

environment.

(a)

(b)

Figure 8. Application Examples: (a)Virtual Fireman Training

System, (b) Virtual Tennis Simulation System.

Virtual Tennis Simulation System

We also developed a virtual tennis simulation system with

TPA (See Figure 8(b)), since it maintains 60 Hz Monocular

refresh rate, which can be effective to display the high-speed

movement of the ball. Two of the users are players facing the

same screen watching the movements of their opponent’s

avatar and the virtual tennis ball from their own perspective.

The third one is an audience enjoying the game. These co-

located users can roam in a virtual environment dependently.

Such features are achieved with the help of the multi-user

display system.

Here, TPA is composed of three adjusted Benq MS524

projectors. It can achieve 360Hz refresh rate. We adjusted

two projectors by rotating their color wheels. The

modification is simple and easy. What is more, each adjusted

projector in the TPA can still be used as a color projector

alone after adjusting the setup of its output hue from the

projector setup menu or the graphics card menu. So the

incurred cost is very low too.

A Microsoft Kinect is adopted to capture the movements and

swing speeds of two players. According to these data, the

route and speed of the return ball is simulated and displayed

on the screen with TPA.

DISCUSSION

We further discuss and analyze the performance and

implementation of our systems in this section. Besides the

items listed in Table 1, we also discuss issues regarding

brightness decrease, frame rate decay, rainbow effect, and

potential crosstalk.

The number of projectors and users: Projection-based

multi-user stereoscopic display extends single viewpoint of

classical 3D projection technology to multiple viewpoints,

so that each user can watch an exclusive stereoscopic video

conforming to his or her position and view. The key issue

is to overcome the multi-way video overlapping projection.

When the user capacity of multi-user 3D display system

is n, 2n-way video is needed in the screen area. Although

optical polarization can only distinguish 2-way video,

active stereoscopic can upgrade user capacity through a

timesharing display, and double the number of users

combining with optical polarization. The optical polarization

is used in Riege [14], C1×6, TPA and DPA, but it plays

different roles. In Riege [14] and TPA (only needed when 2

TPA serve 6 users), it is used to distinguish two groups of

users, who perceive stereoscopic visions with active stereos.

In the C1×6 System and DPA, it is used to distinguish left

and right eyes, in which circumstance both left and right

images are projected to the screen simultaneously.

TPA can implement stereoscopic projection for three users

based on three projectors. Combined with the optical

polarization, it can also extend three-user arrays to six users,

resulting in more flexibility to users. On the other hand, DPA

is constructed based on two projectors without modification;

and with the help of dedicated shutter glasses we designed,

DPA can flexibly support two to six users.

Modification of hardware: As TPA only needs a simple

rotation to adjust the sequence of the color wheel of the

projector without any other modification, its construction is

much easier and simpler than the C1×6 System, resulting in

the lower cost of a TPA system. Additionally, since color

wheels in the C1×6 System are removed and a fixed color

filter is used in one of the primary colors, causing the inner

temperature out of limits and thus requiring extra cooling

modules; as a result, the modified projector cannot work on

its own. However, TPA only changes the color sequence of

the color wheels; thus it avoids the heating problem and each

adjusted projector can still be used as a color projector alone.


Paper 599 Page 7

We design dedicated shutter glasses using the force-to-zero

driving strategy, which can reduce the switching time of the

LCD shutter, thus avoiding crosstalks at high refresh rates.

Since only two projectors and the dedicated shutter glasses

are needed, and no modification is necessary for the projector,

the realization of DPA is much easier and its cost is lower

than those of the C1×6 System and TPA. Therefore DPA can

provide more convenient environment construction for co-

located collaborative virtual environments.

Refresh rate: The refresh rate of the present popular

commercial 3D projector is 120Hz, showing 60 frame left-

eye image and 60 frame right-eye image per second. For

multiple users, the refresh rate per interval can be increased

by parallel projection of multiple projectors. A relatively

simple model is composed of two projectors for parallel

projection, combined with an optical polarization, to achieve

the 120Hz refresh rate for two users. When the number of

users is more than 2, the monocular refresh rate of each user

is scaled down; in this case, no physical or circuit

modification of the projector is necessary, but dedicated

shutter glasses are needed which is simple to implement, as

DPA does. Another model is to modify the projector

structure through the triple-projector parallel working to

increase the refresh rate to 360Hz and shorten the opening

time of the shutter glasses to 1/3, as done in TPA and the

C1×6 System. The refresh rate decay of DPA can lead to

image flicker to different extents: for 6 users at 20Hz the

flicker is noticeable; for 4 users at 30Hz the flicker is slight;

and for 3 users at 40Hz, the flicker is too subtle to perceive.

Thus serving 3 to 4 users with DPA will be a proper choice.

Crosstalk: In multi-user projection, the image contents of

two adjacent frames are not consistent, since they may be

perceived by different users from different perspectives.

Thus, crosstalk images are much more noticeable in multi-

user projection than those in a normal one. To avoid crosstalk,

high switch speed of LC shutter is required. Our force-to-

zero driving strategy of the LCD shutter reduces the

switching time effectively to lower the occurrence of

crosstalk. As we employ the force-to-zero driving strategy in

the design of the dedicated shutter glasses, the switching time

of the LCD shutter is decreased; as a result, DPA can avoid

crosstalk at a high refresh rate.

Perceived brightness: The perceived brightness of our

systems is evalued by having participants watching the same

virtual scene with a normal single view 3D projector and our

multi-user display systems. We adjust the image brightness

of our systems to make the participants obtain similar

brightness perception as the normal projection. For TPA, the

flow of light per interval is maintained with three projectors

working concurrently, while the perceived brightness is

decreased by roughly 10%, as the shutter glasses have a

lower duty cycle. For DPA, the duty cycle maintains, while

the lower refresh rate and the attached polarizer lead to

approximately 18% decrease in perceived brightness.

Rainbow effect: The rainbow effect is caused by the defect

of the single-chip DLP projector, which projects R-G-B bit

planes of the same image in sequence. As a result, the

audience who are extremely sensitive to the sense of color

may experience the asynchronization of colors. Statistics

show that the proportion of these people is quite small. In

TPA, the R-G-B bit planes are projected at the same time by

triple projectors to minimize the risk of the rainbow effect;

but there is still possibility of rainbow effect in DPA.

CONCLUSION

A multi-user stereoscopic projection display system extends

the single view to a multi-view display, which is an effective

display means to realize multi-user co-located collaboration.

Each user can watch the virtual scene through his/her single

correct perspective view based on his/her own position or

viewpoint while increasing their sense of reality experience

and immersion.

In this paper, two projection-based multi-user stereoscopic

display systems are introduced. One is called TPA, which is

based on a triple-projector array and can provide a 120Hz

active stereo for three or six users. The other is called DPA,

which is a dual-projector and easy-realized system providing

individual stereoscopic video stream for 2 to 6 users. Both

systems are flexible and easy to be implemented by

researchers or engineers. They can produce multi-view

systems supporting collocated collaboration such as the

multi-user virtual fireman training system and the virtual

tennis simulation system. The systems can also be used in

many other applications such as synchronously playing

PowerPoints with different languages and playing movies

with multi-language subtitles for multi-users.

Two problems remain to be settled to further improve the

multi-user stereoscopic projection display technology. One

is to enhance the refresh rate of the projector image. The

inner image refresh rate of certain types of projectors is more

than 120Hz. For example, the Panasonic PT-HZ900 type

LCD projector and the triple-chip DLP projector have a

refresh rate of 480Hz; and the double-speed DLP projector

with RGBRGB6 phase color wheel has a rate of 240Hz.

However, these projectors remain outputting with the refresh

rate of 120Hz. For instance, the Panasonic PT-HZ900

projects each image four times to perfect crosstalk and

residue shadow, while the 240Hz DLP projector projects

each image twice to improve the rainbow effect.

Theoretically, higher refresh rate of the above types of

projectors may be used to extend user capacity. The second

problem lies in the open/close switch rate of the shutter

glasses. If the number of users increases, the open/close

switch rate of the shutter should be improved in response,

and the experiment has revealed overdriving zero crossing

driving might be used for the open/close switch rate of LC

shutter to support 360Hz or higher. There is much room for

improvement regarding multi-user projection display.


Paper 599 Page 8

ACKNOWLEDGMENTS

We would like to thank all reviewers for their valuable

comments. This work is supported by the China National

Key Research and Development Project (2016YFB1001403),

the National Natural Science Foundation of China under

Grant (61472225), the Shandong Provincial Science and

Technology Development Program (2016GGX106001), and

the China Postdoctoral Science Foundation (2017M620284).

REFERENCES

1. Maneesh Agrawala, Andrew C. Beers, Bernd Frohlich,

Pat Hanrahan, Ian McDowall, and Mark Bolas. 1997.

The two-user Responsive Workbench: support for

collaboration through individual views of a shared

space. In Proceedings of the 24th annual conference on

Computer graphics and interactive techniques, 327-

332. http://doi.org/10.1145/258734.258875

2. Barco. 2017. Virtual surgery table. Retrieved August

22, 2017 from http://www.barco.com/en

3. Stephan Beck, André Kunert, Alexander Kulik, and

Bernd Froehlich. 2013. Immersive group-to-group

telepresence. IEEE Transactions on Visualization and

Computer Graphics 19, 4: 616-625.

4. Wei Y. Chen, Nicolas Ladeveze, Céline Clavel, Daniel

Mestre, and Patrick Bourdot. 2015. User cohabitation

in multi-stereoscopic immersive virtual environment

for individual navigation tasks. In Virtual Reality (VR),

47-54. http://dx.doi.org/10.1109/VR.2015.7223323

5. Niranjan Damera-Venkata, and Nelson L. Chang.

2007. Realizing super-resolution with superimposed

projection. In Computer Vision and Pattern

Recognition, 1-8.

http://dx.doi.org/10.1109/CVPR.2007.383463

6. Sunkwang Hong, B. Berkeley, and Sang S. Kim. 2005.

Motion image enhancement of LCDs. In IEEE

International Conference on Image Processing, 11-17.

http://dx.doi.org/10.1109/ICIP.2005.1529980

7. Abhijit Karnik, Walterio Mayol-Cuevas, and Sriram

Subramanian . 2012. MUSTARD: a multi user see

through AR display. In Proceedings of the SIGCHI

Conference on Human Factors in Computing Systems

2541-2550.

http://doi.acm.org/10.1145/2207676.2208641

8. Alexander Kulik, André Kunert, Stephan Beck, Roman

Reichel, Roland Blach, Armin Zink, and Bernd

Froehlich. 2011. C1×6: a stereoscopic six-user display

for co-located collaboration in shared virtual

environments. ACM Transactions on Graphics 30, 6:

188.

9. Roman Lissermann, Jochen Huber, Martin Schmitz,

Jürgen Steimle, and Max Mühlhäuser. 2014. Permulin:

mixed-focus collaboration on multi-view tabletops. In

Proceedings of the 32nd annual ACM conference on

Human factors in computing systems 3191-3200.

http://dx.doi.org/10.1145/2556288.2557405

10. Jonathan Marbach. 2009. Image blending and view

clustering for multi-viewer immersive projection

environments. In Virtual Reality Conference 51-54.

http://dx.doi.org/10.1109/VR.2009.4810998

11. Wojciech Matusik, Clifton Forlines, and Hanspeter

Pfister. 2008. Multiview user interfaces with an

automultiscopic display. In Proceedings of the working

conference on Advanced visual interfaces 363-366.

http://doi.acm.org/10.1145/1385569.1385634

12. Brice Pollock, Melissa Burton, Jonathan W. Kelly,

Stephen Gilbert, and Eliot Winer. 2012. The right view

from the wrong location: Depth perception in

stereoscopic multi-user virtual environments. IEEE

Transactions on Visualization and Computer Graphics

18, 4: 581-588.

13. Tom Peterka, Robert L. Kooima, Javier I. Girado,

Jinghua Ge, Daniel J. Sandin, Andrew Johnson, Jason

Leigh, Jurgen Schulze, and Thomas A. DeFanti. 2007.

Dynallax: solid state dynamic parallax barrier

autostereoscopic VR display. In Virtual Reality

Conference 155-162.

http://dx.doi.org/10.1109/VR.2007.352476

14. Kai Riege, T. Holtkamper, G. Wesche, and B. Frohlich.

2006. The bent pick ray: An extended pointing

technique for multi-user interaction. In IEEE

Symposium on 3D User Interfaces 62-65.

15. Andreas Simon. 2005. First-person experience and

usability of co-located interaction in a projection-based

virtual environment. In Proceedings of the ACM

symposium on Virtual reality software and technology

23-30. http://doi.acm.org/10.1145/1101616.1101622

16. Jun Wang, Kyeongyuk Min, and Jongwha Chong.

2007. A hybrid image coding in overdriving for motion

blur reduction in LCD. In Entertainment Computing

263-270.

https://doi.org/10.1007/978-3-540-74873-1_32

17. Hong D. Yu, Hui Y. Li, Wei S. Sun, Wei Gai, Ting T.

Cui, Chu T. Wang, Dong D. Guan, Yi J. Yang, Cheng

L. Yang, and Wei Zeng. 2014. A two-view VR

shooting theater system. In Proceedings of the 13th

ACM SIGGRAPH International Conference on Virtual-

Reality Continuum and its Applications in Industry

223-226. https://doi.org/10.1145/2670473.2670497


Paper 599 Page 9

Documents

Two Kinds of Novel Multi-user Immersive Display Systems