Ide - Visual presentation of hand image modulates visuo–tactile temporal order judgment

7/27/2019 Ide - Visual presentation of hand image modulates visuotactile temporal order judgment

1/8

1 3

Exp Brain Res (2013) 228:4350

DOI 10.1007/s00221-013-3535-z

RESEARCH ARTICLE

Visual presentation of hand image modulates visuotactile

temporal order judgment

Masakazu Ide Souta Hidaka

Received: 2 January 2013 / Accepted: 17 April 2013 / Published online: 12 May 2013

Springer-Verlag Berlin Heidelberg 2013

and make them difficult to distinguish from each other in

the temporal domain.

Keywords Visuotactile interaction Audiovisual

interaction Temporal order judgment Visual hand

image Body image

Introduction

We constantly receive a variety of information through our

sensory organsthe eyes, ears, skin, etc.and organize

such information in order to create a systematic perception

of this world (Stein and Meredith 1993). For this purpose,

we must discriminate between the information derived

from the same sources and that derived from different

sources or objects. Temporal consistency/synchronization

is a particularly important cue involved in doing this (Cal-

vert et al. 2004; Driver and Spence 2000). The manner in

which the perceptual systems distinguish among a variety

of inputs in the temporal domain among different sensory

modalities have been examined by applying a direct tem-

poral task such as a temporal order judgment (TOJ) task

in which participants are asked to directly judge relation-

ships between two or a few stimuli in the temporal domain

(Shore and Spence 2005; Spence et al. 2001). In typical

TOJ tasks, two stimuli are presented with some stimulus

onset asynchronies (SOAs), and participants are asked to

judge which stimulus was presented first. Just noticeable

difference (JND) is a widely used index to estimate the crit-

ical limit within which the participants cannot accurately

discriminate the temporal order of the stimuli. Thus, a

larger JND indicates worse TOJ (Hirsh and Sherrick1961).

For example, the JND of a TOJ between a simple visual

stimulus and a simple tactile stimulus (visuotactile TOJ,

Abstract Perceptual systems can distinguish among a

variety of inputs in the temporal domain, including evendifferent sensory inputs. This process has been investigated

mainly by using a temporal task (temporal order judg-

ment: TOJ). For example, studies have reported estimated

critical limits (just noticeable difference: JND) of the TOJ

between a visual stimulus and a tactile stimulus (visuo

tactile TOJ, e.g., flashes and vibrations) fell within a cer-

tain temporal range. Recent studies have also suggested

that the visual presentation of a hand image could modulate

visuotactile integrations in the temporal domain, but these

studies did not thoroughly examine such effects by using

temporal tasks. Here, we investigated the effect of visual

presentation of a hand image on visuotactile TOJ. In our

experiments, a visual stimulus was presented on the index

finger of a hand image and a tactile stimulus was presented

on the index finger of a participants hand. We found that

the JND of visuotactile TOJ became larger when a for-

ward hand image was presented than when inverted hand

or arrow images were presented. However, this effect was

not observed for the TOJ between an auditory stimulus and

a visual stimulus. Thus, the visual presentation of a hand

image whose angle corresponds to that of ones own hand

could selectively degrade visuotactile TOJ. This finding

indicates that visual hand images implicitly enhance the

internal proximity between the visual and tactile stimuli

Electronic supplementary material The online version of this

article (doi:10.1007/s00221-013-3535-z) contains supplementary

material, which is available to authorized users.

M. Ide (*) S. Hidaka

Department of Psychology, Rikkyo University, 1-2-26, Kitano,

Niiza-shi, Saitama 352-8558, Japan

e-mail: [email protected]
http://dx.doi.org/10.1007/s00221-013-3535-zhttp://dx.doi.org/10.1007/s00221-013-3535-z


2/8

44 Exp Brain Res (2013) 228:4350

1 3

e.g., flashes and vibrations) was estimated to be approxi-

mately 20 ms for participants who were trained and given

feedback (e.g., Hirsh and Sherrick 1961; see also Harrar

and Harris 2008; Spence et al. 2001; Spence et al. 2003).

Using a more obscure visual stimulus (Gaussian blob),

Fujisaki and Nishida (2009) reported that the JND of a

visuotactile TOJ was approximately 80 ms. Despite some

variations based on sensory pairs or experimental situa-tions, studies have consistently demonstrated that the esti-

mated critical limits (i.e., JND) of TOJ tasks fell within a

certain temporal range (i.e., approximately 2080 ms).

It has been also reported that proximities between sen-

sory inputs have modulatory effects on TOJ tasks. For

example, Spence et al. (2003) reported that a visuotactile

TOJ and a TOJ between a simple auditory stimulus and a

simple visual stimulus (audiovisual TOJ) became more

precise when these stimulus pairs were presented at dif-

ferent positions than when they were presented at a cor-

responding location. Vatakis et al. (2007) also showed that

the JND of an audiovisual TOJ was larger when the targetauditory and visual stimuli were presented within a tempo-

ral stream of audiovisual distracters than when they were

presented in isolation. Moreover, Parise and Spence (2009)

found that the JND of an audiovisual TOJ was larger

when the stimuli were presented as having crossmodal con-

gruency (e.g., large visual stimuli and low-pitched sound)

than when they were presented as an incongruent pair (e.g.,

large visual stimuli and high-pitched sound) (see also Bien

et al. 2012). These findings suggest that the spatial, tem-

poral, or synesthetic proximity of stimuli have modulatory

effects on direct temporal tasks.

A recent study has demonstrated that the presentation

of a hand image has modulatory effects on the tempo-

ral aspects of a tactile judgment (i.e., changes in reaction

time). Igarashi et al. (2004) presented a tactile stimulus

(vibration) on the tip or base position of the index finger of

participants hands and asked participants to judge the posi-

tion of the vibration as soon as possible. Concurrently, a

line drawing of a hand image was presented in front of the

participants, and a visual distractor (circle) was presented

at either the tip or the base position of the index finger of

the hand image. The reaction time to the tactile position

judgment increased or decreased when the position of the

visual distracter in the hand image was congruent or incon-

gruent with that of the tactile stimulus on the participants

hand, respectively. Further, this spatial congruency effect

was found to be stronger when the angle of the hand image

was within the range in which a participants hand could

be rotated, relative to when the angle of the hand was out

of this range. The presentation of a hand image could also

modulate visuotactile interactions in the temporal domain.

Shimada et al. (2009) investigated the temporal window

of the rubber hand illusion (Botvinick and Cohen 1998;

Ehrsson et al. 2004). As in a typical rubber hand illusion

technique, they presented a dummy hand in front of each

participant, and the participants hand was hidden from

view. The dummy and the participants hand were simul-

taneously stroked using paintbrushes. The participants per-

ceived that the stroke arose from the stroked position of

the dummy hand rather than from that of the participants

actual hand. Importantly, it was found that the rubber handillusion could reliably occur even when the SOA between

the visual (stimulation of the dummy hand) and tactile

(stimulation of the participants actual hand) stimuli was

300 ms. For the occurrence of this illusion, it is important

that the participants do not detect a temporal gap between

the visual and tactile stimuli (Armel and Ramachandran

2003; Ehrsson et al. 2004; Tsakiris and Haggard 2005).

Thus, the presentation of a hand image could degrade the

detectability of temporal asynchrony between a visual stim-

ulus and a tactile stimulus far larger than that reported for

simple stimuli (2080 ms).

While these findings indicate that the visual presenta-tion of a hand image has modulatory effects on visuo

tactile interactions in the temporal domain, these effects

have not been thoroughly examined using direct temporal

tasks. To fill this gap, the current study investigated whether

the visual presentation of a hand image could modulate

visuotactile TOJ. A visual stimulus was presented on the

index finger of a hand image, and a tactile stimulus was

presented on the index finger of a participants hand. We

found that the JND of the visuotactile TOJ became larger

when a forward hand image (whose angle corresponded to

that of a participants hand) was presented than when the

inverted hand or arrow images were presented (Experiment

1). However, this effect was not observed for the audio

visual TOJ (Experiment 2). Thus, the visual presentation

of a hand image, whose angle was corresponded to that of

ones own hand, could selectively degrade the visuotactile

TOJ, indicating that visual hand images would implic-

itly enhance the internal proximity between the visual and

tactile stimuli and make them difficult to distinguish from

each other in the temporal domain.

Experiment 1

The aim of Experiment 1 was to investigate the effects of the

visual presentation of a hand image on visuotactile TOJ.

In the experimental condition, we presented a forward hand

image. We also introduced two control conditions in which

an inverted hand and arrow images were presented. This was

done to investigate whether the simple presentation of hand-

like images or directional information of the forward hand

image could also be effective. A situation without the pres-

entation of images was also included as a Baseline condition.


3/8

45Exp Brain Res (2013) 228:4350

1 3

Methods

Participants and apparatus

Written consent was obtained from each participant before

the experiments were conducted. The experiments were

approved by the local ethics committee at Rikkyo Univer-

sity. Ten volunteers and the two authors were involved inExperiment 1. All the participants had normal or corrected-

to-normal vision.

The experiment was controlled by the Dell Precision

T5500 workstation and MATLAB (MathWorks) with the

Psychophysics Toolbox (Brainard 1997; Pelli 1997). The

visual stimuli were presented on a CRT display (EIZO,

FlexScan T566, 17 inch) with a resolution of 1,024 768

pixels and refresh rate of 100 Hz. The participants observed

the display with their heads placed on a chinrest. The tactile

stimuli were presented through an audio interface (Roland,

EDIROL FA-66) and an attachable speaker (Eishindenki,

Attachable Speaker M-PZT-02) for generating vibration.An acrylic box filled with sponge was attached to the

speaker in order to present the stimuli solely on the tip of

a participants index finger. An opaque shield occluded the

participants left hand and the vibrator. We confirmed that

the onset of the visual and tactile stimuli was synchronized

by using a digital oscilloscope (OWON, PDS5022TFT).

The experiment was conducted in a dark room.

Stimulus

A red circle (0.6; 27.85 cd/m2) was presented as a fixa-

tion point on a white background (110.4 cd/m2; Fig. 1a).

A gray circle (1 in diameter; 72.43 cd/m2) was pre-

sented at an eccentricity of 5 along the horizontal plane

for 10 ms. Three images (forward hand, inverted hand,

and arrow; Fig. 1b) were presented as black line draw-

ings (12.7 8.5). A vibration stimulus (400-Hz sinusoi-

dal burst) was presented on the tip of each participants left

index finger on the palm side for 10 ms with 1 ms of cosine

ramp at the onset and offset. The gray circle and the vibra-

tion were presented concurrently and transiently, while the

fixation point and the images were constantly presented

during a trial. These parameters, which were in accordance

with our preliminary experiment, were selected to control

for any possible visual dominance effects; visual stimuli

tended to be judged as temporally preceding stimuli pre-

sented in a different modality (Rutschmann and Link1964;

Smith 1933). Therefore, the intensities of a multisensory

stimulus need to be modified appropriately in order for

them to be judged as simultaneous when they are presented

concurrently (Jaskowski et al. 1990). We thus modified the

parameters such that the intensity of the tactile stimulus

was equivalent to that of the visual stimulus.

Procedure

We asked each participant to place his/her own left hand

with the index finger up on the tactile device and then to

look at the fixation circle. There were four image condi-

tions (Fig. 1b): In the Forward-hand condition, the left-

hand image was presented and its angle was kept consistent

with that of the participants own left hand. In contrast, in

theInverse-hand condition, the hand image was rotated by

180 such that the angle of the hand image did not corre-

spond to that of the participants own hand. In the Arrow

condition, an arrow image containing similar directional

information as the Forward-hand condition was presented.

These conditions were tested in a separate block (within-

participants block design), and each image was constantly

presented during each block. We also introduced a situation

without the presentation of images, and this was called the

Baseline condition. The position of the visual and tactile

stimuli was fixed in all image conditions. After an interval

Fig. 1 A schematic illustration of the experimental conditions in

Experiment 1. a A gray circle was consistently presented at an eccen-

tricity of 5 from the fixation point. A vibration was presented on

the tip of each participants left index finger, with an opaque shield

occluding their hand. Both the gray circle and the vibration were pre-

sented for 10 ms each with various SOAs. b The image conditions.

A left-hand image having an angle consistent with that of each par-

ticipants own left hand (forward-hand condition; a); a hand image

rotated by 180 as compared to the position of the participants own

hand (Inverse-hand condition; b); an arrow image pointing to the

same direction as in the Forward-hand condition (Arrow condition;

c); no presentation of images (Baseline condition; d) (color figure

online)


4/8

46 Exp Brain Res (2013) 228:4350

1 3

randomly assigned in each trial (2,000, 2,080, or 2,100 ms),

the visual stimuli were presented with one of the follow-

ing stimulus onset asynchronies (SOAs):40, 80, 160,

and 320 ms. Negative values indicate a situation in which

the visual stimulus was presented first, and positive values

indicate a situation in which the tactile stimulus was pre-

sented first. The participants were asked to judge which

stimulus was presented first (visual or tactile) by pressing acorresponding key.

The experiment consisted of two sessions. In the prac-

tice session, the participants completed 24 trials [SOAs

(8) Repetitions (3)] without images (Baseline condition).

The main session consisted of 640 trials: Image conditions

(4) SOAs (8) Repetitions (20). The order of the condi-

tions (blocks) was randomized and counterbalanced among

the participants. SOAs were also randomly introduced

in each trial and counterbalanced among the blocks. The

experiment lasted approximately 45 min.

Results and discussion

We plotted averaged proportions of tactile first

responses as a function of SOAs in each image condi-

tion (Fig. 2a) to obtain psychometric functions. Then,

we estimated the 75 and 25 % threshold points by fit-

ting a cumulative Gaussian distribution function onto

each participants data by using the maximum likeli-

hood method and calculated the slope of the functions

as JND by using the following formula: (75 % thresh-

old25 % threshold)/2. In order to compare JNDs among

the image conditions, we calculated normalized JNDs by

subtracting the JND of each image condition from that

of the Baseline condition (Fig. 2b). A one-way repeated-

measures analysis of variance (ANOVA) indicated a

significant main effect among these image conditions

[F(2, 22) = 4.93, p < .05]. A post hoc test (Tukeys

HSD, p < .05) revealed that the JND of the Forward-

hand condition was larger than that of the Inverse-hand

and Arrow conditions.

Since the simple presentation of the hand-like image

(the inverted hand image) and the directional information

of the forward hand image (the arrow image) did not seem

to have any effect (Fig. 2b), we believe that the visual pres-

entation of the hand image, whose angle was corresponded

to that of ones own hand (the forward hand image), selec-

tively degraded the TOJ between the visual stimulus and

the tactile stimulus. Previous studies reported that the inter-

actions between visual and tactile stimuli occurred espe-

cially when the angle of the visual hand image was within

the range in which the participants hands could be rotated

(Ide 2013; Igarashi et al. 2004). In line with these findings,

the forward hand image was found to dominantly have a

modulatory effect.

Experiment 2

The TOJ between the visual stimulus and the tactile stimulus

may have degraded in Experiment 1 merely because the vis-

ual presentation of the forward hand image simply attracted

visual attention. To check this possibility, we investigated

whether the visual presentation of the forward hand image

could also modulate the TOJ between an auditory stimulus

and a visual stimulus. We hypothesized that the modula-

tory effect of the forward hand image found in Experiment 1

could also be observed on the audiovisual TOJ if the atten-

tional distraction/capture effect acted as a decisive factor.

Fig. 2 The results of Experiment 1. a Averaged proportions of tac-

tile first responses as a function of SOAs for the four image con-

ditions. The 75 and 25 % threshold points were estimated, and the

slope of the functions was calculated as JND by using the following

formula: (75 % threshold25 % threshold)/2. b Normalized JNDs of

the image conditions versus the Baseline condition. The error bars

denote the standard error of the mean (N= 12). An asteriskindicates

the condition that was significantly different from the others (p < .05)


5/8

47Exp Brain Res (2013) 228:4350

1 3

Methods

Ten volunteers and the two authors were involved in

Experiment 2. All participants had normal or corrected-

to-normal vision. Seven volunteers were newly recruited

for this experiment. In this experiment, a white noise burst

(10 ms) was monaurally presented through an audio inter-

face and a headphone (Sennheiser, HDA 200; Fig. 3). Thisauditory stimulus was presented at a 70-dB sound pressure

level with 1 ms of cosine ramp at the onset and offset. The

vibration was not presented in this experiment, but the par-

ticipants placed their left hands with the index finger up on

the acrylic box, as in Experiment 1. The participants were

asked to judge which stimulus (auditory or visual) was pre-

sented first. Except for the stated differences, the appara-

tus, stimuli, and procedures of Experiments 1 and 2 were

identical.

Results and discussion

We plotted averaged proportions of auditory first

responses as a function of SOAs in each image condition

(Fig. 4a) to obtain psychometric functions. As in Experi-

ment 1, we then calculated normalized JNDs (Fig. 4b).

A one-way repeated-measures ANOVA did not indicate a

significant main effect [F(2, 22) = 1.67,p= .21].

The results showed that JNDs were not significantly dif-

ferent among the image conditions, thereby indicating that

the visual presentation of the forward hand image did not

modulate the TOJ between the auditory stimulus and the

visual stimulus. Therefore, we could assume that the atten-

tional distraction/capture effect induced by the forward

hand image would not be a decisive factor for the results

obtained in Experiment 1, and that the forward hand image

could be effective only with regard to the TOJ between the

visual stimulus and the tactile stimulus.

One might assume that, in Experiment 2, the JND of

the Forward-hand and the Inverse-hand conditions would

appear to be larger than that of the Arrow condition such

that these differences would be similar to the difference

between the Forward-hand and the Arrow conditions in

Experiment 1. In order to confirm these possibilities, we

calculated the statistical power (1 ) and the effect sizes

Fig. 3 A schematic illustration of the experimental conditions in

Experiment 2. A white noise burst was presented via a headphone.

Except this, all other experimental conditions were consistent with

those in Experiment 1 (see Fig. 1)

Fig. 4 Results of Experiment 2. a Averaged proportions of auditory

first responses as a function of SOAs for the four image conditions.

JNDs were calculated using the same formula as that used for Experi-

ment 1. b Normalized JNDs of the image conditions. Error bars

denote the standard error of the mean (N= 12)


6/8

48 Exp Brain Res (2013) 228:4350

1 3

of the results of Experiments 1 and 2 and compared them

by referring to the criteria suggested by Cohen (1988)

and Ellis (2010). With regard to the results of Experi-

ment 1, the one-way repeated-measures ANOVA con-

ducted had high statistical power (0.99). The effect size

was not small (2p= 0.31). The JND of the Forward-hand

condition had medium (dD= 0.58) and large (dD= 1.07)

effect sizes compared to the Inverse-hand and the Arrowconditions, respectively. In contrast, the effect size was

negligible (dD = 0.17) between the Inverse-hand and

the Arrow conditions. As for Experiment 2, the one-way

repeated-measures ANOVA conducted also had sufficient

statistical power (0.72). However, the effect size was neg-

ligible (2p= 0.13). We also found that the effect sizes

between the Forward-hand and the Arrow conditions and

the Inverse-hand and the Arrow conditions were below

medium level (dD= 0.39 and dD= 0.49, respectively).

The effect size between the Forward-hand and the Inverse-

hand conditions was negligible (dD= 0.05). These data

confirmed that the statistical power of the ANOVAs wassufficient in both experiments, indicating that our results,

based on probability statistics, would be reliable. Consist-

ent with the probability statistics results, we also found

that while the effect sizes of Experiment 1 were above the

small level for the ANOVA and above the medium or large

level between the Forward-hand and the other conditions,

those of Experiment 2 were below the small level for the

ANOVA and below the medium level between the For-

ward-hand and the Arrow conditions and the Inverse-hand

and the Arrow conditions. We could thus conclude that the

visual presentation of the forward hand image could be

effective, especially for visuotactile TOJ.

General discussion

A recent study suggested that the presentation of a hand

image has modulatory effects on visuotactile interactions

in the temporal domain (Shimada et al. 2009). However,

that study did not employ direct temporal tasks. The cur-

rent study investigated the effect of visual presentation of

a hand image in a direct temporal task to assess the TOJ

between the visual stimulus and the tactile stimulus. The

results showed that the JNDs of the visuotactile TOJ were

larger when the forward hand image was presented than

when the inverted hand image or the arrow image was pre-

sented (Experiment 1). In contrast, the JNDs of the TOJ

between the auditory stimulus and the visual stimulus did

not differ among these image conditions (Experiment 2).

These findings suggest that the visual presentation of the

hand image corresponding to the angle of ones own hand

(the forward hand image) could selectively degrade the

visuotactile TOJ.

In Experiment 1, we confirmed that the simple presen-

tation of a hand-like image (the inverted hand image) and

the directional information of a forward hand image (the

arrow image) did not affect the visuotactile TOJ. One

might assume that the visual presentation of a hand image

similar to ones own hand (the forward hand image) selec-

tively attracted visual attention to the display, and thus, the

TOJ between the visual stimulus and the tactile stimuluswas degraded. If this were the case, the visual presentation

of the forward hand image would have also degraded the

audiovisual TOJ. However, the results of Experiment 2

did not indicate any such effect. Therefore, we concluded

that the effect of visual presentation of the forward hand

image observed in Experiment 1 might not be explained

solely by attentional distraction/capture effects.

A notable point is that only the forward hand image,

whose angle corresponded to that of the participants own

hand, could have a modulatory effect on the visuotactile

TOJ. Similarly, Igarashi et al. (2004) reported that the spa-

tial congruency effect between the visual distractors pre-sented on a hand image and the tactile stimuli presented

on the participants own hands became stronger when the

angle of the hand image was within the range in which

the participants hands could be rotated. Ide (2013) also

showed that the rubber hand illusion dominantly occurred

when a dummy hand image rotated at 0, 45, 90, and

315 in a clockwise direction. These angles were firmly

corresponding to the range of motion of the participants

own hands. Our finding that the forward hand image domi-

nantly had a modulatory effect on the visuotactile TOJ is

consistent with these previous findings.

Our results could appear to be explained by the percep-

tual degradation effect of a visual hand image on tactile

stimuli. While the visual hand or body image enhanced

discrimination/detection performance for supra-threshold

tactile stimuli (Kennett et al. 2001; Taylor-Clarke et al.

2004; Tipper et al. 1998), the image also showed an inter-

ference effect on performance when a tactile stimulus was

presented near the threshold (Harris et al. 2007; Mirams

et al. 2010). It could thus be assumed that the forward hand

image selectively degraded the perceived intensity of the

tactile stimulus such that the TOJ between a visual and a

tactile stimulus was impaired. However, in the current

study, the tactile stimulus was presented at supra-threshold

and was clearly perceived; we modified the intensity of

the tactile stimulus such that it was as clearly and equally

strong as the visual stimulus, in order to exclude any pos-

sible visual dominance effects on the results (see also the

Methods section in Experiment 1). We further con-

firmed that there was no biasing effect; if the perceived

intensity of the tactile stimulus was degraded and partici-

pants attention or preference was directed to the visual

stimulus, then a consistent visual first response would


7/8

49Exp Brain Res (2013) 228:4350

1 3

be obtained. Consequently, the point of subjective simul-

taneity (PSS) of the psychometric functions would shift

(Spence and Parise 2010). However, we did not find any

differences in PSS between the conditions (Supplementary

Fig. 1). Although it was reported that the visual hand image

increased somatosensory intracortical inhibition, especially

when two adjacent fingers were stimulated simultaneously

(Cardini et al. 2011), the tactile stimulus was delivered toa single finger in our experiment. On the basis of these

results, we conclude that the degradation effect of the vis-

ual hand image on tactile stimuli cannot fully explain the

current findings.

It has been demonstrated that the internal proximity

of stimuli has modulatory effects on multisensory TOJ

(Bertelson and Aschersleben 2003; Keetels and Vroomen

2005; Spence et al. 2003; Zampini et al. 2005). In par-

ticular, TOJ was degraded when an auditory stimulus and

a visual stimulus or a visual stimulus and a tactile stimu-

lus were presented at a spatially close position, relative

to when they were spatially apart (Spence et al. 2003).Vatakis et al. (2007) also reported that an audiovisual

TOJ became less precise when the stimuli were presented

within a temporal stream of audiovisual distracters than

when they were presented in isolation. In addition, Par-

ise and Spence (2009) found that audiovisual TOJ was

degraded for the stimuli with crossmodal synesthetic

congruency rather than for those without such congru-

ency. These findings indicate that multisensory TOJ was

degraded when the internal proximity of stimuli was

maintained or enhanced. Interestingly, in the situation

where the stimuli were spatially apart, unimodal visual

TOJ still worsened when the stimuli were perceptually

grouped (Nicol and Shore 2007). On the basis of these

findings, we assume that in the current study, the for-

ward hand image would enhance the internal proximity

between the visual stimulus presented on the hand image

and the tactile stimulus presented on the participants own

hand (note that these stimuli were consistently presented

on the index finger of the hand image and the actual

hand.) This would make the stimuli perceptually grouped

and/or difficult to distinguish from each other in the tem-

poral domain, thus impairing the visuotactile TOJ. This

idea would also be supported by the current results show-

ing that the impairment of visuotactile TOJ was observed

for only the forward hand image and that the effect selec-

tively occurred for visuotactile stimuli (not for audio

visual stimuli).

Some neurophysiological studies have provided evi-

dence supporting the present findings (Duhamel et al.

1998; Graziano 1999; Iriki et al. 1996). For example, Iriki

et al. (1996) showed that monkeys neurons in the intra-

parietal sulcus (IPS) were activated by not only the tac-

tile stimulus but also the visual stimulus that appeared to

move so as to touch their hands. Furthermore, after mon-

keys were trained to move their hands while viewing their

hands on a display, the neurons in the IPS were activated

solely by the visual hand image on the display (Iriki et al.

2001). Similarly, in a functional magnetic resonance imag-

ing (fMRI) study, the activation of the human IPS was

modulated when the dummy hand image was presented

at the actual hand position, as well as on the presenta-tion of actual hand image (Makin et al. 2007). Graziano

et al. (2000) also reported that monkeys neurons in area 5

(superior parietal lobe), which were generally activated by

visual and somatosensory information of their own hands,

were also strongly activated when monkeys saw a dummy

hand placed just above their own hands (which were out of

view). These findings indicate that visual stimulation to the

hand image and tactile stimulation to ones own hand could

be represented as similar inputs in the brain. Using some

psychophysical and neuropsychological techniques, future

research should investigate in detail whether the internal

proximity between the visual and tactile stimuli and thepossibly related IPS activations could actually be enhanced

by the presentation of the forward hand image.

The current study showed that the visual presentation

of a hand image impaired TOJ between the visual stimulus

presented on the hand image and the tactile stimulus pre-

sented on the participants own hands only when the hand

image whose angle was corresponded to that of ones own

hand was presented. Since this effect was not observed

for the audiovisual TOJ, attentional distraction/attraction

to the forward hand image could not be a decisive factor

in this result. In addition, since the maintenance/enhance-

ment of the internal proximity of multimodal stimuli could

degrade the TOJ of those stimuli, and the visual hand

images and visual stimulation to the hand images could

be represented as related to or similar to the touch-related

stimuli in the brain, our results suggest that the visual

presentation of the hand image corresponding to that of

ones own hand implicitly enhances the internal proximity

between visual and tactile stimuli and makes them difficult

to distinguish from each other in the temporal domain.

Acknowledgments We thank Miho Kondo for her support in con-

ducting the experiments and Shoko Yabukis help in preparing the fig-

ures included in this paper. We are also grateful to the two anonymousreviewers for their valuable and insightful comments. This study was

supported by a Grant-in-Aid from Rikkyo University Special Fund for

Research (No. 18120030) and a Grant-in-Aid for Specially Promoted

Research from JSPS (No. 19001004).

References

Armel KC, Ramachandran VS (2003) Projecting sensations to exter-

nal objects: evidence from skin conductance response. Proc R

Soc Land Biol Sci 270:14991506


8/8

50 Exp Brain Res (2013) 228:4350

1 3

Bertelson P, Aschersleben G (2003) Temporal ventriloquism: cross-

modal interaction on the time dimension. 1. Evidence from

auditory-visual temporal order judgment. Int J Psychophysiol

50:147155

Bien N, ten Oever S, Goebel R, Sack AT (2012) The sound of size:

crossmodal binding in pitch-size synesthesia: a combined TMS,

EEG and psychophysics study. Neuroimage 59:663672

Botvinick M, Cohen J (1998) Rubber hands feel touch that eyes see.

Nat 391:756

Brainard DH (1997) The Psychophysics Toolbox. Spat Vis

10:433436

Calvert GA, Spence C, Stein BE (eds) (2004) The hand book of mul-

tisensory processes. MIT Press, Cambridge

Cardini F, Longo MR, Haggard P (2011) Vision of the body modulates

somatosensory intracortical inhibition. Cereb Cor 21:20142022

Cohen J (1988) Statistical power analysis for the behavioral sciences,

2nd edn. Lawrence Erlbaum, New Jersey

Driver J, Spence C (2000) Multisensory perception: beyond modular-

ity and convergence. Curr Biol 10:731735

Duhamel J, Colby CI, Goldberg ME (1998) Ventral intraparietal area

of the macaque: congruent visual and somatic response proper-

ties. J Neurophysiol 79:126136

Ehrsson HH, Spence C, Pashingham RE (2004) Thats my hand!

Activity in premotor cortex reflects feeling of ownership of a

limb. Science 305:875877

Ellis PD (2010) The essential guide to effect sizes: statistical power,

meta-analysis, and the interpretation of research results. Cam-

bridge University Press, New York

Fujisaki W, Nishida S (2009) Audio-tactile superiority over visuotac-

tile and audiovisual combinations in the temporal resolution of

synchrony perception. Exp Brain Res 198:245259

Graziano MSA (1999) Where is my arm? The relative role of vision

and proprioception in the neuronal representation of limb posi-

tion. PNAS 96:1041810421

Graziano MSA, Cooke DF, Taylor CSR (2000) Coding the location of

the arm by sight. Science 290:7821786

Harrar V, Harris LR (2008) The effect of exposure to asynchronous

audio, visual, and tactile stimulus combinations on the perception

of simultaneity. Exp Brain Res 186:517524Harris JA, Arabzadeh E, Moore CA, Clifford CWG (2007) Nonin-

formative vision causes adaptive changes in tactile sensitivity.

J Neurosci 27:71367140

Hirsh IJ, Sherrick CE Jr (1961) Perceived order in different sense

modalities. J Exp Psychol 62:423432

Ide M (2013) The effect of anatomical plausibility of hand angle on

the rubber hand illusion. Percept 42:103111

Igarashi Y, Kitagawa N, Ichihara S (2004) Vision of pictorial hand

modulates visual-tactile interactions. Cogn Affect Behav Neuro-

sci 4:182192

Iriki A, Tanaka M, Iwamura Y (1996) Coding of modified body

schema during tool use by macaque postcentral neurons. Neu-

roReport 7:23252330

Iriki A, Tanaka M, Obayashi S, Iwamura Y (2001) Self-images in the

video monitor coded by monkey intraparietal neurons. NeurosciRes 40:163173

Jaskowski P, Jaroszyk F, Hojan-Jezierska D (1990) Temporal-order

judgments and reaction time for stimuli of different modalities.

Psychol Res 52:3538

Keetels M, Vroomen J (2005) The role of spatial disparity and hemi-

fields in audio-visual temporal order judgments. Exp Brain Res

167:635640

Kennett S, Taylor-Clarke M, Haggard P (2001) Noninformative vision

improves the spatial resolution of touch in humans. Curr Biol

11:11881191

Makin TR, Holmes NP, Zohary E (2007) Is that near my hand? Multi-

sensory representation of peripersonal space in human intrapari-

etal sulcus. J Neurosci 27:731740

Mirams L, Poliakoff E, Brown J, Lloyd DM (2010) Vision of body

increase interference on the somatic signal detection task. Exp

Brain Res 202:787794

Nicol JR, Shore DL (2007) Perceptual grouping impairs temporal res-

olution. Exp Brain Res 183:141148

Parise CV, Spence C (2009) When birds of a feather flock together:

synesthetic correspondences modulate audiovisual integration in

non-synesthetes. PLoS ONE 4:e5664

Pelli DG (1997) The Video Toolbox software for visual psychophys-

ics: transforming numbers into movies. Spat Vis 10:437442

Rutschmann J, Link R (1964) Perception of temporal order of stimuli

differing in sense mode and simple reaction time. Percept Mot

Skills 18:345352

Shimada S, Fukuda K, Hiraki K (2009) Rubber hand illusion under

delayed visual feedback. PLoS ONE 4:e6185

Shore DL, Spence C (2005) Prior entry. In: Itti L, Rees G, Tsotsos JK

(eds) Neurobiology of attention. Elsevierm, Academic Press, San

Diego, pp 8995

Smith WF (1933) The relative quickness of visual and auditory per-

ception. J Exp Psy 16:239257

Spence C, Parise C (2010) Prior entry: a review. Conscious Cogn

19:364379

Spence C, Shore DI, Klein RM (2001) Multisensory prior entry. J Exp

Psychol Gen 130:799832

Spence C, Baddeley R, Zampini M, James R, Shore DI (2003) Multi-

sensory temporal order judgments: when two locations are better

than one. Percept Psychophys 65:318328Stein BE, Meredith MA (1993) The merging of the senses. MIT

Press, Cambridge

Taylor-Clarke M, Kennett S, Haggard P (2004) Persistence of visual-

tactile enhancement in humans. Neurosci Lett 354:2225

Tipper SP, Lloyd D, Shorland B, Dancer C, Howard LA, McGlone F

(1998) Vision influences tactile perception without proprioceptive

orienting. Neuro Rep 9:17411744

Tsakiris M, Haggard P (2005) The rubber hand illusion revised: visu-

otactile integration and self-attribution. J Exp Psychol Hum Per-

cept Perform 31:8091

Vatakis A, Bayliss L, Zampini M, Spence C (2007) The influence

of synchronous audiovisual distractors on audiovisual temporal

order judgments. Percept Psychophys 69:298309

Zampini M, Brown T, Shore DI, Maravita A, Roder B, Spence C

(2005) Audiotactile temporal order judgments. Acta Psychol118:277291

Documents

Ide - Visual presentation of hand image modulates visuo–tactile temporal order judgment