Ide - Visual presentation of hand image modulates visuo–tactile temporal order judgment

Embed Size (px)

Citation preview

  • 7/27/2019 Ide - Visual presentation of hand image modulates visuotactile temporal order judgment

    1/8

    1 3

    Exp Brain Res (2013) 228:4350

    DOI 10.1007/s00221-013-3535-z

    RESEARCH ARTICLE

    Visual presentation of hand image modulates visuotactile

    temporal order judgment

    Masakazu Ide Souta Hidaka

    Received: 2 January 2013 / Accepted: 17 April 2013 / Published online: 12 May 2013

    Springer-Verlag Berlin Heidelberg 2013

    and make them difficult to distinguish from each other in

    the temporal domain.

    Keywords Visuotactile interaction Audiovisual

    interaction Temporal order judgment Visual hand

    image Body image

    Introduction

    We constantly receive a variety of information through our

    sensory organsthe eyes, ears, skin, etc.and organize

    such information in order to create a systematic perception

    of this world (Stein and Meredith 1993). For this purpose,

    we must discriminate between the information derived

    from the same sources and that derived from different

    sources or objects. Temporal consistency/synchronization

    is a particularly important cue involved in doing this (Cal-

    vert et al. 2004; Driver and Spence 2000). The manner in

    which the perceptual systems distinguish among a variety

    of inputs in the temporal domain among different sensory

    modalities have been examined by applying a direct tem-

    poral task such as a temporal order judgment (TOJ) task

    in which participants are asked to directly judge relation-

    ships between two or a few stimuli in the temporal domain

    (Shore and Spence 2005; Spence et al. 2001). In typical

    TOJ tasks, two stimuli are presented with some stimulus

    onset asynchronies (SOAs), and participants are asked to

    judge which stimulus was presented first. Just noticeable

    difference (JND) is a widely used index to estimate the crit-

    ical limit within which the participants cannot accurately

    discriminate the temporal order of the stimuli. Thus, a

    larger JND indicates worse TOJ (Hirsh and Sherrick1961).

    For example, the JND of a TOJ between a simple visual

    stimulus and a simple tactile stimulus (visuotactile TOJ,

    Abstract Perceptual systems can distinguish among a

    variety of inputs in the temporal domain, including evendifferent sensory inputs. This process has been investigated

    mainly by using a temporal task (temporal order judg-

    ment: TOJ). For example, studies have reported estimated

    critical limits (just noticeable difference: JND) of the TOJ

    between a visual stimulus and a tactile stimulus (visuo

    tactile TOJ, e.g., flashes and vibrations) fell within a cer-

    tain temporal range. Recent studies have also suggested

    that the visual presentation of a hand image could modulate

    visuotactile integrations in the temporal domain, but these

    studies did not thoroughly examine such effects by using

    temporal tasks. Here, we investigated the effect of visual

    presentation of a hand image on visuotactile TOJ. In our

    experiments, a visual stimulus was presented on the index

    finger of a hand image and a tactile stimulus was presented

    on the index finger of a participants hand. We found that

    the JND of visuotactile TOJ became larger when a for-

    ward hand image was presented than when inverted hand

    or arrow images were presented. However, this effect was

    not observed for the TOJ between an auditory stimulus and

    a visual stimulus. Thus, the visual presentation of a hand

    image whose angle corresponds to that of ones own hand

    could selectively degrade visuotactile TOJ. This finding

    indicates that visual hand images implicitly enhance the

    internal proximity between the visual and tactile stimuli

    Electronic supplementary material The online version of this

    article (doi:10.1007/s00221-013-3535-z) contains supplementary

    material, which is available to authorized users.

    M. Ide (*) S. Hidaka

    Department of Psychology, Rikkyo University, 1-2-26, Kitano,

    Niiza-shi, Saitama 352-8558, Japan

    e-mail: [email protected]

    http://dx.doi.org/10.1007/s00221-013-3535-zhttp://dx.doi.org/10.1007/s00221-013-3535-z
  • 7/27/2019 Ide - Visual presentation of hand image modulates visuotactile temporal order judgment

    2/8

    44 Exp Brain Res (2013) 228:4350

    1 3

    e.g., flashes and vibrations) was estimated to be approxi-

    mately 20 ms for participants who were trained and given

    feedback (e.g., Hirsh and Sherrick 1961; see also Harrar

    and Harris 2008; Spence et al. 2001; Spence et al. 2003).

    Using a more obscure visual stimulus (Gaussian blob),

    Fujisaki and Nishida (2009) reported that the JND of a

    visuotactile TOJ was approximately 80 ms. Despite some

    variations based on sensory pairs or experimental situa-tions, studies have consistently demonstrated that the esti-

    mated critical limits (i.e., JND) of TOJ tasks fell within a

    certain temporal range (i.e., approximately 2080 ms).

    It has been also reported that proximities between sen-

    sory inputs have modulatory effects on TOJ tasks. For

    example, Spence et al. (2003) reported that a visuotactile

    TOJ and a TOJ between a simple auditory stimulus and a

    simple visual stimulus (audiovisual TOJ) became more

    precise when these stimulus pairs were presented at dif-

    ferent positions than when they were presented at a cor-

    responding location. Vatakis et al. (2007) also showed that

    the JND of an audiovisual TOJ was larger when the targetauditory and visual stimuli were presented within a tempo-

    ral stream of audiovisual distracters than when they were

    presented in isolation. Moreover, Parise and Spence (2009)

    found that the JND of an audiovisual TOJ was larger

    when the stimuli were presented as having crossmodal con-

    gruency (e.g., large visual stimuli and low-pitched sound)

    than when they were presented as an incongruent pair (e.g.,

    large visual stimuli and high-pitched sound) (see also Bien

    et al. 2012). These findings suggest that the spatial, tem-

    poral, or synesthetic proximity of stimuli have modulatory

    effects on direct temporal tasks.

    A recent study has demonstrated that the presentation

    of a hand image has modulatory effects on the tempo-

    ral aspects of a tactile judgment (i.e., changes in reaction

    time). Igarashi et al. (2004) presented a tactile stimulus

    (vibration) on the tip or base position of the index finger of

    participants hands and asked participants to judge the posi-

    tion of the vibration as soon as possible. Concurrently, a

    line drawing of a hand image was presented in front of the

    participants, and a visual distractor (circle) was presented

    at either the tip or the base position of the index finger of

    the hand image. The reaction time to the tactile position

    judgment increased or decreased when the position of the

    visual distracter in the hand image was congruent or incon-

    gruent with that of the tactile stimulus on the participants

    hand, respectively. Further, this spatial congruency effect

    was found to be stronger when the angle of the hand image

    was within the range in which a participants hand could

    be rotated, relative to when the angle of the hand was out

    of this range. The presentation of a hand image could also

    modulate visuotactile interactions in the temporal domain.

    Shimada et al. (2009) investigated the temporal window

    of the rubber hand illusion (Botvinick and Cohen 1998;

    Ehrsson et al. 2004). As in a typical rubber hand illusion

    technique, they presented a dummy hand in front of each

    participant, and the participants hand was hidden from

    view. The dummy and the participants hand were simul-

    taneously stroked using paintbrushes. The participants per-

    ceived that the stroke arose from the stroked position of

    the dummy hand rather than from that of the participants

    actual hand. Importantly, it was found that the rubber handillusion could reliably occur even when the SOA between

    the visual (stimulation of the dummy hand) and tactile

    (stimulation of the participants actual hand) stimuli was

    300 ms. For the occurrence of this illusion, it is important

    that the participants do not detect a temporal gap between

    the visual and tactile stimuli (Armel and Ramachandran

    2003; Ehrsson et al. 2004; Tsakiris and Haggard 2005).

    Thus, the presentation of a hand image could degrade the

    detectability of temporal asynchrony between a visual stim-

    ulus and a tactile stimulus far larger than that reported for

    simple stimuli (2080 ms).

    While these findings indicate that the visual presenta-tion of a hand image has modulatory effects on visuo

    tactile interactions in the temporal domain, these effects

    have not been thoroughly examined using direct temporal

    tasks. To fill this gap, the current study investigated whether

    the visual presentation of a hand image could modulate

    visuotactile TOJ. A visual stimulus was presented on the

    index finger of a hand image, and a tactile stimulus was

    presented on the index finger of a participants hand. We

    found that the JND of the visuotactile TOJ became larger

    when a forward hand image (whose angle corresponded to

    that of a participants hand) was presented than when the

    inverted hand or arrow images were presented (Experiment

    1). However, this effect was not observed for the audio

    visual TOJ (Experiment 2). Thus, the visual presentation

    of a hand image, whose angle was corresponded to that of

    ones own hand, could selectively degrade the visuotactile

    TOJ, indicating that visual hand images would implic-

    itly enhance the internal proximity between the visual and

    tactile stimuli and make them difficult to distinguish from

    each other in the temporal domain.

    Experiment 1

    The aim of Experiment 1 was to investigate the effects of the

    visual presentation of a hand image on visuotactile TOJ.

    In the experimental condition, we presented a forward hand

    image. We also introduced two control conditions in which

    an inverted hand and arrow images were presented. This was

    done to investigate whether the simple presentation of hand-

    like images or directional information of the forward hand

    image could also be effective. A situation without the pres-

    entation of images was also included as a Baseline condition.

  • 7/27/2019 Ide - Visual presentation of hand image modulates visuotactile temporal order judgment

    3/8

    45Exp Brain Res (2013) 228:4350

    1 3

    Methods

    Participants and apparatus

    Written consent was obtained from each participant before

    the experiments were conducted. The experiments were

    approved by the local ethics committee at Rikkyo Univer-

    sity. Ten volunteers and the two authors were involved inExperiment 1. All the participants had normal or corrected-

    to-normal vision.

    The experiment was controlled by the Dell Precision

    T5500 workstation and MATLAB (MathWorks) with the

    Psychophysics Toolbox (Brainard 1997; Pelli 1997). The

    visual stimuli were presented on a CRT display (EIZO,

    FlexScan T566, 17 inch) with a resolution of 1,024 768

    pixels and refresh rate of 100 Hz. The participants observed

    the display with their heads placed on a chinrest. The tactile

    stimuli were presented through an audio interface (Roland,

    EDIROL FA-66) and an attachable speaker (Eishindenki,

    Attachable Speaker M-PZT-02) for generating vibration.An acrylic box filled with sponge was attached to the

    speaker in order to present the stimuli solely on the tip of

    a participants index finger. An opaque shield occluded the

    participants left hand and the vibrator. We confirmed that

    the onset of the visual and tactile stimuli was synchronized

    by using a digital oscilloscope (OWON, PDS5022TFT).

    The experiment was conducted in a dark room.

    Stimulus

    A red circle (0.6; 27.85 cd/m2) was presented as a fixa-

    tion point on a white background (110.4 cd/m2; Fig. 1a).

    A gray circle (1 in diameter; 72.43 cd/m2) was pre-

    sented at an eccentricity of 5 along the horizontal plane

    for 10 ms. Three images (forward hand, inverted hand,

    and arrow; Fig. 1b) were presented as black line draw-

    ings (12.7 8.5). A vibration stimulus (400-Hz sinusoi-

    dal burst) was presented on the tip of each participants left

    index finger on the palm side for 10 ms with 1 ms of cosine

    ramp at the onset and offset. The gray circle and the vibra-

    tion were presented concurrently and transiently, while the

    fixation point and the images were constantly presented

    during a trial. These parameters, which were in accordance

    with our preliminary experiment, were selected to control

    for any possible visual dominance effects; visual stimuli

    tended to be judged as temporally preceding stimuli pre-

    sented in a different modality (Rutschmann and Link1964;

    Smith 1933). Therefore, the intensities of a multisensory

    stimulus need to be modified appropriately in order for

    them to be judged as simultaneous when they are presented

    concurrently (Jaskowski et al. 1990). We thus modified the

    parameters such that the intensity of the tactile stimulus

    was equivalent to that of the visual stimulus.

    Procedure

    We asked each participant to place his/her own left hand

    with the index finger up on the tactile device and then to

    look at the fixation circle. There were four image condi-

    tions (Fig. 1b): In the Forward-hand condition, the left-

    hand image was presented and its angle was kept consistent

    with that of the participants own left hand. In contrast, in

    theInverse-hand condition, the hand image was rotated by

    180 such that the angle of the hand image did not corre-

    spond to that of the participants own hand. In the Arrow

    condition, an arrow image containing similar directional

    information as the Forward-hand condition was presented.

    These conditions were tested in a separate block (within-

    participants block design), and each image was constantly

    presented during each block. We also introduced a situation

    without the presentation of images, and this was called the

    Baseline condition. The position of the visual and tactile

    stimuli was fixed in all image conditions. After an interval

    Fig. 1 A schematic illustration of the experimental conditions in

    Experiment 1. a A gray circle was consistently presented at an eccen-

    tricity of 5 from the fixation point. A vibration was presented on

    the tip of each participants left index finger, with an opaque shield

    occluding their hand. Both the gray circle and the vibration were pre-

    sented for 10 ms each with various SOAs. b The image conditions.

    A left-hand image having an angle consistent with that of each par-

    ticipants own left hand (forward-hand condition; a); a hand image

    rotated by 180 as compared to the position of the participants own

    hand (Inverse-hand condition; b); an arrow image pointing to the

    same direction as in the Forward-hand condition (Arrow condition;

    c); no presentation of images (Baseline condition; d) (color figure

    online)

  • 7/27/2019 Ide - Visual presentation of hand image modulates visuotactile temporal order judgment

    4/8

    46 Exp Brain Res (2013) 228:4350

    1 3

    randomly assigned in each trial (2,000, 2,080, or 2,100 ms),

    the visual stimuli were presented with one of the follow-

    ing stimulus onset asynchronies (SOAs):40, 80, 160,

    and 320 ms. Negative values indicate a situation in which

    the visual stimulus was presented first, and positive values

    indicate a situation in which the tactile stimulus was pre-

    sented first. The participants were asked to judge which

    stimulus was presented first (visual or tactile) by pressing acorresponding key.

    The experiment consisted of two sessions. In the prac-

    tice session, the participants completed 24 trials [SOAs

    (8) Repetitions (3)] without images (Baseline condition).

    The main session consisted of 640 trials: Image conditions

    (4) SOAs (8) Repetitions (20). The order of the condi-

    tions (blocks) was randomized and counterbalanced among

    the participants. SOAs were also randomly introduced

    in each trial and counterbalanced among the blocks. The

    experiment lasted approximately 45 min.

    Results and discussion

    We plotted averaged proportions of tactile first

    responses as a function of SOAs in each image condi-

    tion (Fig. 2a) to obtain psychometric functions. Then,

    we estimated the 75 and 25 % threshold points by fit-

    ting a cumulative Gaussian distribution function onto

    each participants data by using the maximum likeli-

    hood method and calculated the slope of the functions

    as JND by using the following formula: (75 % thresh-

    old25 % threshold)/2. In order to compare JNDs among

    the image conditions, we calculated normalized JNDs by

    subtracting the JND of each image condition from that

    of the Baseline condition (Fig. 2b). A one-way repeated-

    measures analysis of variance (ANOVA) indicated a

    significant main effect among these image conditions

    [F(2, 22) = 4.93, p < .05]. A post hoc test (Tukeys

    HSD, p < .05) revealed that the JND of the Forward-

    hand condition was larger than that of the Inverse-hand

    and Arrow conditions.

    Since the simple presentation of the hand-like image

    (the inverted hand image) and the directional information

    of the forward hand image (the arrow image) did not seem

    to have any effect (Fig. 2b), we believe that the visual pres-

    entation of the hand image, whose angle was corresponded

    to that of ones own hand (the forward hand image), selec-

    tively degraded the TOJ between the visual stimulus and

    the tactile stimulus. Previous studies reported that the inter-

    actions between visual and tactile stimuli occurred espe-

    cially when the angle of the visual hand image was within

    the range in which the participants hands could be rotated

    (Ide 2013; Igarashi et al. 2004). In line with these findings,

    the forward hand image was found to dominantly have a

    modulatory effect.

    Experiment 2

    The TOJ between the visual stimulus and the tactile stimulus

    may have degraded in Experiment 1 merely because the vis-

    ual presentation of the forward hand image simply attracted

    visual attention. To check this possibility, we investigated

    whether the visual presentation of the forward hand image

    could also modulate the TOJ between an auditory stimulus

    and a visual stimulus. We hypothesized that the modula-

    tory effect of the forward hand image found in Experiment 1

    could also be observed on the audiovisual TOJ if the atten-

    tional distraction/capture effect acted as a decisive factor.

    Fig. 2 The results of Experiment 1. a Averaged proportions of tac-

    tile first responses as a function of SOAs for the four image con-

    ditions. The 75 and 25 % threshold points were estimated, and the

    slope of the functions was calculated as JND by using the following

    formula: (75 % threshold25 % threshold)/2. b Normalized JNDs of

    the image conditions versus the Baseline condition. The error bars

    denote the standard error of the mean (N= 12). An asteriskindicates

    the condition that was significantly different from the others (p < .05)

  • 7/27/2019 Ide - Visual presentation of hand image modulates visuotactile temporal order judgment

    5/8

    47Exp Brain Res (2013) 228:4350

    1 3

    Methods

    Ten volunteers and the two authors were involved in

    Experiment 2. All participants had normal or corrected-

    to-normal vision. Seven volunteers were newly recruited

    for this experiment. In this experiment, a white noise burst

    (10 ms) was monaurally presented through an audio inter-

    face and a headphone (Sennheiser, HDA 200; Fig. 3). Thisauditory stimulus was presented at a 70-dB sound pressure

    level with 1 ms of cosine ramp at the onset and offset. The

    vibration was not presented in this experiment, but the par-

    ticipants placed their left hands with the index finger up on

    the acrylic box, as in Experiment 1. The participants were

    asked to judge which stimulus (auditory or visual) was pre-

    sented first. Except for the stated differences, the appara-

    tus, stimuli, and procedures of Experiments 1 and 2 were

    identical.

    Results and discussion

    We plotted averaged proportions of auditory first

    responses as a function of SOAs in each image condition

    (Fig. 4a) to obtain psychometric functions. As in Experi-

    ment 1, we then calculated normalized JNDs (Fig. 4b).

    A one-way repeated-measures ANOVA did not indicate a

    significant main effect [F(2, 22) = 1.67,p= .21].

    The results showed that JNDs were not significantly dif-

    ferent among the image conditions, thereby indicating that

    the visual presentation of the forward hand image did not

    modulate the TOJ between the auditory stimulus and the

    visual stimulus. Therefore, we could assume that the atten-

    tional distraction/capture effect induced by the forward

    hand image would not be a decisive factor for the results

    obtained in Experiment 1, and that the forward hand image

    could be effective only with regard to the TOJ between the

    visual stimulus and the tactile stimulus.

    One might assume that, in Experiment 2, the JND of

    the Forward-hand and the Inverse-hand conditions would

    appear to be larger than that of the Arrow condition such

    that these differences would be similar to the difference

    between the Forward-hand and the Arrow conditions in

    Experiment 1. In order to confirm these possibilities, we

    calculated the statistical power (1 ) and the effect sizes

    Fig. 3 A schematic illustration of the experimental conditions in

    Experiment 2. A white noise burst was presented via a headphone.

    Except this, all other experimental conditions were consistent with

    those in Experiment 1 (see Fig. 1)

    Fig. 4 Results of Experiment 2. a Averaged proportions of auditory

    first responses as a function of SOAs for the four image conditions.

    JNDs were calculated using the same formula as that used for Experi-

    ment 1. b Normalized JNDs of the image conditions. Error bars

    denote the standard error of the mean (N= 12)

  • 7/27/2019 Ide - Visual presentation of hand image modulates visuotactile temporal order judgment

    6/8

    48 Exp Brain Res (2013) 228:4350

    1 3

    of the results of Experiments 1 and 2 and compared them

    by referring to the criteria suggested by Cohen (1988)

    and Ellis (2010). With regard to the results of Experi-

    ment 1, the one-way repeated-measures ANOVA con-

    ducted had high statistical power (0.99). The effect size

    was not small (2p= 0.31). The JND of the Forward-hand

    condition had medium (dD= 0.58) and large (dD= 1.07)

    effect sizes compared to the Inverse-hand and the Arrowconditions, respectively. In contrast, the effect size was

    negligible (dD = 0.17) between the Inverse-hand and

    the Arrow conditions. As for Experiment 2, the one-way

    repeated-measures ANOVA conducted also had sufficient

    statistical power (0.72). However, the effect size was neg-

    ligible (2p= 0.13). We also found that the effect sizes

    between the Forward-hand and the Arrow conditions and

    the Inverse-hand and the Arrow conditions were below

    medium level (dD= 0.39 and dD= 0.49, respectively).

    The effect size between the Forward-hand and the Inverse-

    hand conditions was negligible (dD= 0.05). These data

    confirmed that the statistical power of the ANOVAs wassufficient in both experiments, indicating that our results,

    based on probability statistics, would be reliable. Consist-

    ent with the probability statistics results, we also found

    that while the effect sizes of Experiment 1 were above the

    small level for the ANOVA and above the medium or large

    level between the Forward-hand and the other conditions,

    those of Experiment 2 were below the small level for the

    ANOVA and below the medium level between the For-

    ward-hand and the Arrow conditions and the Inverse-hand

    and the Arrow conditions. We could thus conclude that the

    visual presentation of the forward hand image could be

    effective, especially for visuotactile TOJ.

    General discussion

    A recent study suggested that the presentation of a hand

    image has modulatory effects on visuotactile interactions

    in the temporal domain (Shimada et al. 2009). However,

    that study did not employ direct temporal tasks. The cur-

    rent study investigated the effect of visual presentation of

    a hand image in a direct temporal task to assess the TOJ

    between the visual stimulus and the tactile stimulus. The

    results showed that the JNDs of the visuotactile TOJ were

    larger when the forward hand image was presented than

    when the inverted hand image or the arrow image was pre-

    sented (Experiment 1). In contrast, the JNDs of the TOJ

    between the auditory stimulus and the visual stimulus did

    not differ among these image conditions (Experiment 2).

    These findings suggest that the visual presentation of the

    hand image corresponding to the angle of ones own hand

    (the forward hand image) could selectively degrade the

    visuotactile TOJ.

    In Experiment 1, we confirmed that the simple presen-

    tation of a hand-like image (the inverted hand image) and

    the directional information of a forward hand image (the

    arrow image) did not affect the visuotactile TOJ. One

    might assume that the visual presentation of a hand image

    similar to ones own hand (the forward hand image) selec-

    tively attracted visual attention to the display, and thus, the

    TOJ between the visual stimulus and the tactile stimuluswas degraded. If this were the case, the visual presentation

    of the forward hand image would have also degraded the

    audiovisual TOJ. However, the results of Experiment 2

    did not indicate any such effect. Therefore, we concluded

    that the effect of visual presentation of the forward hand

    image observed in Experiment 1 might not be explained

    solely by attentional distraction/capture effects.

    A notable point is that only the forward hand image,

    whose angle corresponded to that of the participants own

    hand, could have a modulatory effect on the visuotactile

    TOJ. Similarly, Igarashi et al. (2004) reported that the spa-

    tial congruency effect between the visual distractors pre-sented on a hand image and the tactile stimuli presented

    on the participants own hands became stronger when the

    angle of the hand image was within the range in which

    the participants hands could be rotated. Ide (2013) also

    showed that the rubber hand illusion dominantly occurred

    when a dummy hand image rotated at 0, 45, 90, and

    315 in a clockwise direction. These angles were firmly

    corresponding to the range of motion of the participants

    own hands. Our finding that the forward hand image domi-

    nantly had a modulatory effect on the visuotactile TOJ is

    consistent with these previous findings.

    Our results could appear to be explained by the percep-

    tual degradation effect of a visual hand image on tactile

    stimuli. While the visual hand or body image enhanced

    discrimination/detection performance for supra-threshold

    tactile stimuli (Kennett et al. 2001; Taylor-Clarke et al.

    2004; Tipper et al. 1998), the image also showed an inter-

    ference effect on performance when a tactile stimulus was

    presented near the threshold (Harris et al. 2007; Mirams

    et al. 2010). It could thus be assumed that the forward hand

    image selectively degraded the perceived intensity of the

    tactile stimulus such that the TOJ between a visual and a

    tactile stimulus was impaired. However, in the current

    study, the tactile stimulus was presented at supra-threshold

    and was clearly perceived; we modified the intensity of

    the tactile stimulus such that it was as clearly and equally

    strong as the visual stimulus, in order to exclude any pos-

    sible visual dominance effects on the results (see also the

    Methods section in Experiment 1). We further con-

    firmed that there was no biasing effect; if the perceived

    intensity of the tactile stimulus was degraded and partici-

    pants attention or preference was directed to the visual

    stimulus, then a consistent visual first response would

  • 7/27/2019 Ide - Visual presentation of hand image modulates visuotactile temporal order judgment

    7/8

    49Exp Brain Res (2013) 228:4350

    1 3

    be obtained. Consequently, the point of subjective simul-

    taneity (PSS) of the psychometric functions would shift

    (Spence and Parise 2010). However, we did not find any

    differences in PSS between the conditions (Supplementary

    Fig. 1). Although it was reported that the visual hand image

    increased somatosensory intracortical inhibition, especially

    when two adjacent fingers were stimulated simultaneously

    (Cardini et al. 2011), the tactile stimulus was delivered toa single finger in our experiment. On the basis of these

    results, we conclude that the degradation effect of the vis-

    ual hand image on tactile stimuli cannot fully explain the

    current findings.

    It has been demonstrated that the internal proximity

    of stimuli has modulatory effects on multisensory TOJ

    (Bertelson and Aschersleben 2003; Keetels and Vroomen

    2005; Spence et al. 2003; Zampini et al. 2005). In par-

    ticular, TOJ was degraded when an auditory stimulus and

    a visual stimulus or a visual stimulus and a tactile stimu-

    lus were presented at a spatially close position, relative

    to when they were spatially apart (Spence et al. 2003).Vatakis et al. (2007) also reported that an audiovisual

    TOJ became less precise when the stimuli were presented

    within a temporal stream of audiovisual distracters than

    when they were presented in isolation. In addition, Par-

    ise and Spence (2009) found that audiovisual TOJ was

    degraded for the stimuli with crossmodal synesthetic

    congruency rather than for those without such congru-

    ency. These findings indicate that multisensory TOJ was

    degraded when the internal proximity of stimuli was

    maintained or enhanced. Interestingly, in the situation

    where the stimuli were spatially apart, unimodal visual

    TOJ still worsened when the stimuli were perceptually

    grouped (Nicol and Shore 2007). On the basis of these

    findings, we assume that in the current study, the for-

    ward hand image would enhance the internal proximity

    between the visual stimulus presented on the hand image

    and the tactile stimulus presented on the participants own

    hand (note that these stimuli were consistently presented

    on the index finger of the hand image and the actual

    hand.) This would make the stimuli perceptually grouped

    and/or difficult to distinguish from each other in the tem-

    poral domain, thus impairing the visuotactile TOJ. This

    idea would also be supported by the current results show-

    ing that the impairment of visuotactile TOJ was observed

    for only the forward hand image and that the effect selec-

    tively occurred for visuotactile stimuli (not for audio

    visual stimuli).

    Some neurophysiological studies have provided evi-

    dence supporting the present findings (Duhamel et al.

    1998; Graziano 1999; Iriki et al. 1996). For example, Iriki

    et al. (1996) showed that monkeys neurons in the intra-

    parietal sulcus (IPS) were activated by not only the tac-

    tile stimulus but also the visual stimulus that appeared to

    move so as to touch their hands. Furthermore, after mon-

    keys were trained to move their hands while viewing their

    hands on a display, the neurons in the IPS were activated

    solely by the visual hand image on the display (Iriki et al.

    2001). Similarly, in a functional magnetic resonance imag-

    ing (fMRI) study, the activation of the human IPS was

    modulated when the dummy hand image was presented

    at the actual hand position, as well as on the presenta-tion of actual hand image (Makin et al. 2007). Graziano

    et al. (2000) also reported that monkeys neurons in area 5

    (superior parietal lobe), which were generally activated by

    visual and somatosensory information of their own hands,

    were also strongly activated when monkeys saw a dummy

    hand placed just above their own hands (which were out of

    view). These findings indicate that visual stimulation to the

    hand image and tactile stimulation to ones own hand could

    be represented as similar inputs in the brain. Using some

    psychophysical and neuropsychological techniques, future

    research should investigate in detail whether the internal

    proximity between the visual and tactile stimuli and thepossibly related IPS activations could actually be enhanced

    by the presentation of the forward hand image.

    The current study showed that the visual presentation

    of a hand image impaired TOJ between the visual stimulus

    presented on the hand image and the tactile stimulus pre-

    sented on the participants own hands only when the hand

    image whose angle was corresponded to that of ones own

    hand was presented. Since this effect was not observed

    for the audiovisual TOJ, attentional distraction/attraction

    to the forward hand image could not be a decisive factor

    in this result. In addition, since the maintenance/enhance-

    ment of the internal proximity of multimodal stimuli could

    degrade the TOJ of those stimuli, and the visual hand

    images and visual stimulation to the hand images could

    be represented as related to or similar to the touch-related

    stimuli in the brain, our results suggest that the visual

    presentation of the hand image corresponding to that of

    ones own hand implicitly enhances the internal proximity

    between visual and tactile stimuli and makes them difficult

    to distinguish from each other in the temporal domain.

    Acknowledgments We thank Miho Kondo for her support in con-

    ducting the experiments and Shoko Yabukis help in preparing the fig-

    ures included in this paper. We are also grateful to the two anonymousreviewers for their valuable and insightful comments. This study was

    supported by a Grant-in-Aid from Rikkyo University Special Fund for

    Research (No. 18120030) and a Grant-in-Aid for Specially Promoted

    Research from JSPS (No. 19001004).

    References

    Armel KC, Ramachandran VS (2003) Projecting sensations to exter-

    nal objects: evidence from skin conductance response. Proc R

    Soc Land Biol Sci 270:14991506

  • 7/27/2019 Ide - Visual presentation of hand image modulates visuotactile temporal order judgment

    8/8

    50 Exp Brain Res (2013) 228:4350

    1 3

    Bertelson P, Aschersleben G (2003) Temporal ventriloquism: cross-

    modal interaction on the time dimension. 1. Evidence from

    auditory-visual temporal order judgment. Int J Psychophysiol

    50:147155

    Bien N, ten Oever S, Goebel R, Sack AT (2012) The sound of size:

    crossmodal binding in pitch-size synesthesia: a combined TMS,

    EEG and psychophysics study. Neuroimage 59:663672

    Botvinick M, Cohen J (1998) Rubber hands feel touch that eyes see.

    Nat 391:756

    Brainard DH (1997) The Psychophysics Toolbox. Spat Vis

    10:433436

    Calvert GA, Spence C, Stein BE (eds) (2004) The hand book of mul-

    tisensory processes. MIT Press, Cambridge

    Cardini F, Longo MR, Haggard P (2011) Vision of the body modulates

    somatosensory intracortical inhibition. Cereb Cor 21:20142022

    Cohen J (1988) Statistical power analysis for the behavioral sciences,

    2nd edn. Lawrence Erlbaum, New Jersey

    Driver J, Spence C (2000) Multisensory perception: beyond modular-

    ity and convergence. Curr Biol 10:731735

    Duhamel J, Colby CI, Goldberg ME (1998) Ventral intraparietal area

    of the macaque: congruent visual and somatic response proper-

    ties. J Neurophysiol 79:126136

    Ehrsson HH, Spence C, Pashingham RE (2004) Thats my hand!

    Activity in premotor cortex reflects feeling of ownership of a

    limb. Science 305:875877

    Ellis PD (2010) The essential guide to effect sizes: statistical power,

    meta-analysis, and the interpretation of research results. Cam-

    bridge University Press, New York

    Fujisaki W, Nishida S (2009) Audio-tactile superiority over visuotac-

    tile and audiovisual combinations in the temporal resolution of

    synchrony perception. Exp Brain Res 198:245259

    Graziano MSA (1999) Where is my arm? The relative role of vision

    and proprioception in the neuronal representation of limb posi-

    tion. PNAS 96:1041810421

    Graziano MSA, Cooke DF, Taylor CSR (2000) Coding the location of

    the arm by sight. Science 290:7821786

    Harrar V, Harris LR (2008) The effect of exposure to asynchronous

    audio, visual, and tactile stimulus combinations on the perception

    of simultaneity. Exp Brain Res 186:517524Harris JA, Arabzadeh E, Moore CA, Clifford CWG (2007) Nonin-

    formative vision causes adaptive changes in tactile sensitivity.

    J Neurosci 27:71367140

    Hirsh IJ, Sherrick CE Jr (1961) Perceived order in different sense

    modalities. J Exp Psychol 62:423432

    Ide M (2013) The effect of anatomical plausibility of hand angle on

    the rubber hand illusion. Percept 42:103111

    Igarashi Y, Kitagawa N, Ichihara S (2004) Vision of pictorial hand

    modulates visual-tactile interactions. Cogn Affect Behav Neuro-

    sci 4:182192

    Iriki A, Tanaka M, Iwamura Y (1996) Coding of modified body

    schema during tool use by macaque postcentral neurons. Neu-

    roReport 7:23252330

    Iriki A, Tanaka M, Obayashi S, Iwamura Y (2001) Self-images in the

    video monitor coded by monkey intraparietal neurons. NeurosciRes 40:163173

    Jaskowski P, Jaroszyk F, Hojan-Jezierska D (1990) Temporal-order

    judgments and reaction time for stimuli of different modalities.

    Psychol Res 52:3538

    Keetels M, Vroomen J (2005) The role of spatial disparity and hemi-

    fields in audio-visual temporal order judgments. Exp Brain Res

    167:635640

    Kennett S, Taylor-Clarke M, Haggard P (2001) Noninformative vision

    improves the spatial resolution of touch in humans. Curr Biol

    11:11881191

    Makin TR, Holmes NP, Zohary E (2007) Is that near my hand? Multi-

    sensory representation of peripersonal space in human intrapari-

    etal sulcus. J Neurosci 27:731740

    Mirams L, Poliakoff E, Brown J, Lloyd DM (2010) Vision of body

    increase interference on the somatic signal detection task. Exp

    Brain Res 202:787794

    Nicol JR, Shore DL (2007) Perceptual grouping impairs temporal res-

    olution. Exp Brain Res 183:141148

    Parise CV, Spence C (2009) When birds of a feather flock together:

    synesthetic correspondences modulate audiovisual integration in

    non-synesthetes. PLoS ONE 4:e5664

    Pelli DG (1997) The Video Toolbox software for visual psychophys-

    ics: transforming numbers into movies. Spat Vis 10:437442

    Rutschmann J, Link R (1964) Perception of temporal order of stimuli

    differing in sense mode and simple reaction time. Percept Mot

    Skills 18:345352

    Shimada S, Fukuda K, Hiraki K (2009) Rubber hand illusion under

    delayed visual feedback. PLoS ONE 4:e6185

    Shore DL, Spence C (2005) Prior entry. In: Itti L, Rees G, Tsotsos JK

    (eds) Neurobiology of attention. Elsevierm, Academic Press, San

    Diego, pp 8995

    Smith WF (1933) The relative quickness of visual and auditory per-

    ception. J Exp Psy 16:239257

    Spence C, Parise C (2010) Prior entry: a review. Conscious Cogn

    19:364379

    Spence C, Shore DI, Klein RM (2001) Multisensory prior entry. J Exp

    Psychol Gen 130:799832

    Spence C, Baddeley R, Zampini M, James R, Shore DI (2003) Multi-

    sensory temporal order judgments: when two locations are better

    than one. Percept Psychophys 65:318328Stein BE, Meredith MA (1993) The merging of the senses. MIT

    Press, Cambridge

    Taylor-Clarke M, Kennett S, Haggard P (2004) Persistence of visual-

    tactile enhancement in humans. Neurosci Lett 354:2225

    Tipper SP, Lloyd D, Shorland B, Dancer C, Howard LA, McGlone F

    (1998) Vision influences tactile perception without proprioceptive

    orienting. Neuro Rep 9:17411744

    Tsakiris M, Haggard P (2005) The rubber hand illusion revised: visu-

    otactile integration and self-attribution. J Exp Psychol Hum Per-

    cept Perform 31:8091

    Vatakis A, Bayliss L, Zampini M, Spence C (2007) The influence

    of synchronous audiovisual distractors on audiovisual temporal

    order judgments. Percept Psychophys 69:298309

    Zampini M, Brown T, Shore DI, Maravita A, Roder B, Spence C

    (2005) Audiotactile temporal order judgments. Acta Psychol118:277291