IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 44, NO. 4, AUGUST 2014 545

Biosignals Sensing by Novel Use of BidirectionalMicrophones in a Mobile Phone for Ubiquitous

Healthcare Monitoring

Kajiro Watanabe, Member, IEEE, Yosuke Kurihara, Ko Watanabe,Toshihiro Azami, Shoko Nukaya, and Hiroshi Tanaka

Abstract—This paper describes a novel application of the bidirectionalaudio microphones in mobile phones to measure biosignals for ubiqui-tous health monitoring. The bidirectional microphone has two pressure-detecting ports. A pressure-detecting film receives the differential pressurebetween the ports. We apply this idea to biosignal measurements for a mo-bile phone. We used the microphone as 1) a sphygmograph at the fingertip,2) a stethoscope in which a delay time sensor of pulse wave arrivals fromthe neck near the carotid or chest near the heart to the fingertip, and 3) abed sensing device for biosignals. When used as a sphygmograph, it detectsthe wave similar to the second derivative of the conventional optical pulseoxymeter output wave. When used as a stethoscope, it detects cardiac soundand the pulse wave and respiration. With two microphones, the delay timefrom the neck carotid to the fingertip is detected.

Index Terms—Bidirectional microphone, low-frequency microphone,mobile phone, noninvasive bed sensing, sphygmograph, stethoscope.

I. INTRODUCTION

The advanced communication and sensor technology in the mobilephone could be used for daily health monitoring, management, andincipient diagnosis. Actigraphy, a small watch-type biosensor, can beeasily connected to a mobile phone and/or network to monitor not onlydaily activity but also sleep abnormalities [1]–[3]. Other emerging de-vices include a bed sensing device to monitor biosignals [4]–[7], toestimate sleep stages [8], and to monitor health at home [9]. Watanabeet al. [10] have designed a low-frequency microphone and presenteda bed sensing method to monitor the pulse wave, respiration, snoring,and body movement by a mobile phone that can replace a conventionalaudio microphone [11]. It is an alternative implementation of the pneu-matic method [4], which uses an air cushion. In this paper, we describea device that uses inexpensive conventional bidirectional audio micro-phones. The device detects the pulse wave at the fingertip in the sameway that a plethysmogram detects cardiac sound, and that a stethoscopedetects the pulse wave and respiration, determines the delay times of

Manuscript received August 31, 2013; revised March 18, 2014; acceptedApril 23, 2014. Date of publication June 26, 2014; date of current version July11, 2014. This paper was recommended by Associate Editor E. P. Blasch.

K. Watanabe is with the Department of Advanced Sciences, Faculty ofScience and Engineering, Hosei University, Tokyo 184-8584, Japan (e-mail:bob@hosei.ac.jp).

Y. Kurihara is with the Department of Industrial and Systems Engineering,College of Science and Engineering, Aoyama Gakuin University, Kanagawa252-5258, Japan (e-mail: kurihara@ise.aoyama.ac.jp).

K. Watanabe is with the Division of Advanced Therapeutical Sciences,Tokyo Medical and Dental University, Tokyo 113-8510, Japan (e-mail:ko-wata@sic.shibaura-it.ac.jp).

T. Azami is with Fujitsu Limited, Kawasaki 211-8588, Japan (e-mail:azami@jp.fujitsu.com).

S. Nukaya is with the Division of Advanced Therapeutical Sciences,Tokyo Medical and Dental University, Tokyo 113-8510, Japan (e-mail:nukaya@bioinfo.tmd.ac.jp).

H. Tanaka is with the University Center for Information Medicine,Tokyo Medical and Dental University, Tokyo 113-8510, Japan (e-mail:tanaka@cim.tmd.ac.jp).

Digital Object Identifier 10.1109/THMS.2014.2320945

Fig. 1. Directional microphone whose two ports are in isolated spaces. (a)Structure of the bi-directional microphone and how directivity occurs. (b) Twoports placed in two isolated spaces. (c) Frequency response of a bidirectionalmicrophone with separated front and back ports with leak.

pulse waves from body positions such as the neck carotid area to thefingertip, and could be used as a bed sensing device. Here, we focus onsensing by a microphone.

II. METHOD

A. Use of Bidirectional Microphone as a Low-Frequency High-GainOmnidirectional Microphone

Fig. 1(a)shows the structure of a bidirectional microphone. It hastwo pressure-detecting ports. One is located on the front side and theother is located on the back side. The acoustic pressure that arrivesat the pressure-detecting electret film through both ports acts as a

2168-2291 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

546 IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 44, NO. 4, AUGUST 2014

Fig. 2. Mounting of microphones on the mobile phone housing.

Fig. 3. Mobile phone housing and the frequency response of the low-frequencymicrophone. (a) Mobile phone housing with two bi-directional microphones.(b) Frequency response of the bidirectional microphone and that set, as shownin Figs. 1(b) or 2.

differential. If the frequency of the pressure change is low and/or thewavelength is long and both ports are in a space at the same pressure,the electret film hardly moves, which leads to low-cut and/or high-pass filtering characteristics. If the high-frequency acoustic pressurepropagates to the microphone from the side direction, the pressurearrives at the electret film from both sides simultaneously and pushesthe film from the front and back sides, and thus, the film hardly moves.However, if the acoustic pressure propagates from the front- or backside, it arrives at the film with some delay time, and therefore, the filmcan more easily move than in the case when the pressure is from theside direction. Thus, the pressure from the front or back direction ismore sensitively detected than that from the side direction, and thisfact yields the characteristic directivity of the microphone. Fig. 1(b)shows a different microphone setting. The front port is in space 1, andthe back port is in space 2 isolated from space 1. In this setting, thebidirectional microphone detects the changes in pressures in the twospaces independently. The change in pressure in the front-side spacedoes not influence the back-side space even if the changing frequency is

Fig. 4. Sphygmogram measured by microphone and pulse oxymeter. (a) Hold-ing the mobile phone housing and covering microphone #1 by thumb fingertipto detect the plethysmogram. (b) Pressure pulse by microphone. (c) Secondderivative of the sphygmogram by pulse oxymeter for reference. (d) Spectra ofthe microphone. (e) Spectra of the pulse oxymeter.

low. Thus, the bidirectional microphone, as shown in Fig. 1(b), makesthe audio microphone a low-frequency omnidirectional microphonewith a high gain for the changes in pressures in the two separatedspaces. We apply this novel idea to mobile phone microphones. Thefront port of a microphone set housing is covered by a flexible film witha narrow closed space, listed as “space 1.” The back port is in “space2,” which is inside the mobile phone housing.

Note that the space inside the housing is directly connected withthe outer space in the low-frequency range. Here, we consider a math-ematical model of the microphone based on the pneumatics. Even ifthe front port is closed, there unavoidably exists a small orifice andair flows through the orifice if there is a pressure difference betweenspaces 1 and 2.

The following constants and variables are used to build the mathe-matical model:

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 44, NO. 4, AUGUST 2014 547

Fig. 5. Cardiac sounds and the spectra measured by the proposed microphoneand the conventional electrical stethoscope. (a) Touching an electrical stetho-scope to bare chest with both conventional and low frequency microphones.(b) Proposed microphone. (c) Conventional electrical stethoscope. (d) Proposedmicrophone. (e) Conventional electrical stethoscope.

V volume of semiclosed space 1;R gas constant;T absolute temperature around the microphone;r flow resistance determined by Hagen–Poiseuille’s Law, when air

passes through the orifice;fc cutoff frequency of microphone;t time;f frequency;s differential operator in Laplace transform;p1 (t) pressure in space 1;p2 (t) pressure in space 2;Δp(t) difference of pressures in spaces 1 and 2;q(t) air flow in number of moles passing through the orifice.

Fig. 6. Pulses at the fingertip and chest on the heart with cloth and the pulsetransmit time. (a) Holding the mobile phone housing and covering microphone#1 by a thumb fingertip and covering #2 by a chest over a closes, pulse wave atfingertip and cardiac sound including pulse wave as well as respiration motionby the low-frequency stethoscope were measured. (b) Pulse transit time fromthe chest to the fingertip.

From the ideal gas law and Hagen–Poiseuille’s Law, the pressurein space 1 and air flow through the orifice and pressure difference aredescribed as follows:

p1 (t) =RT

V

∫q(t)dt

q(t) =p2 (t) − p1 (t)

r

Δp(t) = p2 (t) − p1 (t).

The transfer function of the difference pressure ΔP (s) detectedby the bidirectional microphone with respect to the pressure P1 (s) isobtained as follows by Laplace transforming and arranging the afore-mentioned three equations:

ΔP (s) =r VRT

s

1 + r VRT

sP1 (s)

which shows the high-pass filtering characteristic with the cutoff fre-quency fc = (RT )/(2πrV). From the transfer function above, the fre-quency response of Δp(t) with respect to p1 (t) is shown inFig. 1(c).

As the cross-sectional area of the orifice becomes larger, the flowresistance becomes lower and the cutoff frequency becomes higher. Inthe ultimate state, when r = 0, spaces 1 and 2 are in the same space

548 IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 44, NO. 4, AUGUST 2014

Fig. 7. Pulses at the fingertip and neck carotid and the pulse transmit time.(a) Holding the mobile phone housing and covering microphone #1 by a thumbfingertip and covering microphone #2 by the neck near the carotid to detect thetwo pulse waves at the fingertip and carotid. (b) Pulse transit time from the neckcarotid to the fingertip.

and the low-frequency changes in the pressure in space 1 cannot bedetected. Conversely, as the cross-sectional area of the orifice becomessmaller, the flow resistance becomes higher and the cutoff frequencybecomes lower. In the ultimate state when r = ∞, spaces 1 and 2 arecompletely separated, and the pressure in space 1 can be detected fromthe dc component. The cutoff frequency is controlled both by the flowresistance and the volume of space 1. The microphone is composed ofpneumatic elements of air flow through the orifice, dynamic elements ofmass, stiffness and damper of the electret film, and electronic elementsof the FET and circuit, as described in the literature [10]. Among these,the pneumatic elements of the air flow determine the low-frequencycharacteristics, while those of the electret film determine the high-frequency characteristics by mechanical resonance. Thus, separationof the front and back ports with an unavoidable orifice or leak changesonly the characteristics in the low-frequency range without influencingthe high audio frequency range.

B. Mounting of Microphones on Model Mobile Phone Housing andBiosensing

Fig. 2 shows an example of the bidirectional microphones mountedon the model mobile phone housing. Here, two microphones are set.Microphone #1 is mounted where a finger (thumb) touches and holdsthe housing. Microphone #2 is mounted at the normal microphone

position. For both microphones, the front port of the bidirectional mi-crophone is in the closed space covered by a narrow chamber witha flexible front cover or by the human body, such as a bare finger,neck, or chest. The microphones mounted, as shown in Fig. 2, showthe low-frequency characteristics; we refer to these microphones as thelow-frequency microphones.

Mobile phones with the two low-frequency microphones detect thefollowing various biosignals: 1) plethysmogram of the fingertip; 2)cardiac sound including the pulse wave and respiration wave; 3) pulsedelay time from the chest heart area to the fingertip; 4) pulse delaytime from the neck carotid area to the fingertip; and 5) the pulse wave,respiration, snoring, and body movement of a person on bed.

III. VERIFICATION EXPERIMENT

A. Measurement Device and System

The microphone used was a bidirectional microphone EM114(Primo Co., Ltd., Tokyo) with a diameter of 10 mm and length of 5 mm.Fig. 3(a) shows a model mobile phone housing with size of 150 mm× 65 mm × 23 mm in which two bidirectional microphones weremounted, as shown in Fig. 2. The shape of space 1 in the front side ofthe microphone is 10 mm in diameter by 3 mm in depth. Fig. 3(b) showsthe frequency responses of the bidirectional microphone alone and thatof the low-frequency microphone mounted, as shown in Fig. 3(a).

The bidirectional microphone detects the pressure in the fre-quency range of 10 Hz–10 kHz with high-pass filtering char-acteristics with the cutoff frequency around 200 Hz and theresonance frequency around 1.5 kHz. The microphone mounted, asshown in Fig. 3(a), increases the gain by more than 50 dB at 1 Hz. Thegain of −33 dB (0dB = 1V/Pa) with flat characteristics is shown bythe solid line in Fig. 3(b).

B. Experiment Procedures

The measurement experiments were carried out by the dual mi-crophone system shown in Fig. 3(a). The five biosignals above weremeasured not only by the proposed device, shown in Fig. 3(a), but alsoby the conventional equipment as a reference. The sampling interval forall measurements was 1 ms, and the gain of the microphone amplifierwas 6 dB with the flat frequency range of 0.1 Hz–10 kHz.

IV. EXPERIMENTAL RESULTS

A. Plethysmogram

As shown in Fig. 4(a), when holding the housing and covering mi-crophone #1 in Fig. 3(a) by a thumb fingertip, it detected the plethys-mogram as the pressure change. Fig. 4(b) and (c) shows the pressurepulse waves measured by the microphone, as well as by an optical pulseoxymeter of the photocell type (Mizuno Co. Ltd., Digital Monitor 30MB-1012). The fingertip pressure could be measured by lightly push-ing the flexible film covering of the microphone; holding the fingertiplike for the pulse oxymeter was not required. The pulse oxymeter had along delay due to the low-pass filtering characteristics of the photocell.The periods obtained by both measurements are the same. Fig. 4(d)and (e) shows the spectra of the measurements. The pressure mea-sured by the microphone shows a more conspicuous peak includinghigher components in the pulse frequency than the conventional op-tical pulse oxymeter. The proposed device detects richer informationthan the conventional optical pulse oxymeter. The pulse wave of an op-tical oxymeter shows how hemoglobin in blood moves in the fingertip,i.e., the displacement of blood movement, whereas that of the pressureshows the force that is approximately proportional to the acceleration

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 44, NO. 4, AUGUST 2014 549

Fig. 8. Bed sensing. (a) Setting under the mobile phone housing under a pillow to detect the pulse wave, respiration movement, snoring, and body movement ofa person in bed. (b) Biosignals when the mobile phone housing was set under a pillow. Heart beat, respiration, snoring, and body movement are measured.

of the optical sphygmogram, and the corelation with the pressure pulsewave was 0.92. Thus, in the sphygmogram analysis, the pressure pulsewave can be used instead of the acceleration of the optical sphygmo-gram.

B. Stethoscope

As shown in Fig. 5(a), when the electrical stethoscope wastouched to the bare chest, the cardiac sounds were measuredboth by the conventional electrical stethoscope (Fuji Scope F-812)and the proposed microphone arrangement. Fig. 5(b) and (c) shows themeasured cardiac sounds. The proposed microphone arrangement wasmore sensitive than the conventional electrical stethoscope. This is dueto the high sensitivity of the microphone, as shown in Fig. 3(b). Fig. 5(d)and (e) shows the spectra of the cardiac sounds in Fig. 5(b) and (c),revealing a clear difference. The conventional electrical stethoscopedetected the spectrum of cardiac sound of higher than 7 Hz, whereasthe proposed one detected that from around 0.1 Hz containing the com-bined spectra of respiration, pulse wave and its harmonics, and cardiacsound.

C. Simultaneous Measurements of Pulses at the Fingertipand Chest Over the Clothes

As shown in Fig. 6(a), when holding the housing in Fig. 3(a) andlightly pushing the cover of microphone #1 by the thumb and pressingthe housing toward the chest over clothes, it detects a pulse wave at thethumb and the cardiac sound including the respiration and the pulsewave. Fig. 6(b) shows the simultaneous measurements. The proposedsystem detected the cardiac sound even over clothes, whereas the con-ventional stethoscope could not. The wave of the thumb fingertip isdelayed by about 0.16 s from the cardiac sound. The delay time fromthe heart to fingertip is related to blood pressure [11], [12]. Thus, thedelay time could provide blood-pressure-related information.

D. Simultaneous Measurements of Pulses of the Fingertipand Neck Near the Carotid

As shown in Fig. 7(a), when holding the housing and coveringmicrophone #1 by a thumb fingertip and covering microphone #2 bythe neck near the carotid, the two microphones detect two pulse waves.Fig. 7(b) shows the simultaneous measurements. The pulse wave in the

550 IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 44, NO. 4, AUGUST 2014

fingertip is delayed by about 0.11 s from the pulse wave in the necknear the carotid, which is also related to the blood pressure.

E. Bed Sensing of Pulse Wave, Respiration, Snoring,and Body Movement

As shown in Fig. 8(a), when the housing is set under a pillow,microphone #2 detects the pulse wave, respiration, snoring, and bodymovement. Fig. 8(b) shows the biosignals. The signals were as clearas those from conventional pneumatic measurements described in theliterature [4].

V. DISCUSSION

We have described a novel application of a bidirectional microphonemounted in a model mobile phone housing. The strategy is to make theaudio microphone a low-frequency microphone with high sensitivityby setting one pressure-detecting port in one closed space. By thisstrategy, the cutoff frequency expanded from 200 to 0.5 Hz or less andthe gain increased by more than 50 dB in the low-frequency range. Weapplied this idea to a mobile phone microphone that works not onlyas an audio microphone but also as a biosignal detecting sensor: asa sphygmograph at the fingertip, as a stethoscope detecting the delaytime for a pulse to arrive from the neck near the carotid or on the chestnear the heart to the fingertip, and as a bed sensing device. When itis used as a plethysmogram, the proposed microphone system detectsthe pulse wave with faster response and more conspicuous spectra thanthe conventional optical pulse oxymeter, and the wave is similar to thesecond derivative of the optical pulse wave obtained by an oxymeterwith a correlation of 0.92. When used as a stethoscope, it detects notonly the cardiac sound but also the pulse wave and respiration. Further,it detects the biosignals even over clothes, which is not possible with theconventional electrical stethoscope. When the dual microphone systemis used, it detects the delay time of the pulse waves from the necknear the carotid or chest near the heart and the fingertip. Finally, whenused as a bed sensing device and set under a pillow or bed mattress,it detects the biosignals the same as the conventional pneumatic airmattress method.

In this study, a microphone with a diameter of 10 mm was used toinvestigate the feasibility, but it would need to be smaller in an actualsystem. A small condenser microphone can be realized as described inthe literature [10]. Mobile phones with our microphone system couldsupport ubiquitous health care services.

The delay time from the neck carotid to the fingertip or from thechest near the heart to the fingertip measured by the dual microphonesystem provides interesting medical information associated with theblood pressure. Conventionally, the delay time of a photoplethysmo-gram from electrocardiography is used [11]–[13], whereas our methodcan measure the delay time only by the dual microphone.

REFERENCES

[1] T. Morgenthaler, C. Alessi, L. Friedman, J. Owens, V. Kapur, B.Boehlecke, T. Brown, A. Chesson Jr, J. Coleman, T. Lee-Chiong, J. Pancer,and T. J. Swick, “Practice parameters for the use of actigraphy in the as-sessment of sleep and sleep disorders,” SLEEP, vol. 30, no. 4, pp. 519–529,2007.

[2] B. Sivertsen, S. Omvik, O. E. Havik, S. Pallesen, B. Bjorvatn, G. H.Nielsen, S. Straume, and I. H. Nordhus, “A Comparison of actigraphy andpolysomnography in older adults treated for chronic primary insomnia,”SLEEP, vol. 29, no. 10, pp. 1353–1356, 2006.

[3] N. L. Johnson, H. L. Kirchner, C. L. Rosen, A. Storfer-Isser, L. N. Cartar,S. Ancoli-Israel, J. L. Emancipator, A. M. Kibler, and S. Redline, “Sleepestimation using wrist actigraphy in adolescents with and without sleepdisordered breathing: A comparison of three data modes,” SLEEP, vol. 30,no. 7, pp. 899–905, 2007.

[4] K. Watanabe, T. Watanabe, H. Watanabe, H. Ando, T. Ishikawa, andK. Kobayashi, “Noninvasive measurement of heartbeat, respiration, snor-ing and body movement of a subject in bed via a pneumatic method,”IEEE Trans. Biomed. Eng., vol. 52, no. 12, pp. 2100–2107, Dec. 2005.

[5] N. Bu, N. Ueno, and O. Fukuda, “Monitoring of respiration and heartbeatduring sleep using a flexible piezoelectric film sensor and empirical modedecomposition,” in Proc. IEEE Eng. Med. Biol. Soc., 2007, pp. 1362–1366.

[6] X. Zhu, W. Chen, T. Nemoto, Y. Kanemitsu, K. Kitamura, K. Yamakoshi,and D. Wei, “Real-time monitoring of respiration rhythm and pulse rateduring sleep,” IEEE Trans. Biomed. Eng., vol. 53, no. 12, pp. 2553–2563,Dec. 2006.

[7] D. C. Mack, J. T. Patrie, P. M. Suratt, R A. Felder, and M. A. Alwan,“Development and preliminary validation of heart rate and breathing ratedetection using a passive, ballistocardiography-based sleep monitoringsystem,” IEEE Trans. Inf. Technol. Biomed., vol. 13, no. 1, pp. 111–120,Jan. 2009.

[8] T. Watanabe and K. Watanabe, “Noncontact method for sleep stage esti-mation,” IEEE Trans. Biomed. Eng., vol. 51, no. 10, pp. 1735–1748, Oct.2004.

[9] K. Watanabe, Y. Kurihara, and H. Tanaka, “Ubiquitous health monitoringat home—Sensing of human biosignals on flooring, on tatami mat, in thebathtub, and in the lavatory,” IEEE Sensors J., vol. 9, no. 12, pp. 1847–1855, Dec. 2009.

[10] K. Watanabe, Y. Kurihara, T. Nakamura, and H. Tanaka, “Design of a low-frequency microphone for mobile phones and its application to ubiquitousmedical and healthcare monitoring,” IEEE Sensors J., vol. 10, no. 5,pp. 934–941, Mar. 2010.

[11] W. Chen, T. Kobayashil, S. Ichikawa, Y. Takeuchi, and T. Togawa, “Con-tinuous estimation of systolic blood pressure using the pulse arrival timeand intermittent calibration,” Med. Biol. Eng. Comp., vol. 3, pp. 569–574,2000.

[12] U. Yoon, J. H. Cho, and G. Yoon, “Non-constrained blood pressure moni-toring using ECG and PPG for personal healthcare,” J. Med. Syst., vol. 33,pp. 261–266, 2009.

[13] C. Ahlstrom, A. Johansson, F. Uhlin, T. Lanne, and P. Ask, “Noninvasiveinvestigation of blood pressure changes using the pulse wave transit time:A novel approach in the monitoring of hemodialysis patients,” Jpn. Soc.Artif. Organs, vol. 8, pp. 192–197, 2005.