Fall Detection System for Older People Ágoston Srp, Dr. Ferenc Vajda Ágoston Mihály Srp PhD Student Department of Control Engineering and Information Technology, Budapest University of Technology and Economics H-1117 Budapest, Magyar tudósok körútja 2.

Tel.:(+36 1) 463-4025

Fax: (+36 1) 463-2204

E-mail: srp.agoston@iit.bme.hu

Dr. Ferenc Vajda Associate Professor Department of Control Engineering and Information Technology, Budapest University of Technology and Economics H-1117 Budapest, Magyar tudósok körútja 2.

Tel.:(+36 1) 463-2089

Fax: (+36 1) 463-2204

E-mail: vajda@iit.bme.hu

Abstract

Several studies have presented different issues of population ageing including the need of enhancing care

systems for older people using smart technologies. Automatic fall detection without the necessity of constant

inspection has been one of the research points of this field that is in its early stages. This paper introduces a

new method for this issue based on signals of a special, human retina like optical sensor system, which also

ensures the privacy of patients. This method applies an advanced learning system which provides results based

on previously recorded and trained data streams of real falls. According to the experimental results this

method looks more robust than earlier solutions; however, in vivo tests will be executed in the forthcoming

several months in various nursing homes of Europe to enable us to draw final conclusions.

Keywords: homecare, fall detection, privacy

1. Introduction

Human society in general and Europe in particular is experiencing major demographic changes since the turn of

the 20th century. The unprecedented rise of the populations average age will cause a massive increase in the

population with age of 65 and over within our lifetime (Fig. 1). Society will face two major challenges: elderly

care services will require more investment due to the increasing and ageing population of seniors, while a

simultaneous shortage of skilled care-givers will be caused by the decreasing population of working age. The

ever further increasing life expectancy due to technological and medical progress will further exacerbate this

imbalance. Some solutions such as video monitoring already exist and are commonly used in nursing

institutions, but they are far from ideal solutions since they still require human resources for monitoring and

compromise Patients’ privacy. The other widely spread solution of “panic buttons” does not work sufficiently,

as well. For the independently living older people fall is one of the major health hazards [Duthie, 1989]:

approximately 30% of people of age 65 or above fall at least once each year. Medical service institutions have

an even higher percentage, which can be expected, since their inhabitants are as a rule frailer and their general

condition is worse. In principle only a fraction of these falls require immediate or any kind of medical attention,

but those cases are usually exacerbated by the time passing between the fall and its discovery. Additional

psychological trauma is not uncommon in these instances. The result is fear from falling, which causes trough

decreased mobility and activity further worsening of the patients’ general condition.

Fig. 1. The changing age structure in the EU1

Existing solutions, like video monitoring and patient raised alarms, have proven to be insufficient, ineffective,

or both. Video monitoring is commonly used in nursing institutions but has major drawbacks. From a legal and

comfort aspect the patients’ privacy is compromised, and humans are still needed for the actual monitoring.

The patient raised alarm has proven to be ineffective. It depends on the capability and willingness of patients

to raise the alarm. A fixed alarm may not be reachable; a wearable device may be forgotten. A fall resulting in

unconsciousness automatically renders the alarm useless. To conclude: existing solutions are usually

insufficient, ineffective and/or infeasible. An effective and robust automated monitoring system is needed.

Requirements for such a system include unobtrusiveness (it is important for the older people that their need

for this monitoring is not apparent), reliability (patients and caregivers depend on the system to raise an alarm

in an emergency) and robustness (the system has to be able to handle unusual or unknown situations reliably).

In this paper we examine an automated fall detection scheme involving “asynchronous temporal-contrast”

(ATC) vision sensors and Neural Networks. ATC vision sensors are a new biologically inspired technology using a

different approach to sensing, the so-called silicon-retina. Because of the nature of this type of sensor

traditional image processing methods are not applicable, since there is no image in the traditional sense. We

introduce some newly developed event based methods to fully realize the potential of this new technology,

and the fall detection methods we found to be most feasible for use with an overhead stereo ATC sensor.

1 ’Eurostat 2008, EUROPOP2008, convergence scenario’

2. A Human Retina like Sensor

In an Asynchronous Temporal-Contrast Vision Sensor (ATC sensor) each pixel independently measures local

relative intensity changes, in continuous time and generates events. The output of the sensor is an asynchronous

stream of timestamped digital pixel “addresses”. These addresses identify the position of the pixel sending the

event. The timed address-events indicate scene reflectance changes. [Lichtsteiner, Posch and Delbruck, 2008]

There are several important things to note about ATC sensors. Normal cameras have a defined frame rate and

fast events may be missed. They transmit a large amount of worthless data representing the same constant scene,

and take pictures thereby potentially compromising the privacy of patients.

Because of their asynchronous nature ATC sensors miss no movement regardless of how fast. They are only

sensitive to changes, so no bandwidth, computing resources or power consumption is wasted on analysing the

same content over and over again. ATC sensors provide no picture, but feature sets belonging to the changing

pixels including polarity (on/off: direction of changing in lightness, i.e. lightening/darkening), time and location

(i.e. position of the pixel in the sensing array) of the event.

One of the most important facts for fall detection to note is that ATC sensors realize background segmentation -

the differentiation of content and background - by the very nature of their sensing. As only the dynamic content

is sensed, the background is discarded. Thus, the need for difficult and computationally expensive background

segmentation is eliminated, making optical fall detection possible not only in laboratory conditions, but in real

life environments. The use of ATC sensors makes realizing an effective, reliable and robust automated

monitoring system for fall detection possible.

An additional advantage of the ATC sensor in the context of elderly care is the lack of picture. Only silhouettes

shown in this paper can be extracted at all and even then only under laboratory conditions (Fig. 2). This is a huge

advantage and helps to alleviate privacy concerns – equally for patients, relatives, caregivers and regulatory

institutions.

Fig. 2. Separate sensor images (left and middle) and 3D data (right)

3. Feature Extraction

The advantage of the ATC sensor also presents a challenge: conventional image processing techniques are not

applicable, since there are no images in the traditional sense. There are ways to circumvent this, but the

advantages are easily lost in the process.

Traditional techniques need image frames to work with, which have to be generated from the Timed Address

Events (TAE) of the sensor (Fig. 3). It is possible to avoid this issue by generating images from the TAEs, but this

may be used as a stopgap at best and it is suitable only for comparison purposes. The real solution for dealing

with data from ATC sensors is to have algorithms operate with TAEs. Development of these however requires a

completely different mindset.

Falls have many possible indicators: changes in height, velocity and / or acceleration, bounding box of the

changing silhouette and its ratios, relative positions of anatomy etc. Some of these are especially suited for

TAE processing, others less. A number of them require the tracking of a specific point, which requires careful

consideration regarding the choice of the point to be tracked.

3.1. Special Points of Interest

One such point is the highest point on the body, which in most cases is the top of the head (Fig. 3 left).

Unfortunately this is not universally true (Fig. 3 right), as during a fall, or even during normal movements arms

or legs may become the highest point, causing a discontinuity in the path of the tracked point.

Fig. 3 Highest point feature problem: Top of the head (left) or elbow (right)?

2

Another point is the Centre of Gravity (COG) of the body (e.g. [Fu et al, 2008]). It is very easy to compute and is

influenced by small movements just slightly.

A very similar point is the COG of the head. During a fall the head of a person is the part of the body which

moves with the highest velocity. This causes a high Event Rate on the sensor (which is analogous to good

visibility) and also makes using velocity or acceleration features easier. The problem is that separation of the

head from the rest of the body would require additional computational capacity and is difficult to accomplish in

a small, hardly visible device.

The final possibility we considered was to track the shoulder-line of the individual. It is an anatomical regularity

that during normal everyday movement of the head and upper body the shoulders move at last. This ensures

insensitivity to small motions of the head in activities of daily living (ADL). Unfortunately, computation of the

shoulder-line is also a significant problem.

2 Images in bottom row are only illustrations

After analysing the possibilities and considering their advantages and the problems associated with them, we

decided to use two special points of interest in our fall detection scheme, namely the highest point and the

Centre of Gravity.

3.2. Features of Behaviour Analysis

Several features are associated with the chosen points of interest. One of the more simplistic features is height

[Fu et al, 2008]. If the height of a tracked point falls below a previously set value an alarm is raised. This

method is very simple and has some flaws: An alarm may be triggered by normal behaviour such as leaning

down, crouching etc.

A somewhat more sophisticated approach applies the vertical velocity and/or acceleration of the tracked

points. Many categories of falls include a quasi-free-fall component, where vertical velocity shows a linear

increase, which can be used to indicate falls [Noury et al, 2007]. Not all falls have such a component, but in

conjunction with other features (e.g. position) it is quite useful, since sensitivity can be increased and false

alarm rate reduced. Using a-priori information [Li, Zhou and Stankovic, 2008] (e.g. knowledge of the position of

the bed) may help to reduce the false alarm rate, but such solutions need on-site calibration of the system,

which raises installation costs.

An even more sophisticated procedure monitors the whole velocity vector. Sudden changes of great magnitude

in velocity vector may indicate an unusual event: Striking a hard surface during a fall or impact with the floor

may cause significant changes in the direction and magnitude of the velocity vector. Velocity and acceleration

vectors can easily be measured with wearable devices, but are difficult to compute accurately using optical

sensors.

Another widely used feature is the bounding box of the silhouette [Anderson et al, 2006; Juang and Chang,

2007] (Fig. 4, Fig. 5). Conveniently, the silhouette is readily obtained from TAE data.

Fig. 4. Bounding box in standing

position

Fig. 5. Bounding box in lying position

We found the bounding box feature to be potentially promising for fall detection, particularly its width to

height ratio [Anderson et al, 2006; Juang and Chang, 2007], which differs significantly in erect (Fig. 4) and prone

(Fig. 5) positions. Regrettably, using solely this feature lying down would easily be misclassified as a fall.

A 3D bounding box may have some drawbacks. Such a body is characterized by 3 dimensions: height, width and

depth. These lead to 3 possible ratios encoding the direction of the supine subject, which is irrelevant to fall

detection. The basic concept however remains viable, so we conceived using a bounding cylinder. Its height to

diameter ratio is high for a standing and low for supine subject. The direction the subject is lying in does not

factor into the ratio due to the circular base. We investigated the possibility of using the relative positions of

head and torso or the relative orientation of torso and thighs, as described by Nyan, Tay and Mah (2008) to

indicate falls. To the best of our knowledge these features are at present even under laboratory conditions with

worn markers difficult to obtain, making their use impracticable.

In our fall detection scheme we selected the position, speed and acceleration of the COG of the body and of

the highest point and the Bounding Cylinder with its height to diameter ratio. Cooperatively used these

features may be a step towards realizing a reliable and robust automated fall detection system.

3.3. 2D Filter Algorithm

Analysing the sensor and its data we found the output to be rather noisy. In conventional applications this kind

of salt & pepper noise (akin to a grainy image) is easily eliminated. Those methods are not directly applicable

with TAE data; therefore we implemented a new filter to reduce the noise of the sensors (Event space filter).

Without going into detail, the filter works by removing solitary TAEs (i.e. TAEs that do not have a sufficient

number of other TAEs in their spatial and/or temporal vicinity).

Fig. 6. Filter comparison – Median filter applied after conversion to image

We compared our algorithm with a widely used conventional filter (Median Filter). For this purpose we

converted sets of TAEs to conventional images. The unfiltered “image” of the sensor is shown on the left in

Fig. 6. While the conventional filter removed (after conversion) all noise it also removed valuable data

(Fig. 6, middle). Our filter eliminated somewhat less noise; however it preserved significantly larger portion of

the data.

3.4. 3D Filter Algorithm

Following a successful verification of the 2D filter and the feature extraction algorithms we applied them to

analyse our data. The bounding cylinder feature performed significantly below our expectations. We made a

detailed analysis of the 3D data and found that a seemingly insignificant portion of the noise had slipped

through the filtering. Because of the calculation method of the bounding cylinder this was enough to

significantly affect that feature. Analysis showed the residual noise TAEs to be at a significant spatial distance

from the subject’s silhouette in 3D space. Using this knowledge we assumed patients’ height to be less than 7.9

feet (2.4 meters) and developed a (3D) filtering method. Using this algorithm, at first the COG of the silhouette

is determined, then all TAEs are examined according to their distance from the COG. All TAEs farther than 3.9

feet (1.2 meters) are discarded as noise.

Application of the described filter significantly improved the quality of all features.

3.5. Effect of feature extraction

Examined as a whole the extracted features seem to be well suited for fall detection. During a fall there is a

significant decrease in the height of the highest point (Fig. 7, left) and the COG (Fig. 7, middle), and the

corresponding velocities and accelerations also distinctly differ. Both the number of TAEs and the BC ratio have

marked differences in case of falls compared to non-falls (Hiba! A hivatkozási forrás nem található., right).

Fig. 7. Extracted features

Careful examination of the results also indicated possible problems. It is apparent, that parts of the sequence

are very similar to a fall, despite not being one. The only marked difference is the low number of TAEs during

these. Further tests will show if the subsequent parts of the system will be able to distinguish these, or if this

issue will need to be handled separately.

4. Algorithmic Learning of falls

A major problem faced in fall detection is summed up in a single question: What is a fall? We are unable to

satisfactorily answer this question, since we are incapable of algorithmically describing what a fall is.

Consequently algorithmic approaches face severe difficulties in fall detection. However, we can give plenty of

examples of falls and not-falls.

A major focus of machine learning research is to automatically learn to recognize complex patterns and make

intelligent decisions based on data [Nyan, Tay, and Mah, 2008]. In the final analysis a fall is nothing more than a

spatial and temporal pattern, therefore it is possible to use a learning system to distinguish falls and non-falls.

After careful consideration of the possible approaches of machine learning we chose to use an artificial neural

network (ANN) as the learning part of our fall detection scheme.

4.1. Artificial Neural Network

The basic unit of an ANN is the neuron. It basically applies a transfer function to the weighted sum of its inputs.

An ANN learns by example: during the teaching process (training) these weights are adjusted using the so

called backpropagation algorithm. It features parallel response to stimuli and information is processed in a

similar way to the human brain: A large number of highly interconnected elements work in parallel to solve a

specific problem.

Defining a neural networks structure is an essential part of the design process; the results may depend highly

on it. The network should have enough neurons in a suitable structure for the task, but should not have much

more to prevent overfitting (i.e. when a network learns the example set, not the underlying relation in the data

set) and to prevent excessive training times. It must also be noted that too large networks may become

impossible to train. We are in the process of investigating several possible structures for the neural network.

4.2. Training the Neural Network

When using neural networks the training data used has a major influence on the results. Insufficient or

unsuitable training data greatly reduces the chance of success. Thus it is essential to have training data of

sufficient volume, quality and variety.

We used a set of falls from standing as training data and a separate set as test data. We limited this initial

investigation to a single category to simplify and shorten training, as our goal in this phase was to obtain a

proof of concept.

4.3. Simulation results

Fig. 8. Simulation results: NN output above and fall occurrences below

Our preliminary results (Fig. 8) with the selected partially trained candidate network are promising. Even

partially trained the network is able to recognize falls from a standing position. Although false alarm rate is still

high, no falls were missed (the figure shows only the final output, falls have been marked with vertical red

lines). Because of the false alarm rate post processing algorithms are necessary. Using them the ratio of false

alarm rates may get eliminated drastically without increasing the number of missed falls. We can state that we

have been able to prove that the initial concept is sound.

5. Future work

At this moment we are running training trials on a number of different neural networks with our current data

to determine the most suitable structure. We are currently in the process of acquiring a suitably large and

varied database of falls and not-falls, which will serve as the basis for the training data necessary to extend fall

detection capability to arbitrary types of falls.

In vivo trials of the system will begin in the second quarter of 2011 in select care institutions. The developed fall

detection system will not be a stand-alone device, but integrated into the existing system of the company

EVERON. The new capability will be added to the existing system in select apartment units of the Senioren

Wohnpark Weser GmbH in Germany and the Yrjö ja Hanna in Finland. The goal of these trials is twofold: The

first goal is, of course, to test the system in a real life environment. The second is to gather realistic data. While

we do not wish any participants in the trials ill, we hope to record as many real falls as possible. This would

greatly increase our ability to better the fall detection system.

Based on our results so far we are confident that the final system will be able to reliably and robustly

distinguish falls from not-falls, raising an alarm as and when a fall occurs, with high confidence, and a low

number of false alarms.

Acknowledgements

This work has been supported by the scientific program of the „Development of quality-oriented and

harmonized R+D+I strategy and functional model at BME” (Project ID: TÁMOP-4.2.1/B-09/1/KMR-2010-0002).

The authors gratefully acknowledge the contributions of the Hungarian National Office for Research and

Technology (NKTH) and the European Committee in the frames of the AAL Joint Programme (AAL-2008-1,

Project "CARE - Smart Private Homes for Elderly Persons").

References

Anderson, D. et al (2006): „Recognizing Falls from Silhouettes” in Proceedings of the 28th IEEE EMBS Annual

International Conference, New York City, USA, August 30September 3, 2006, pp. 63886391

Duthie, E (1989): „Falls” in Medical Clinics of North America, April-June 1989, pp. 13211335

Fu, Z., et al (2008): „An Address-Event Fall Detector for Assisted Living Applications” in IEEE Transactions on

Biomedical Circuits and Systems, June 2008, vol. 2 no 2, pp. 8896

Juang, C.-F.and Chang, C.-M. (2007): „Human Body Posture Classification by a Neural Fuzzy Network and Home

Care System Application” in IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and

Humans, vol. 37, no 6, November 2007, pp. 984994

Li, Q., Zhou, G. and Stankovic, J. A. (2008): „Accurate, Fast Fall Detection Using Posture and Context

Information” in ACM SenSys 2008, Raleigh, NC, USA, pp. 443444

Lichtsteiner, P., Posch, C. and Delbruck, T. (2008): „A 128×128 120 dB 15 µs Latency Asynchronous Temporal

Contrast Vision Sensor” in IEEE Journal of Solid-State Circuits, vol. 43, no. 2, February 2008, pp. 566576

Noury, N. et al (2007): „Fall detection – Principles and Methods” in Proceedings of the 29th Annual

International Conference of the IEEE EMBS Cité Internationale, Lyon, France, August 23-26, 2007, pp. 16631666

Nyan, M. N., Tay, F.E.H., Mah, M.Z.E. (2008): „Application of motion analysis system in pre-impact fall

detection” in Journal of Biomechanics 41 pp. 22972304