Virtual Medical Instrument for OTOROB Based on LabVIEW for Acquiring Multiple Medical Instrument LCD Reading Using Optical

Character Recognition

Muralindran Mariappan 1, Vigneswaran Ramu 1, Thayabaren Ganesan 1, Brendan Khoo 1 and

Kumarheshan Vellian 1 1 Robotics & Bio-Medical Engineering Research Group (RoBiMed), Universiti Malaysia Sabah, Malaysia

Abstract. OTOROB is a telemedicine mobile robot for orthopaedic surgeons that have remote presence capability to diagnose patients in remote area. As a telemedicine robot, it requires a set of medical instrument for a doctor to diagnostic patients. This paper presents a method to acquire the multiple medical instruments’ LCD reading using the concept of virtual medical instrument based on LabVIEW and its optical character recognition (OCR) module. The data obtained is then transmitted to OTOROB’s remote computer over internet. The medical instrument LCD screens and a USB camera are placed inside a box with a light source. Image captured by the camera is processed by LabVIEW OCR system. The OCR system is trained continuously until it is able to recognize characters consistently. The result shows that the virtual medical instrument is able to acquire and transmit the medical instrument LCD reading with very good accuracy and repeatability.

Keywords: virtual instrument, telemedicine, OTOROB, LabVIEW, optical character recognition.

1. Introduction Virtual medical instrument refers to a system that integrates software and hardware to form a usable

medical device on computer [1]. As Personal Computers (PC) are becoming more affordable, the usages of virtual instruments are increasing drastically in biomedical field. The most popular software used to create virtual instrument in general is National Instrument (NI) LabVIEW. It provides user friendly interface and extensive tools to interface, model, process and display virtually any instrument.

Telemedicine refers to the application of medical data and information which is transferred from one location to another over a communication. Virtual medical instrument plays a vital role in telemedicine as a system that obtains the medical instrument data required by doctors in remote location to diagnostic a patient. By implementing this concept, our research team is developing a telemedicine mobile robot aimed for orthopaedic surgeons called OTOROB [2]. One of the most important part in OTOROB is the medical instrument module which requires a reliable virtual medical instrument that is able to transmit data over the internet to a doctor operating OTOROB in remote location.

M.Lascu in [3] and B.Grinstead in [4] used data acquisition card (DAQ) to acquire biomedical signal from patient’s body and utilized LabVIEW software to process the signal to obtain usable data. Medical data such as Electrocardiogram was displayed on a virtual medical instrument.

T.Xin in [5] created a mixed type wireless telemedicine monitor centre system using LabVIEW, general packet radio service (GPRS) and Digital Signal Processor (DSP). A telemedicine central unit receives patient vital signs which were captured and transmitted by a PDA over the internet using GPRS and also by DSP boards over LAN.

A study by Y.Lin described an automatic instrumental reading system using CMOS camera and neural network system to do optical character recognition on an ocular optical instrument [6]. Conversion

2011 International Conference on Biomedical Engineering and Technology IPCBEE vol.11 (2011) © (2011) IACSIT Press, Singapore

70

algorithms together with self-learning back propagation neural network were created to process the image captured. R.P.Ghugardare in [7] did a similar research to obtain seven-segment display reading using OCR. Both of these systems are very hard to be implemented on other application as it will require the algorithm to be rewritten.

Research done by I.Lita in [8] and Y.Zhong in [9] share the similarity in which both of them used webcam and LabVIEW Vision module to detect moving objects. Images quality from a normal webcam is sufficient for LabVIEW to process and result obtained by both research proved that LabVIEW Vision module is very advance in term of features and at the same time very easy to use.

Research in [3], [4], [5], [8], and [9] proved that LabVIEW is easy to utilize yet can produce advance virtual instrument that is superior to traditional medical instrument. Besides using DAQ card and electronic circuits to acquire and process raw signal, another way to obtain biomedical data is by doing image processing on the existing medical instrument itself. This method has an advantage over previous one in term of biomedical output data reliability. It is known that medical instruments sold in market are tested rigorously to make sure that it performs up to standard and display accurate patient vital signs. By directly capturing its display, the obtained medical data is definitely accurate.

To acquire alpha numerical data from medical instrument’s display, an image processing method called optical character recognition (OCR) will be used. LabVIEW includes an OCR module which makes it easier to implement on the virtual medical instrument. According to orthopaedic doctor, the basic medical instruments required for orthopaedic diagnostic are Vascular Doppler, Blood Pressure Monitor and Pulse Oximeter.

2. Virtual Medical Instrument

2.1. Architecture The virtual medical instrument is designed to acquire and process LCD readings from three different

medical instruments in real-time. Figure 1 shows the block diagram of the system’s architecture.

Fig. 1: Virtual Medical Instrument Architecture

Referring to Figure 1, the USB camera represents a normal 1.3 Megapixel webcam which is placed in a specific position so that it can capture readings on all three medical instruments’ LCD at the same time. This will tremendously reduce the time required to capture and process the image compared to processing it separately. This camera is connected to a computer through USB.

Local computer hosts the LabVIEW software which includes Vision Assistant add-on that is required for processing the image acquired previously. Vision Assistant contains Optical Character Recognition (OCR) training module which is essential for translating medical instrument readings from image to data. As the video feed is coming in from USB camera, LabVIEW captures images of the medical instrument LCD and at the same time the OCR system performs the conversion in real-time. This is the key point of this research.

The data obtained will be stored on a document or excel file for external usage. This file will then be used by OTOROB’s graphical user interface (GUI) to display the medical instrument and other data to user. The reason for creating separate GUI is to ensure that OTOROB will continue to function even when there is failure in one of the on-board system.

Instrument LCD 1 (Vascular Doppler)

USB Camera Display LCD

Reading on GUI

Instrument LCD 2 (Blood Pressure

Monitor)

Instrument LCD 3 (Pulse Oximeter)

Local Computer

LABVIEW Vision Processing and OCR

Transmit Reading through Internet

Display LCD Reading on GUI

Remote Computer

Internet

71

The GUI will also be able to transmit medical instrument data over the internet to a remote computer which hosts a receiver GUI. It will only transmit the readings when there is changed on the medical instrument’s LCD reading in order to reduce bandwidth usage. The remote user can be located anywhere in the world and still be able to receive the data as long as internet connection is available. This is a part of OTOROB’s telemedicine system.

2.2. Medical Instrument LCD Placement In order to realize the proposed configuration, all three medical instruments’ casing is dismantled and

the LCD of each instrument is separated from its main board. LCD 1 represents the display of Vascular Doppler, LCD 2 for Blood Pressure Monitor and LCD 3 for Pulse Oximeter. Wire connection to each LCD is extended so that they can be joined together to form cascading LCD screen such as shown in Figure 2.

Fig. 2: Medical Instrument LCD Placement

By placing the USB camera at the specific distance from the LCD screens, it is able to capture image that is covered within the yellow line envelope which includes all three LCD screens.

To address the lighting for camera, the camera and LCDs are enclosed within a box to eliminate ambient light. Two white light emitting diodes (LED) are placed behind the camera to provide uniform light source which is crucial for acquiring consistent image.

2.3. Acquiring and Processing LCD Reading The image captured by USB camera is processed by LabVIEW Vision Assistant module. Initially, the

module is configured to search for characters within the specified areas of the image which contained display of all three medical instrument LCD readings such as in Figure 3(a).

Similar to the system in [7], the LabVIEW Vision Assistant OCR system converts the image into grey scale to differentiate dark and light pixels. Thresholding is then applied to convert the image to binary image such as in Figure 3(b). A uniform thresholding technique is chosen on the LabVIEW OCR options in order to compensate uneven image illumination and shadows. Then, LabVIEW OCR intelligence system creates a histogram with peak for maximum density of dark pixels compared to light pixels density and valley for their absence. The corresponding character is then segmented from the peak and the result is shown on the ‘Text Read’ tab on LabVIEW OCR system such as in Figure 3(b).

The image is then compared with a database of pre-trained character image stored in the OCR system to recognize the character it represents.

72

(a) (b) Fig. 3: Labview LCD Reading Area and OCR System

2.4. LabVIEW OCR Training Training the LabVIEW OCR system requires capturing the LCD display image repeatedly until the

system is able to recognize a character accurately. For this purpose, each LCD display OCR is trained separately in order to make it more organisable.

Each time an image is processed, the system outputs the instrument LCD reading in data format which is then compared to an OCR database. If the data is incorrect, the correct value is updated to OCR system database in order for it to learn the specific pattern of each character. After training the OCR repeatedly for specific number of times it is able to output correct character accurately. Figure 4 shows the Labview OCR before and after training.

Fig. 4: OCR before and after training

2.5. Experimental Results Once the LabVIEW OCR system is trained and able to accurately recognize characters on all three

instrument LCD screens separately, it is combined into one complete system. The OCR reading data is compared with the display on medical instruments’ LCD and the result is shown in Table 1.

TABLE 1 COMPARISION BETWEEN ACTUAL LCD READING AND OCR READING

Test LCD 1 Reading

OCR Reading

LCD 2 Reading

OCR Reading

LCD 3 Reading

OCR Reading

1 104 104 119 119 98 98 2 124 124 76 76 80 80 3 59 59 140 140 99 99 4 77 77 91 91 85 85 5 192 192 96 96 92 92 6 61 61 65 65 97 97 7 134 134 102 102 90 90 8 148 148 62 62 100 100 9 91 91 99 99 90 90 10 85 85 39 39 89 89

The result shows that the system is able to recognize all three medical instruments accurately which means LabVIEW OCR system is trained adequately. The surrounding ambient light does not affect the system’s accuracy as the USB camera and LCD screens are enclosed inside a box.

73

The converted OCR data is read and transmitted over the internet by OTOROB’s GUI. The Virtual Medical Instrument System transmission is tested on both broadband and dial-up internet connection. Since the GUI will only transmit data when there is change to LCD reading data, the internet bandwidth required is reduced tremendously. As a result, this system is able to work at acceptable rate in most broadband and narrowband internet.

2.6. Conclusion The work presented in this paper describes a method of obtaining medical instrument LCD readings

using LabVIEW Vision Assistant and OCR modules. The usage of LabVIEW in this project considerably reduced the time required to create a working system. Moreover, the system is proven to be very accurate and reliable in long run. This is essential to ensure that OTOROB operates flawlessly. This paper also highlights the LCD screens positioning, camera positioning and LabVIEW OCR system training. The experimental result shows that acquiring medical instrument LCD reading using OCR is successful with very good accuracy and reliability. Consequently, this system is applicable in OTOROB due to its easiness to implement, cost effectiveness, low bandwidth usage and high accuracy.

3. Acknowledgements The authors would like to acknowledge the funding received from MOSTI, Code No: FRG0108-TK-

1/2007.

4. References [1] L. Chengwei, Z. Limei, and H. Xiaoming, “The Study on Virtual Medical Instrument based on LabVIEW.,”

Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference, vol. 4, Jan. 2005, pp. 4072-5.

[2] M. Iftikhar and M. Mariappan, “Otorob (Ortho Robot) with Docmata (Doctor�s Eye): Role of Remote Presence in Developing Countries,” 2009 Second International Conference on Advances in Human-Oriented and Personalized Mechanisms, Technologies, and Services, 2009, pp. 51-56.

[3] M. Lascu and D.A.N. Lascu, “LabVIEW Based Biomedical Signal Acquisition and Processing,” Image (Rochester, N.Y.), 2007, pp. 38-43.

[4] B. Grinstead and M.E. Parten, “Biomedical signal acquisition using "Labview"” Proceedings. 11th IEEE Symposium on Computer-Based Medical Systems (Cat. No.98CB36237), pp. 157-161.

[5] T. Xin, G. Xing-ming, and C. Min, “Design and Realization of Distributed Wireless Telemedicine Monitor Center Based on Virtual Instruments,” 3rd International Conference on Bioinformatics and Biomedical Engineering , 2009. ICBBE 2009., 2009, pp. 1-4.

[6] Y. Lin, “Study on the Automatic Instrumental Reading System Based on Image Processing,” 2007 International Conference on Mechatronics and Automation, Aug. 2007, pp. 3870-3875.

[7] R.P. Ghugardare, S.P. Narote, P. Mukherji, and P.M. Kulkarni, “Optical character recognition system for seven segment display images of measuring instruments,” TENCON 2009 - 2009 IEEE Region 10 Conference, Nov. 2009, pp. 1-6.

[8] I. Lita, D.A. Visan, and I.B. Cioc, “LabVIEW Application for Movement Detection Using Image Acquisition and Processing,” IEEE 16th International Symposium for Design and Technology in Electronic Packaging (SIITME), 2010, 2010, pp. 225-228.

[9] Y. Zhong, “Research on network of remote real-time surveillance system based on LabVIEW,” 2009 7th IEEE International Conference on Industrial Informatics, Jun. 2009, pp. 60-65.

74