Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
SOFTWARE AND HARDWARE CO-SIMULATION PLATFORM FOR IMAGE PROCESSING
JOHNNY KONG JAK KAN
UNIVERSITI TEKNOLOGI MALAYSIA
i
“I hereby declare that I have read this project report and in my
opinion this project report is sufficient in terms of scope
and quality for the award of the degree of
Master of Engineering (Electrical - Computer and Microelectronics System)”
Signature : ...............................................
Name of supervisor : PROF. DR. MOHAMED KHALIL HANI
Date : 15 JANUARY 2014
i
SOFTWARE AND HARDWARE CO-SIMULATION PLATFORM
FOR IMAGE PROCESSING
JOHNNY KONG JAK KAN
A project report submitted in partial fulfilment of the
requirements for the award of the degree of
Master of Engineering (Electrical - Computer and Microelectronics System)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia
JANUARY 2014
iii
My genuine dedications to,
My beloved daddy, mummy and sister;
My enduring supervisor;
My compassionate friends;
And the love ones;
Who are always there for me,
Every step of the way,
Thanks!
iv
ACKNOWLEDGEMENTS
First and foremost I would like to convey my deepest gratitude to my supervisor,
Prof. Dr. Mohamed Khalil who has been the most influential person who always guide,
motivate, support, and advise me throughout my thesis completion with his inspiring and
constructive comments and valuable feedbacks.
Furthermore, I would like to pay special tribute to my seniors, Miss Lee, Mr. Sia,
Mr. Liew and my cherished friends, who are willing to guide me with patience and
keenness. Thank you so much for deserving a lot of time and passion to guide, teach,
motivate and share their experiences and knowledge with me.
In short, I would like to extend my deepest and sincere appreciation to my beloved
family, especially to my mother, who are always support me, encourage me, and listen to
me along my research. They are my source of inspiration and my greatest refuge. And
also for my special person, Miss Hii, thank you for understand me and support me.
v
ABSTRACT
Nowadays, modern SoCs have larger scale and complexity. Modelling hardware design archicteture of SoCs normally required RTL design. Since, the system design is getting larger, it is usually partitioned into two parts: software and hardware. For the sack of achieving real time performance, it is essential to map some of the algorithms into hardware. Previously, the hardware and software design of a system are done by different people. Therefore, the design lifecycle is longer. Now, market pressures on short design cycle, maintainability and reusability of system design. To reduce design cycle, a new method must be applied. SystemVerilog is the extension of Verilog HDL that improved with a lot of new features added. Direct programming interface (DPI) is one of the new features whereas it allow SV call C function and vice versa. This work proposed to use Altera-Modelsim simulator to run co-simulation on several test cases: 16-bit unsigned adder, greater common divisor (GCD), 9-tap FIR filter and binary median filter. All the hardware design modelling are using SystemVerilog due to DPI technique can only work with SV. 16-bit unsigned adder and GCD test cases are the startup work before a more complex and real world case studies are applied. 9-tap FIR filter is designed and the input data are passed from random number generator from C function. The output of FIR fitler are verified with C function. Due to window size of median filter is small, line buffer technique is applied in this work. It is a straight forward method and suit for design processing window in digital hardware. The proposed work has overcome the problems faced when running co-simulation based on Modelsim simulator using DPI technique.
vi
ABSTRAK SoCs kini mempunyai skala yang lebih besar dan rumit. Kebanyakan reka bentuk untuk perkakasan adalah menggunakan teknik RTL. Sistem reka bentuk telah dipechkan kepada dua jenis, iaitu perisian dan perkakasan. Untuk mencapai prestasi masa nyata, sesetengah algoritma perlu dibentukkan dalam perkakasan. Sebelum ini, reka bentuk perkaksan dan perisian adalah dibuat oleh orang yang berlainan. Oleh itu, kitaran hayat reka bentuk akan menjadi lebih panjang. Buat masa ini, pasaran teknologi mempunyai satu trend, iaitu kitaran hayat reka bentuk yang singkat, penyelenggaran dan pengunaan semlua reka bentuk. Demi mengurangkan kitran hayat reka bentuk, kaedah baru haruslah diaplikasi. SystemVerilog merupakan lanjuatan daripada Verilog HDL telah diubahsuaikan dan penambahan fungsi yang banayak. Antaramuka pemprograman terus ialah salah satu fungis baru yang boleh memanggil fungsi dalam bentuk C ke dalam SV atau sebaliknya. Dalam tesis ini, perisian Altera-Modelsim digunakan untuk menjalankan co-simulation terhadap kes-kes ujian yang dipilih. Antara kes ujian yang dipilih ialah 16-bit unsigned adder, pembahagi sepunya yang lebih besar (GCD), penapis FIR dan penapis median binari. Semua reka bentuk perkakasan dimodelkan dengan menggunakan SV. 16-bit unsigned adder dan GCD merupakan kes-kes ujian asas sebelum kes-kes ujian yang lebih rumit dan sebenar diaplikasikan. Data input penapis FIR dipindahkan dari penjana nombor rawak daripada fungsi C. Output penapis FIR akan disahkan dengan fungsi C. Disebabkkan saiz tetingkap penapis median adalah kecil, teknik line buffer digunakan dalam kes ujian ini. Segala masalah yang akan dihadapi semasa simulasi berdasarkan perisian Modelsim mengguanakan teknik DPI telah diatasi dengan mengaplikasikan kes-kes ujian yang dipilih.
vii
TABLE OF CONTENTS
CHAPTER TITLE PAGE
DECLARATION ............................................. ii
DEDICATION ...................................................................................... iii
ACKNOWLEDGEMENTS .................................................................. iv
ABSTRACT ........................................................................................... v
ABSTRAK............................................................................................. vi
TABLE OF CONTENTS ..................................................................... vii
LIST OF TABLES ................................................................................. x
LIST OF FIGURES .............................................................................. xi
LIST OF ABBREVIATIONS AND SYMBOLS ................................ xiii
LIST OF APPENDICES...................................................................... xv
1 INTRODUCTION ............................................................................................ 1
1.1 Background of Study .............................. Error! Bookmark not defined.
1.2 Problem Statement .................................................................................. 3
1.3 Objectives ............................................................................................... 4
1.4 Scope of the Research ............................................................................. 4
1.5 Expected Contribution ............................................................................ 5
1.6 Thesis Outline ......................................................................................... 5
2 BACKGROUND AND LITERATURE REVIEW .......................................... 7
2.1 Median Filters ......................................................................................... 7
2.2 Design of Binary Median Filter in Digital Hardware ............................. 11
1
viii
2.3 SystemVerilog ...................................................................................... 13
2.4 Direct Programming Interface ............................................................. 175
2.5 Previous Work on Co-simulation using SystemVerilog
3 METHODOLOGY ......................................................................................... 19
3.1 Overall Workflow ................................................................................. 19
3.2 Software Tools and Design Environment .............................................. 21
3.2.1 Mentor Graphic Modelsim ......................................................... 21
3.2.2 Using GUI and Command-line Version of Modelsim ................. 22
3.3 Direct Programming Interface ............................................................... 26
3.3.1 Import Function ......................................................................... 27
3.3.2 Export Function ......................................................................... 27
3.4 Test Cases for Verifying Usability of DPI-C ......................................... 28
3.4.1 Startup Test Cases ...................................................................... 28
3.4.2 9-tap FIR Filter .......................................................................... 29
4 SOFTWARE AND HARDWARE PLATFORM DEVELOPMENT ........... 30
4.1 9-tap FIR Filter ..................................................................................... 30
4.1.1 Datapath Unit ............................................................................. 31
4.1.2 Control Unit ............................................................................... 34
4.1.3 Interfacing between C and SV .................................................... 36
4.2 Noise Removal with Binary Median Filter ............................................ 36
4.2.1 Conversion of RGB Image to Grayscale Image .......................... 38
4.2.2 Datapath Unit Design ............................................................... 399
4.2.3 Control Unit Design ................................................................... 42
4.2.4 Interfacing between C and SV .................................................... 44
4.2.5 Miscellaneous ............................................................................ 45
17
15
39
ix
5 RESULTS AND DISCUSSION ..................................................................... 46
5.1 16 bit Unsigned Adder and GCD .......................................................... 46
5.2 9-tap FIR filter ...................................................................................... 49
5.3 Binary Median Filter ............................................................................. 50
5.4 Limitations ........................................................................................... 54
5.4.1 Software Tools of Modelsim ...................................................... 54
5.4.2 Hardware Design of Binary Median Filter .................................. 54
6 CONCLUSION AND RECOMMENDATION ............................................. 55
6.1 Conclusion ............................................................................................ 55
6.2 Recommendation for Future Works ...................................................... 56
REFERENCES .......................................................................................................... 57
Appendices A-C ..................................................................................................... 60-92
x
LIST OF TABLES
TABLE NO. TITLE PAGE
2.1 Partial of difference between SV and Verilog 14
2.2 SystemVerilog DPI capabilities versus Verilog PLI 16
2.3 Data types mapping between C and SystemVerilog 17
2.4 Previous work on co-simulation using SystemVerilog 18
xi
LIST OF FIGURES
FIGURE NO. TITLE PAGE
2.1 An example of 3x3 window mask for median filter. 8
2.2 An image corrupted with impulse noise [4]. 8
2.3 An image after standard median filtering using 3x3
window mask [4].
9
2.4 Generic window mask for CWM filter [11]. 9
2.5 Block diagram of tri-state median filtering scheme. 10
2.6 Searching block for median filter. 11
2.7 Pipelined of multilayer architecture for median filter. 12
2.8 Implementation of 3x3 filter window. Row buffers are
used to reduce the memory access to one pixel per clock
cycle [12].
13
3.1 Overall workflow 20
3.2 Generic block diagram of interface between
SystemVerilog and C.
21
3.3 The command-line and GUI based of Modelsim. 23
3.4 A successful output display when running DPI-C on
hello module.
25
4.1 Data flow graph (DFG) of 9-tap FIR filter. 32
4.2 Signal data flow graph (SDFG). 32
4.3 Signal flow graph. 33
xii
4.4 Functional block diagram of FIR filter. 33
4.5 ASM chart for FIR filter. 35
4.6 5x5 processing window for binary median filter. 37
4.7 Interfacing between binary median filter and conversion
of RGB image to grayscale.
38
4.8 Generic design of line buffers for window-based
computation [7].
40
4.9 Pixel representation diagram for a 6x6 image. 41
4.10 Pixel shifting in line buffers. 42
4.11 ASM chart for binary median filter. 43
4.12 Portion of C code to create a readable mif file by SV. 44
4.13 Portion of SystemVerilog code from testbench. 44
5.1 Verification of 16-bit unsigned adder using DPI-C. 47
5.2 Functional block diagram of top level entity of GCD. 47
5.3 Random numbers set generated by C code as input for
FIR filter.
49
5.4 A portion of output data from SV side of FIR filter. 50
5.5 Image of finger-vein before filtered. 53
5.6 Image of finger-vein after filtered. 53
xiii
LIST OF ABBREVIATIONS AND SYMBOLS
ALU - Arithmetic logic unit
API - Application programming interface
ASM - Algorithmic sate machine
CWM - Center weighted median
DFG - Data flow graph
DPI - Direct programming interface
FIR - Finite impulse response
FPGA - Field programmable gate array
GUI - Graphic User Interface
HDL - Hardware description language
HDVL - Hardware decription and verification language
MATLAB - Matrix laboratory
MRI - Magnetic resonance imaging
PLI - Programming language interface
RAM - Random access memory
RGB - Red Green Blue
RTL - Register transfer level
SDFG - Signal data flow graph
SFG - Signal flow graph
SM - Standard median
SoC - System-on-chip
SV - SystemVerilog
UCDB - Unified Coverage Database
xiv
VLI - Verilog Procedural Interface
2D - 2-dimensional
xv
LIST OF APPENDICES
APPENDIX TITLE PAGE
A Tutorial on how to use modelsim to run DPI 60
B Sources code for 9-tap FIR filter 68
C Sources code for binary median filter 79
1
CHAPTER 1
INTRODUCTION
This thesis proposes a co-design simulation model that utilizing the state-of-art
SystemVerilog HDVL via direct programming interface (DPI) technique on subsystems
of digital signal processing and image processing. This chapter gives an overview of the
proposed title and, problem statement, objectives, scope of work, expected contribution
and thesis outline.
1.1 Background of Study
Nowadays, modern system-on-chip (SoCs) design is getting more complex and
sophisticated. Hardware modeling or design at register transfer level (RTL) will consume
a lot of time and effort. It is impossible to integrate every algorithms or problems into a
single piece of hardware. Even it is done so, the end product of hardware will be costly
and not suitable for the market right now. The market now pressures on short design cycle,
hight fault coverage, high speed and low cost. Since the register transfer level design now
is getting more complicated, it is advisable that some of the algorithm remain in the
2
software. Hardware design is done only when performance is taken into concern. This is
due to hardware design will speed up the whole system or an application.
There is a report show that functional verification has become the biggest
bottleneck as it consumes roughly 70% of chip development time and efforts. RTL
testbenches have become more complex and difficult to manage [1]. New method has to
be introduced to reduce the verification cycle. Therefore, SystemVerilog (SV) can be used
to overcome drawbacks as aforesaid. SV is the extension of Verilog 2001 HDL, contains
both hardware description language and hardware verification language [2]. It enhances
the design specification method, testbenches language including coverage and assertions
application programming interface (API) and direction programming interface (DPI).
Direct programming interface is an interfacing between SystemVerilog and
foreign programming language. Usually, the foreign programming language is referred as
C or C++ programming language. Designers can call C or C++ code into SystemVerilog
or vice versa by using correct protocol and linking model. Hence by using SV and DPI,
designers are able to develop a fast co-simulation model to their desired applications.
Other than that, designers can use the C/C++ code output result as a golden reference to
verify the hardware model designed in HDVL.
Image processing can be defined as 2D function, f(x,y) where x and y are spatial
coordinates. Amplitude f is called intensity or gray level of the image [4]. Digital image
is a 2D rectangular array of quantized sample values of an image. These quantized values
are known as picture elements, image elements and pixels. Basically, digital image
processing is a processing of an input image to obtain desired output image. Image
analysis is understanding of an image as input to extract some useful information of image
output such as features, texture or shape. Computer vision is known as emulating human
vision such as face recognition, fingerprint recognition, tracking moving object and etc.
Image processing can be applied in many area especially, medical (MRI, X-ray,
Ultrasound), military, untouchable territorial explorations (deep space and deep sea),
security and surveillance (biometric, border control) and entertainment.
3
1.2 Problem Statement
With the massive development of electronic information system in today’s society,
SoCs have bigger and more complicated design. Previously, the hardware and software
design are done separately by different designers. The only way to verify the system
design is using field programmable gate array (FPGA). However, verification through
FPGA will take longer time and more effort due to technical details for FPGA board must
be well-equipped. As aforesaid when the design on any applications is getting bigger and
complex, the design has to be done by the same person. If the conventional method is used,
it will cause more debugging problems and interfacing between hardware and software
design. Therefore, it is necessary to ultilize the SystemVerilog HDVL which provides
interfacing between SV and C. It enables the same designers to do both software and
hardware system design on a co-simulation platform. Hence, the design lifecycle of a
system can be greatly reduced.
To achieve real time performance in image processing’s application, it is advisable
to map some systems design into hardware. It is not a good method to keep every modules
of design in software design because a high power computer is needed in order to achieve
real time performance. Thus, it is better to design an embedded system for a particular
applications. Most of the algorithms of image processing compute intensively such as
multiplication and addition. To impove the performance, it is necessary to partition the
design into hardware and software instead of keeping every algorithms in software.
4
1.3 Objectives
The project aims to develop a co-simulation model based on open source software
tool, Modelsim simulator by using SystemVerilog HDVL via direct programming
interface technique. The main objective are supported by the sub-objectives as follow:
(i) To overcome the problems and discover the limitation of Modelsim simulator
when interfacing between SystemVerilog and C functions.
(ii) To propose an efficient design of background noise removal to achieve the
real time performance.
1.4 Scope of the Research
The scopes of the work for this thesis are:
(i) SystemVerilog HDVL is used throughout the all the hardware modeling and C
programming language is applied for the foreign programming language
(Currently DPI only support C but it is designed to be extensible interface that
can support other languages [25]).
(ii) Open source software tools such as Quartus II 13.0 and Altera-Modelsim 10.1d
are used to ensure this work can be repeated and extended in future.
(iii) Test cases that are applied in this work are limited to these problems: a 16-bit
unsigned adder, greater common divisor, 9-tap FIR filter and binary median
filter.
5
1.5 Expected Contribution
The expected contribution of this work is creation a co-simulation environment
based on a free software tool, Modelsim simulator with limited features. Most of the
technical details as well as limitations about the Modelsim are overcome and discovered
by applying several test cases. Moreover, this work will apply the state-of-art hardware
description and verification language (HDVL), SystemVerilog. SV is the extension of
Verilog HDL with more advanced features such as direct programming interface (DPI).
Due to the new release of SV, not many research or papers are published on hardware
design using SystemVerilog. In SV, it allows the interfacing between hardware design
with software design. In another words, a co-simulation model for software and hardware
designs can be developed by ultilizing this state-of-art HDVL. The software designs here
are meant to C.
1.6 Thesis Outline
This thesis is organised into six chapters. The first chapter presents the
introduction of this work, problem statement, objectives, scope of work and expected
contribution of this thesis.
Chapter 2 is the background and literature review chapter. It gives some
information of subsystem of image processing, hardware design implementation of
median filter. Previous related work on state-of-the-art co-simulation using
SystemVerilog DPI is presented as well.
Chapter 3 decribes the methodology of this work. The commands and workflow
on how to use DPI in Modelsim simulator are illustrated. A startup example using DPI
that printf(“Hello World”) is presented. This is a key point before a more complex test
6
case is applied. The import and export are the main keyword to be used to call C function
or SV are explained.
Chapter 4 mainly exaplains on the algorithm of the applied in the proposed test
cases especially FIR filter and binary median filter. The method being used such as
circular buffers and line buffers for data aligning are discussed. The chapter continues
with more explanation on the hardware design from algorithms (9-tap FIR filter and binary
median filter).
Chapter 5 is the design verification and testing chapter. Results and discussion are
given in this chapter. Chapter 6 concludes the limitation of Modelsim simulator and future
work to further enhance the co-simulation using SystemVerilog DPI.
57
REFERENCES
1. Modi D., Sitapara H., Shah R., Mehul, E., (2011). Integrating MATLAB
with Verification HDLs for Functional Verification of Image and Video
Processing ASIC, 2(2), 258–265.
2. SystemVerilog 3.1a Language Reference Manual Accellera’s Extensions
to Verilog.
3. Eng P. C. (2011), "Finger-Vein Biometric Authenticatin in a System-On-
Chip Based on Field Programmable Gate Arrays". Master, Universiti
Teknologi Malaysia, Skudai.
4. Rafael C. Gonzalez, Richard E. Woods (2002). Digital Image Processing.
(Second Edition) Upper Saddle River, New Jersey: Prentice-Hall.
5. Chin C.C., Ju Y. H., Chih P. H. (2008), "An Adaptive Median Filter for
Image Denoising," Intelligent Information Technology Application. IITA
'08. Second International Symposium on , vol.2, no., pp.346,350, 20-22
Dec. 2008
6. J. Astola and P. Kuosmanen. (1997) “Fundamentals of nonlinear digital
filtering,” CRC Press, Boca Raton, FL.
7. Lee Y. H. (2013), "Finger-Vein Biometric System Implemented in an
Embedded Hardware Base on Field Programmable Gate Arrays". Master,
Universiti Teknologi Malaysia, Skudai.
8. Chen T., Ma K. K., Li H. C., (1999). "Tri-state median filter for image
denoising," Image Processing, IEEE Transactions on , vol.8, no.12,
pp.1834,1838.
9. Ko S. J., Lee Y. H., (1991) "Center weighted median filters and their
applications to image enhancement," Circuits and Systems, IEEE
Transactions on , vol.38, no.9, pp.984,993.
58
10. I. Pitas and A. N. Venetsanopoulos (1990), “Nonlinear digital filters:
principles and applications,” Kluwer, Boston, MA.
11. Sun T., Gabbouj M., Neuvo Y. (1995), "Anaylsis of two-dimensional
center weighted median filters", Multidimensioanl Systems and Signal
Processing, Vol 6, Number 2, Page 159-172
12. Vasicek Z., Sekanina L. (2008),"Novel Hardware Implementation of
Adaptive Median Filters," Design and Diagnostics of Electronic Circuits
and Systems, 2008. DDECS 2008. 11th IEEE Workshop on , vol., no.,
pp.1,6, 16-18.
13. Fahmy S.A., Cheung P. Y K, Luk W., (2005) "Novel FPGA-based
implementation of median and weighted median filters for image
processing," Field Programmable Logic and Applications, 2005.
International Conference on , vol., no., pp.142,147, 24-26.
14. Abadi H.Z.H., Samavi S., Karimi N. (2013), "Image noise reduction by
low complexity hardware median filter," Electrical Engineering (ICEE),
2013 21st Iranian Conference on , vol., no., pp.1,5, 14-16.
15. Leeb S.B., Ortiz A., Kirtley J.L.(1991), "Real-time median filtering with
a fast hardware sorter," Applied Power Electronics Conference and
Exposition, 1991. APEC '91. Conference Proceedings, 1991., Sixth
Annual , vol., no., pp.254,260, 10-15.
16. Hiasat A.A., Al-Ibrahim M.M., Gharaibeh K. M., (1999) "Design and
implementation of a new efficient median filtering algorithm," Vision,
Image and Signal Processing, IEE Proceedings - , vol.146, no.5,
pp.273,278.
17. Chatterjee S., Ray A.K., Karim, R., Biswas A.,(2011) "Architecture
Design for Median Filter," Computer Vision, Pattern Recognition, Image
Processing and Graphics (NCVPRIPG), 2011 Third National Conference
on , vol., no., pp.247,250, 15-17.
18. Modelsim Manual 10.1d.
19. Khalil-Hani, M. Digital Systems: VHDL & Verilog Design. UTM
Skudai: Prentice Hall. 2012.
59
20. Mohamed Khalil Hani, PhD (2013) RTL Design for FPGA with
SystemVerilog, 1st Edition. UTM Skudai. Desktop Publisher.
21. Delgado, M. and Pegalajar, M. A multiobjective genetic algorithm for
obtaining the optimal size of a recurrent neural network for grammatical
inference. Pattern Recognition, 2005. 38(9): 1444–1456. ISSN 00313203.
doi:10.1016/j.patcog.2004.03.026.
22. Cochran, J., Horng, S. and Fowler, J. A multi-population genetic
algorithm tosolve multi-objective scheduling problems for parallel
machines. Computers& Operations Research, 2003. 30(7): 1087–1102.
23. Thompson C. D. (1983), "The VLSI complexity of sorting", IEEE Trans.
Computers, vol. 32, pp. 1171-1184.
24. Karaman M., Onural L., Atalar A. (1990), "Design and implementation
of a general-purpose median filter unit in CMOS VLSI", IEEE J. Solid-
State Circuits, vol. 25, pp. 505-512.
25. S. Sutherland, The Verilog PLI is Dead (maybe) Long Live the System
Verilog DPI! (2004).
26. Ruan, A. W., Liao, Y. B., Li, P., Huang, H. C., & Devices, I. (2010). A
Run- Time RTL Debugging Methodology for FPGA-based Co-
Simulation, 891–895
27. You, M.-K., & Song, G.-Y. (2009). Case study : Co-simulation and co-
emulation environments based on SystemC & SystemVerilog. TENCON
2009 - 2009 IEEE Region 10 Conference, 1–4.
28. Li, W. (2009). Throughput Estimation for ModelSim Simulator Tool
Based I ~ j ~, 1014–1018.
29. Chris S., (2008). SystemVerilog for Verification A Guide to Learning the
Testbench Language Features, 2nd edition. Spring Street, New York:
Springer.