Upload
alvin-tyler
View
223
Download
5
Tags:
Embed Size (px)
Citation preview
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
K. SEKAR, Ph.D.BIOINFORMATICS CENTRE
INDIAN INSTITUTE OF SCIENCEBANGALORE 560 012
INDIA
Voice: (91)-80-3942469 FAX : (91)-80-3600683 (91)-80-3601409 (91)-80-3600551
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
APPROACHES TO DEVELOPING DATA MINING
TOOLS
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Abstract
Bioinformatics is one of the fastest growing interdisciplinary areas in the biological sciences and has explored in such a way that we need powerful tools to organize and analyze the data. An overview will be presented on the general features of data mining tools, techniques and its applications.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Bioinformatics is the fashionable new name for the field previously called computational biology.The name is preferred by many because it puts the emphasis on the data storage and analysis, rather than on the biology, and the field is really data driven.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
The term Bioinformatics is used to encompass almost all computer applications in biological sciences, but was originally coined in the mid 1980’s for the analysis of biological sequence data.
The quantity of known sequences data outweighs protein structural data and by virtue of the genome projects, sequence database are doubling in size every year.
A key challenge of bioinformatics is to analyze the wealth of sequence data in order to understand the amassed information in term of protein structure function and evolution.
Wherever possible, a range of different methods should be used, and the results should be married with all available biological information.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
The primary integrating technology that facilitates access to copious data is the world wide web.
Bioinformatics has provided us with a communication channel to reach and decode all this information in a comprehensive manner.
Both the large information repositories and the specialized tools to query them are held on distributed internet sites, therefore Bioinformatics require sound internet navigation skills.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Comprises the entire collection of information management systems, analysis tools and communication networks supporting biology.
Refers to database-like activities involving persistent sets of data that are maintained in a consistent state over essentially indefinite periods of time.
Encompass the use of algorithmic tools to facilitate biological database analyses.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DATA MINING
Datamining is defined as “exploration and analysis by automatic and semi-automatic means, of large quantities of data in order to discover meaningful patterns and rules”.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
The central challenge is to derive maximum results from the wealth of data.This can be achieved by establishing and maintaining databases and providing search and analysis tools to interpret the data.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DATABASEDatabase is nothing but a collection of quantitative data resulting from experimental measurements or observations in various fields of science.Recently interest in database has been kindled through international efforts to organize and analyze the data and update the knowledge
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
A database is essentially just a store of information.They are usually in the form of simple files (just a flat file, say).You can shove information into this store or retrieve it from the store.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Derived DatabaseOne of the greatest challenges in database research is analyze the database in depth and create derived databases to meet the needs or demands without compromising the sustainability and quality of the existing database. Creating desired database is expected is expected to dramatically reduce the workload of the user community and will serve as a highly focused database.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DBREF 1UNE 1 123 SWS P00593 PA2_BOVIN 23 145 SEQADV 1UNE ASN 122 SWS P00593 LYS 144 CONFLICT SEQRES 1 123 ALA LEU TRP GLN PHE ASN GLY MET ILE LYS CYS LYS ILE SEQRES 2 123 PRO SER SER GLU PRO LEU LEU ASP PHE ASN ASN TYR GLY SEQRES 3 123 CYS TYR CYS GLY LEU GLY GLY SER GLY THR PRO VAL ASP SEQRES 4 123 ASP LEU ASP ARG CYS CYS GLN THR HIS ASP ASN CYS TYR SEQRES 5 123 LYS GLN ALA LYS LYS LEU ASP SER CYS LYS VAL LEU VAL SEQRES 6 123 ASP ASN PRO TYR THR ASN ASN TYR SER TYR SER CYS SER SEQRES 7 123 ASN ASN GLU ILE THR CYS SER SER GLU ASN ASN ALA CYS SEQRES 8 123 GLU ALA PHE ILE CYS ASN CYS ASP ARG ASN ALA ALA ILE SEQRES 9 123 CYS PHE SER LYS VAL PRO TYR ASN LYS GLU HIS LYS ASN SEQRES 10 123 LEU ASP LYS LYS ASN CYS HET CA 124 1 HETNAM CA CALCIUM ION FORMUL 2 CA CA1 2+ FORMUL 3 HOH *134(H2 O1) HELIX 1 1 LEU 2 LYS 12 1 11 HELIX 2 2 PRO 18 ASP 21 1 4 HELIX 3 3 ASP 40 LYS 57 1 18 HELIX 4 4 ASP 59 VAL 63 1 5 HELIX 5 5 ALA 90 LYS 108 1 19 HELIX 6 6 LYS 113 HIS 115 5 3 SHEET 1 A 2 TYR 75 SER 78 0 SHEET 2 A 2 GLU 81 CYS 84 -1 N THR 83 O SER 76 SSBOND 1 CYS 11 CYS 77 SSBOND 2 CYS 27 CYS 123 SSBOND 3 CYS 29 CYS 45 SSBOND 4 CYS 44 CYS 105 SSBOND 5 CYS 51 CYS 98 SSBOND 6 CYS 61 CYS 91 SSBOND 7 CYS 84 CYS 96 LINK CA CA 124 O TYR 28 LINK CA CA 124 O GLY 32 CRYST1 47.120 64.590 38.140 90.00 90.00 90.00 P 21 21 21 4
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
SUB-DERIVED DATABASEEXAMPLE-1
RADHASEKAR SHAMIASEKAR SARADASEKAR
EXAMPLE-2
XAXAXA
KAMALA SARADA YAMAHA KANAGA MANASA VANASA PANAMA
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Adding information
to the database
Software tocollate the required
Information from the database
Analyze the collated information
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
WHY A TOOL?
The amount of information in the world is growing exponentially, and it is becoming impossible to effectively manage the data.Machine assistance is clearly necessary, but the difficulty lies in designing systems and softwares that are capable of discovering “useful” information with minimal human intervention.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
PROTEIN DATA BANK(PDB)
GENOME DATABASE(GDB)
STRUCTURAL CLASSIFICATION OF PROTEINS(SCOP)
CAMBRIDGE STRUCTURAL DATABASE(CSD)
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Protein Data Bank (PDB)
&
Genome Database(GDB)
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
PDB (Protein Data Bank)Anonymous FTP - SERVER
PDB Anonymous FTP – server is up to date and contains all the available 20,317 atomic coordinates of macro molecules (Proteins, Nucleic Acids and Carbohydrates) that have been deposited in the protein databank so far.
For Weekly updatehttp://iris.physics.iisc.ernet.in/index.html
For complete entries click on “COMPLETE LIST OF ALL PDB ENTRIES”
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
PDB-MIRROR site machine
3.06 GHz PIV machine
1 GB RD RAM
240 GB Hard Disk
32 MB Graphics Card
Powered by Red Hat 7.3 Linux Operating System
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Given PDB-Id : 1une
HEADER HYDROLASE 05-NOV-97 1UNE TITLE CARBOXYLIC ESTER HYDROLASE, 1.5 ANGSTROM ORTHORHOMBIC FORM TITLE 2 OF THE BOVINE RECOMBINANT PLA2 COMPND MOL_ID: 1; COMPND 2 MOLECULE: PHOSPHOLIPASE A2; COMPND 3 CHAIN: NULL; COMPND 4 EC: 3.1.1.4; COMPND 5 ENGINEERED: YES SOURCE MOL_ID: 1; SOURCE 2 ORGANISM_SCIENTIFIC: BOS TAURUS; SOURCE 3 ORGANISM_COMMON: BOVINE; SOURCE 4 EXPRESSION_SYSTEM: ESCHERICHIA COLI; SOURCE 5 EXPRESSION_SYSTEM_STRAIN: BL21 (DE3) PLYSS; SOURCE 6 EXPRESSION_SYSTEM_PLASMID: PTO-A2MBL21; SOURCE 7 EXPRESSION_SYSTEM_GENE: MATURE PLA2 KEYWDS HYDROLASE, ENZYME, CARBOXYLIC ESTER HYDROLASE EXPDTA X-RAY DIFFRACTION AUTHOR M.SUNDARALINGAM REVDAT 1 06-MAY-98 1UNE 0
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
REMARK 1 REFERENCE 1 REMARK 1 AUTH K.SEKAR,A.KUMAR,X.LIU,M.-D.TSAI,M.H.GELB, REMARK 1 AUTH 2 M.SUNDARALINGAM REMARK 1 TITL CRYSTAL STRUCTURE OF THE COMPLEX OF BOVINE REMARK 1 TITL 2 PANCREATIC PHOSPHOLIPASE A2 WITH A TRANSITION STATE REMARK 1 TITL 3 ANALOGUE REMARK 1 REF TO BE PUBLISHED
REMARK 1 REFN 0353 REMARK 1 REFERENCE 2 REMARK 1 AUTH K.SEKAR,C.SEKARUDU,M.-D.TSAI,M.SUNDARALINGAM REMARK 1 TITL 1.72A RESOLUTION REFINEMENT OF THE TRIGONAL FORM OF REMARK 1 TITL 2 BOVINE PANCREATIC PHOSPHOLIPASE A2 REMARK 1 REF TO BE PUBLISHED REMARK 1 REFN 0353
REMARK 1 REFERENCE 3 REMARK 1 AUTH K.SEKAR,S.ESWARAMOORTHY,M.K.JAIN,M.SUNDARALINGAM REMARK 1 TITL CRYSTAL STRUCTURE OF THE COMPLEX OF BOVINE REMARK 1 TITL 2 PANCREATIC PHOSPHOLIPASE A2 WITH THE INHIBITOR REMARK 1 TITL 3 1-HEXADECYL-3-(TRIFLUOROETHYL)-SN-GLYCERO-2- REMARK 1 TITL 4 PHOSPHOMETHANOL REMARK 1 REF BIOCHEMISTRY V. 36 14186 1997
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
REMARK 2 RESOLUTION. 1.5 ANGSTROMS. REMARK 3 REFINEMENT. REMARK 3 PROGRAM : X-PLOR 3.1 REMARK 3 AUTHORS : BRUNGER
REMARK 3 DATA USED IN REFINEMENT. REMARK 3 RESOLUTION RANGE HIGH (ANGSTROMS) : 1.5 REMARK 3 RESOLUTION RANGE LOW (ANGSTROMS) : 10.0 REMARK 3 DATA CUTOFF (SIGMA(F)) : 1.0 REMARK 3 DATA CUTOFF HIGH (ABS(F)) : 0.1 REMARK 3 DATA CUTOFF LOW (ABS(F)) : 1000000.0 REMARK 3 COMPLETENESS (WORKING+TEST) (%) : 92. REMARK 3 NUMBER OF REFLECTIONS : 17572
REMARK 3 FIT TO DATA USED IN REFINEMENT. REMARK 3 CROSS-VALIDATION METHOD : NULL REMARK 3 FREE R VALUE TEST SET SELECTION : X-PLOR REMARK 3 R VALUE (WORKING SET) : 0.184 REMARK 3 FREE R VALUE : 0.228 REMARK 3 FREE R VALUE TEST SET SIZE (%) : 7. REMARK 3 FREE R VALUE TEST SET COUNT : 1198 REMARK 3 ESTIMATED ERROR OF FREE R VALUE : 0.24
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
REMARK 3 PARAMETER FILE 1 : PARHCSDX.PRO REMARK 3 PARAMETER FILE 2 : NULL REMARK 3 TOPOLOGY FILE 1 : TOPHCSDX.PRO REMARK 3 TOPOLOGY FILE 2 : NULL REMARK 3 OTHER REFINEMENT REMARKS: NULL REMARK 4 1UNE COMPLIES WITH FORMAT V. 2.2, 16-DEC-1996 REMARK 200 REMARK 200 EXPERIMENTAL DETAILS REMARK 200 EXPERIMENT TYPE : X-RAY DIFFRACTION REMARK 200 DATE OF DATA COLLECTION : 26-JAN-1996 REMARK 200 TEMPERATURE (KELVIN) : 291 REMARK 200 PH : 7.2 REMARK 200 NUMBER OF CRYSTALS USED : 1 REMARK 200 REMARK 200 SYNCHROTRON (Y/N) : N REMARK 200 RADIATION SOURCE : NULL REMARK 200 BEAMLINE : NULL REMARK 200 X-RAY GENERATOR MODEL : R-AXIS IIC REMARK 200 MONOCHROMATIC OR LAUE (M/L) : M REMARK 200 WAVELENGTH OR RANGE (A) : 1.5418 REMARK 200 MONOCHROMATOR : GRAPHITE REMARK 200 OPTICS : NULL REMARK 200
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
REMARK 200 IN THE HIGHEST RESOLUTION SHELL. REMARK 200 HIGHEST RESOLUTION SHELL, RANGE HIGH (A) : 1.5 REMARK 200 HIGHEST RESOLUTION SHELL, RANGE LOW (A) : 1.55 REMARK 200 COMPLETENESS FOR SHELL (%) : 63. REMARK 200 DATA REDUNDANCY IN SHELL : 3.7 REMARK 200 R MERGE FOR SHELL (I) : 0.172 REMARK 200 R SYM FOR SHELL (I) : NULL REMARK 200 FOR SHELL : NULL REMARK 200 REMARK 200 METHOD USED TO DETERMINE THE STRUCTURE: THE HIGH RESOLUTION REMARK 200 ATOMIC COORDINATES OF THE WILD TYPE (PDB ENTRY 1BP2) REMARK 200 WERE USED AS THE STARTING MODEL FOR REFINEMENT. REMARK 200 SOFTWARE USED: X-PLOR REMARK 200 STARTING MODEL: WILD TYPE (PDB ENTRY 1BP2) REMARK 200 REMARK 200 REMARK: NULL REMARK 280 REMARK 290 REMARK 290 CRYSTALLOGRAPHIC SYMMETRY REMARK 290 SYMMETRY OPERATORS FOR SPACE GROUP: P 21 21 21 REMARK 290 REMARK 290 SYMOP SYMMETRY REMARK 290 NNNMMM OPERATOR REMARK 290 1555 X,Y,Z REMARK 290 2555 1/2-X,-Y,1/2+Z REMARK 290 3555 -X,1/2+Y,1/2-Z REMARK 290 4555 1/2+X,1/2-Y,-Z
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DBREF 1UNE 1 123 SWS P00593 PA2_BOVIN 23 145 SEQADV 1UNE ASN 122 SWS P00593 LYS 144 CONFLICT SEQRES 1 123 ALA LEU TRP GLN PHE ASN GLY MET ILE LYS CYS LYS ILE SEQRES 2 123 PRO SER SER GLU PRO LEU LEU ASP PHE ASN ASN TYR GLY SEQRES 3 123 CYS TYR CYS GLY LEU GLY GLY SER GLY THR PRO VAL ASP SEQRES 4 123 ASP LEU ASP ARG CYS CYS GLN THR HIS ASP ASN CYS TYR SEQRES 5 123 LYS GLN ALA LYS LYS LEU ASP SER CYS LYS VAL LEU VAL SEQRES 6 123 ASP ASN PRO TYR THR ASN ASN TYR SER TYR SER CYS SER SEQRES 7 123 ASN ASN GLU ILE THR CYS SER SER GLU ASN ASN ALA CYS SEQRES 8 123 GLU ALA PHE ILE CYS ASN CYS ASP ARG ASN ALA ALA ILE SEQRES 9 123 CYS PHE SER LYS VAL PRO TYR ASN LYS GLU HIS LYS ASN SEQRES 10 123 LEU ASP LYS LYS ASN CYS HET CA 124 1 HETNAM CA CALCIUM ION FORMUL 2 CA CA1 2+ FORMUL 3 HOH *134(H2 O1) HELIX 1 1 LEU 2 LYS 12 1 11 HELIX 2 2 PRO 18 ASP 21 1 4 HELIX 3 3 ASP 40 LYS 57 1 18 HELIX 4 4 ASP 59 VAL 63 1 5 HELIX 5 5 ALA 90 LYS 108 1 19 HELIX 6 6 LYS 113 HIS 115 5 3 SHEET 1 A 2 TYR 75 SER 78 0 SHEET 2 A 2 GLU 81 CYS 84 -1 N THR 83 O SER 76 SSBOND 1 CYS 11 CYS 77 …
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
REMARK 3 FIT IN THE HIGHEST RESOLUTION BIN. REMARK 3 TOTAL NUMBER OF BINS USED : 8 REMARK 3 BIN RESOLUTION RANGE HIGH (A) : 1.5 REMARK 3 BIN RESOLUTION RANGE LOW (A) : 1.55 REMARK 3 BIN COMPLETENESS (WORKING+TEST) (%) : 63. REMARK 3 REFLECTIONS IN BIN (WORKING SET) : 1176 REMARK 3 BIN R VALUE (WORKING SET) : 0.340 REMARK 3 BIN FREE R VALUE : 0.352 REMARK 3 BIN FREE R VALUE TEST SET SIZE (%) : 7. REMARK 3 BIN FREE R VALUE TEST SET COUNT : 81 REMARK 3 ESTIMATED ERROR OF BIN FREE R VALUE : NULL REMARK 3 REMARK 3 NUMBER OF NON-HYDROGEN ATOMS USED IN REFINEMENT. REMARK 3 PROTEIN ATOMS : 957 REMARK 3 NUCLEIC ACID ATOMS : 0 REMARK 3 HETEROGEN ATOMS : 1 REMARK 3 SOLVENT ATOMS : 134 REMARK 3 REMARK 3 B VALUES. REMARK 3 FROM WILSON PLOT (A**2) : NULL REMARK 3 MEAN B VALUE (OVERALL, A**2) : NULL REMARK 3 LOW RESOLUTION CUTOFF (A) : NULL REMARK 3 REMARK 3 CROSS-VALIDATED ESTIMATED COORDINATE ERROR.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining toolsATOM 1 N ALA 1 13.830 17.835 32.697 1.00 11.41 ATOM 2 CA ALA 1 12.869 16.725 32.889 1.00 11.31 ATOM 3 C ALA 1 12.106 16.547 31.592 1.00 12.00 ATOM 4 O ALA 1 12.366 17.226 30.614 1.00 11.37 ATOM 5 CB ALA 1 11.891 17.029 34.056 1.00 11.89 ATOM 6 N LEU 2 11.150 15.638 31.585 1.00 13.43 ATOM 7 CA LEU 2 10.392 15.362 30.376 1.00 14.98 ATOM 8 C LEU 2 9.556 16.543 29.879 1.00 14.65 ATOM 9 O LEU 2 9.465 16.764 28.657 1.00 13.62 ATOM 10 CB LEU 2 9.522 14.116 30.561 1.00 15.03 ATOM 11 CG LEU 2 8.919 13.539 29.291 1.00 17.13 ATOM 12 CD1 LEU 2 10.038 13.103 28.360 1.00 17.29 ATOM 13 CD2 LEU 2 8.027 12.361 29.656 1.00 17.65 ATOM 14 N TRP 3 8.960 17.305 30.796 1.00 14.18 ATOM 15 CA TRP 3 8.157 18.443 30.347 1.00 16.10 ATOM 16 C TRP 3 8.998 19.448 29.543 1.00 14.26 ATOM 17 O TRP 3 8.580 19.864 28.472 1.00 14.34 ATOM 18 CB TRP 3 7.359 19.103 31.491 1.00 19.02 ATOM 19 CG TRP 3 8.163 19.810 32.534 1.00 24.63 ATOM 20 CD1 TRP 3 8.699 19.262 33.683 1.00 25.51 ATOM 21 CD2 TRP 3 8.505 21.199 32.555 1.00 27.29 ATOM 22 NE1 TRP 3 9.348 20.230 34.403 1.00 27.56 ATOM 23 CE2 TRP 3 9.253 21.428 33.743 1.00 28.36 ATOM 24 CE3 TRP 3 8.258 22.278 31.686 1.00 27.60 ATOM 25 CZ2 TRP 3 9.754 22.695 34.083 1.00 28.94 ATOM 26 CZ3 TRP 3 8.761 23.542 32.026 1.00 28.78 ATOM 27 CH2 TRP 3 9.503 23.735 33.216 1.00 29.43
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
GDB-MIRROR site machine
3.06 GHz PIV machine
1 GB RD RAM
240 GB Hard Disk
32 MB Graphics Card
Powered by Red Hat 7.3 Linux Operating System
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Current Dictionary is /Pub/Genome
Upto higher level directory
A thaliana/
C elegans/
H sapiens/
MITOCHONDRIA/
P falciparum/
README
S cerevisiae/
Bacteria/
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
*.faa= FASTA Amino Acid file
*.fna= FASTA Nuclei Acid file
*.gbk= GenBank flat file format
*.gbs= GenBank summary file format
*.ptt= ProTein Table
*.tab= Table to assemble genome
*.val= ASN.1 binary format
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
CAMBRIDGE STRUCTURAL DATABASE
• The CAMBRIDGE STRUCTURAL DATABASE• Software for search, Retrieval Display and
Analysis of CSD contents
The CSD records bibliographic, 2D chemical and 3D structural results from crystallographic analysis of organics, organometallics and metal complexes .Both X-Ray and Neutron Diffraction studies are included for small and medium sized compounds containing upto 500 atoms including hydrogens).
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
THREE DBACOMPONENTS
Database Integrity
Database Security
Database Recovery
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DATABASE INTEGRITY
However, certain safety measures can be built into a database to ensure that errors within the system are minimized.
The major issue for the database management is to ensure that the data in the database is accurate, correct, valid and consistent.Any inconsistency between two or more entries that represent the same entity demonstrates the lack of integrity.
Database technology cannot do very much to protect users against data errors made in the outside world before the data has been entered in the system.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DATA RECOVERY
The most common way to achieve this is to dump the contents of the database with the defined frequency on another medium, magnetic tape or optical disk, which is then stored in the same place.
The process of recovery involves restoring the database to a state which is know to be correct following some kind of failure.
The technique of redundancy is used in the sense that it has to be possible to recover the database to its correct state from information available somewhere else in the system.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DATABASE SECURITY
A password and a list of privileges attach to it are most commonly used to control user access rights to database information.
The DBA has to ensure that adequate measures are taken to prevent unauthorized disclosure, alteration or destruction of both the data within the database and the database software itself.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
THREE COMPONENTS OF DATABASE
Retrieval of the data by end users equipped with suitable analysis and display tools.
Development of a database structure that allows the storage and maintenance of the required data.
Data entry, maintenance and management.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DATABASE ADMINISTRATION
Once the data is entered, it has to be maintained and kept upto date.
The database administrator (DBA) is a person or a group of persons responsible for overall control of database systems.
The DBA is usually not only answerable for the design of the database, but also for choice of DBMS used, its implementation and training of all involved in the database running and use.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
knowledge
Patterns
“Cleaned”data
Target data
Data Selection
Preprocessing &
transformation
Data Mining
Interpolationevaluation &validation
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
PROBLEMS WITH THE DATA
Incomplete data
Noisy data
Temporal data
An extremely large amount of data
Non-textual data
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
INCOMPLETE DATA
Some data may be missing (e.g., some fields may be left blank)
Sometimes, the fact that missing data itself is a valuable piece of information.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
NOISY DATA
The field may contain incorrectly entered information.
We do not know how does this affect the certainty factor (or) confidence level of the results.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
TEMPORAL DATA
Since database grow rapidly, how can data be incrementally added to our results.
What effect should this have in the knowledge discovery process
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
AN EXTREMELY LARGE AMOUNT OF DATA
The option is to perform parallel processing, where n processors, each process approximately 1/n’ th of the data in approximately 1/n’ th of the time.
Some datasets can grow significantly over time.
How should such datasets be processed?
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
NON-TEXTUAL DATA
There are many types of data that need to be manipulated, including image data, multimedia data (Video and Sound), spatial data in GIS and user defined data types.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Stand alone machine application
Web Application
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
PERL
Application programming(Standalone machine)
Applet Programming (Web oriented)
Useful for graphics application over the WWW
Very powerful for string manipulation
Uses CGI as the interface
JAVA
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
WHAT IS PERL?
PERL uses sophisticated pattern matching techniques to scan large amounts of data very quickly.Although optimized for scanning text, PERL can also deal with binary data and can make dbm files look associate arrays
PERL is an interpreted language optimized for scanning arbitrary test files, extracting information from those text files
The language is intended to be practical (easy to use, efficient, complete) rather than beautiful (tiny, elegant and minimal).
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
CGI ( Common Gateway Interface)
CGI performs the task of translation, means translates the needs of clients into server requests and then back translates server replies to clients.
Common Gateway interface (CGI), as its name implies, provides a gateway between a user (Client) and command/logic oriented server.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Client CGI Server
Client Java Servlet Server
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
RMI concept is very useful for multitier architecture EXAMPLE
www.hotmail.com www.google.com
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Remote
machine
Server
Client RMI
Software(Search Engine)
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
WEB-Page
Java Server pages(sun micro systems)
Active server pages(Microsoft corporation)
useful for dynamic web pagecreation
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
GRAPHICAL USER INTERFACE
(GUI)The Programmer can quickly design the user interface by drawing and arranging the screen elements rather than writing the raw code
CGI is easily visualizable to users
It is user friendly
Example:
MS-WINDOWS OPERATING SYSTEMS
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
GUI (Graphical User Interface)
Active X(Microsoft corporation)
Java swing(Sun micro systems)
Buttons, boxes and pull down menus (windows based)
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
VB (Visual Basic) Application development languages.
Supports graphics.
Good for standalone applications.
Web programming is not possible.But it is possible to use script languages(vb script or java script) to make it web oriented.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
VC++System & Application
Programming
Almost same as VB
Additional advantage
System side
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
WORLD WIDE WEB (W W W)
The hyper linked documents are known as HTML documents. They are written in a special language called HTML, stands for Hyper Text Markup Language. The HTML is nothing but ASCII text with embedded tags on it.
World Wide Web is the famous and fastest growing Internet function.It is the way of accessing information already on the Internet using the concept of hypertext to link information.Like FTP, any types of digital documents, images, artwork, movies and sounds on the remote computer can be made hyperlinks.The protocol used for accessing such information is HTTP (Hyper Text Transfer Protocol)
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DBMS & RDBMSDBMS: Dbase
MS-AccessMysql-server
FoxPro (partially RDBMS)
RDBMS: SybaseOracleSQL-server
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
DATABASE
a bunch of tables
TABLES
Store numerous rows of information
FIELDS
The little boxes inside a tables
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
The best way to create your own access database is by using, microsoft access.This tool chips with the professional edition of office-87 and enables you to graphically design your own tables and individual field.
Yet another one my-SQL.
An expensive whopper of a database system called SQL server, which is used in corporation that needs to store huge wads of information.
ORACLE, which is another database format.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Typical Web Search
Keywords
Search Engine
Output
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Flat file
Web Browser
W W W
CGI-Program
HTML
HTML
Form O/p (in HTML)
Form O/p (in HTML)
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Packages developed at theBioinformatics Centre
Raman BuildingIndian Institute of Science
Bangalore 560 012
Principal InvestigatorDr. K. Sekar
E-mail [email protected]
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Search Engines 144.16.71.10 / psst Protein Sequence Search Tool 144.16.71.2/bsdd Biomolecules Segment Display Device 144.16.71.10/msgs Motif Search in Genome Sequence 144.16.71.2/ssep Secondary Structural Elements in Protein
Programmers
1. S.Saravanan2. A.Ajmal Khan3. C.K.Rajesh4. T.Kamaraj5. P.Selvarani6. V.Shanthi7. S.Sirajuddin Sheik
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Database with Search facility144.16.71.2/lsdb Lipase Structural Database144.16.71.2/lysdb Lysozyme Structural Database144.16.71.2/asdb 3D-Amylase Database144.16.71.2/gsdb Globin Structural Database
Programmers1. C.K.Rajesh2. T.Kamaraj
3. P.Sundrarajan 4. P.Selvarani
5. V.Shanthi6. A.S.Zahir Hussain7. S.Sirajuddin Sheik
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Software for Structure analysis & manipulation
144.16.71.146/cap Conformation Angles Package144.16.71.146/rp Ramachandran Plot144.16.71.146/wap Water Analysis Package144.16.71.146/sem Symmetry Equivalent Molecules Generator144.16.71.146/pdbgoodies PDBGOODIES144.16.71.10/gpsm Geometrical Parameters for Small
Molecules144.16.71.146/mbd Measurability of Biovoet difference144.16.71.146/dtf Distribution of Temperature Factor
Programmers
1. C.K.Rajesh2. T.Kamaraj3. P.Sundarajan 4. P.Selvarani5. V.Shanthi6. S.Sirajuddin Sheik
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Water Analysis Package (WAP)V.Shanthi, C.K.Rajesh,J.Jayalakshmi,V.G.Vijay & K.SekarJ.APPL.CRYST. (2002) (in the press)
Protein Sequence Search Tool (PSST 1.1)S.Saravanan,A.Ajmul Khan & K.SekarCURR.SCIENCE, (2000) 550 – 552
PDB Goodies – A Web based GUI to manipulate Protein Data Bank filesA.S.Z.Hussain,V.Shanthi,S.S.Sheik,J.Jeyakanthan,P.Selvarani &K.SekarACTA CRYST. (2002), D58, 1385 – 1386
Ramachandran Plot (RP)S.Sheik,P. Sundararajan,A.S.Z Hussain & K.SekarBIOINFORMATICS (2002) (in the press)
Biomolecules Segment Display Device (BSDD)P.Selvarani,V.Shanthi,C.K.Rajesh,S.Saravanan & K.SekarJ.MOL. GRAPHICS & MODELLING (2002) (in press)
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Take Home Message
Datamining is nothing but exploiting the Hidden Trends in your data.
Create your own derived database.
No one tool or set of tools is universally applicable.
Present the data in a useful format such as graph or table.
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Department of BiotechnologyMinistry of Science & Technology
Govt. of IndiaIndia
&
Jai Vigyan National Science FoundationGovt. of India
India
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
Professor M. Vijayan
Professor N. Balakrishnan
Professor S.M. Rao
Professor S. Ramakumar
Colleagues and Friends
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools
IISc
Bioinformatics Centre & Supercomputer Education and Research Centre
Approaches to developing data mining tools