Data management in the TJ-II multi-layer database

Fusion Engineering and Design 48 (2000) 69–75

Data management in the TJ-II multi-layer database

J. Vega a,*, C. Cremy a, E. Sanchez a, A. Portas a, J.A. Fabregas b,R. Herrera b

a Asociacion EURATOM/CIEMAT para Fusion, A6. Complutense, 22, 28040 Madrid, Spainb CIEMAT Computing Centre, A6. Complutense, 22, 28040 Madrid, Spain

Received 1 July 1999; received in revised form 29 November 1999; accepted 29 March 2000

Abstract

The handling of TJ-II experimental data is performed by means of several software modules. These modulesprovide the resources for data capture, data storage and management, data access as well as general-purpose datavisualisation. Here we describe the module related to data storage and management. We begin by introducing thecategories in which data can be classified. Then, we describe the TJ-II data flow through the several file systemsinvolved, before discussing the architecture of the TJ-II database. We review the concept of the ‘discharge file’ andidentify the drawbacks that would result from a direct application of this idea to the TJ-II data. In order to overcomethese drawbacks, we propose alternatives based on our concepts of signal family, user work-group and data priority.Finally, we present a model for signal storage. This model is in accordance with the database architecture andprovides a proper framework for managing the TJ-II experimental data. In the model, the information is organisedin layers and is distributed according to the generality of the information, from the common fields of all signals (firstlayer), passing through the specific records of signal families (second layer) and reaching the particular informationof individual signals (third layer). © 2000 Elsevier Science S.A. All rights reserved.

Keywords: Data management; TJ-II; Multi-layer database; Data acquisition; Discharge file

www.elsevier.com/locate/fusengdes

1. Outline of the TJ-II data acquisition system

The TJ-II fusion device is a low magnetic shearstellarator with four periods, a major radius of 1.5m, average plasma radius 0.2 m, ECRH power600 kW, and nominal toroidal magnetic field of 1T [1].

The TJ-II Data Acquisition System (DAS) [2]provides a distributed environment for remoteprogramming of all computer systems that handle

hardware involved in experiments, i.e. the VXIdigitisation channels [3] and also CAMAC crates[4], control systems for diagnostics and pro-grammable instrumentation. Experimental dataare transferred to a UNIX central computer(DEC Alpha Server 8400) through a multipleLocal Area Network (LAN) topology, includingETHERNET and FDDI.

Dedicated application software has been devel-oped to manage the TJ-II data. Independent soft-ware modules provide the complete set ofresources required for data capture, data storage,data access [5] and data visualisation. The soft-

* Corresponding author. fax: +34-91-3466124.E-mail address: [email protected] (J. Vega).

0920-3796/00/$ - see front matter © 2000 Elsevier Science S.A. All rights reserved.

PII: S0920 -3796 (00 )00133 -2

J. Vega et al. / Fusion Engineering and Design 48 (2000) 69–7570

ware, written in C and C++, is based on aclient/server architecture and it uses the TCP/IPprotocol.

2. TJ-II data flow

There are two ways experimental data can beintegrated into the TJ-II database. ‘On line inte-gration’ corresponds to experimental data whichare transferred to the data capture’s server aftertheir acquisition. It is important to emphasise thatthese data are liable to be processed before storingthem, either in the acquisition systems themselvesor in the central server. Data that come fromautonomous systems, i.e. systems that are notcontrolled by the data capture software module,as well as data that are produced from analysiscodes, can be integrated at any time (‘Off-lineintegration’). While ‘On-line integration’ is carriedout automatically by the data acquisition soft-ware, ‘Off-line integration’ requires an explicituser procedure (script, program or subroutine) tobe executed. The TJ-II data are classified intothree categories independently the way they wereintegrated. On the one hand, there is the so-calledraw data that cannot be altered. On the otherhand, there is the processed data, which can bemodified as often as required, but only the last

version of the data remains in the database. Fi-nally, there is the temporary data category thatcontains data that are available to users for ashort period of time (typically 1 day). The tempo-rary data was included to aid users in their analy-sis tasks and it allows them to generateintermediate results (that can be treated in thesame way as permanent data) and frees storagespace afterwards.

Other requirement for the TJ-II database isdata qualification capability, i.e. the possibility ofsetting data quality attributes according to experi-mentalists’ criteria. Data, initially stored withoutqualification, can be re-qualified at any moment.

The TJ-II data are moved between differentstorage systems with different degrees of securityand access time. The block diagram shown in Fig.1 illustrates the flow that TJ-II data follow be-tween these storage systems. All data to be addedto the TJ-II database (even temporary data) mustbe stored in a RAID 0+1 file system while thereis not backup copy of these data, thus ensuring ahigh level of protection (RAID stands for Redun-dant Array of Independent Disks [9]). When dataare backed up to tape, they are migrated fromRAID 0+1 to a RAID 5 file system, that alsoprovides data protection (via redundancy), but ata much lower cost than mirroring. At present, theRAID 0+1 file system has a total storage capac-ity of 16 Gbytes, which provides an actual storageof 8 Gbytes due to the mirroring factor. The TJ-IIDAS central server also controls two RAID 5 filesystems, where each file system is a five-disk arraywith 50 Gbytes of total space.

However as the disk space will become ex-hausted sooner or later, it becomes necessary tofree storage for new data. This can be accom-plished by erasing data from disk and restoringthem again from the backup copies when re-quired. In order to avoid the use of backup tapesfor file management, and also in order to insurethe most automated way for data migration toand recall from tapes, we have decided to use aworkstation with the Silicon Graphics Data Mi-gration Facility (DMF) [6]. DMF is a pseudoon-line file system. The DMF provides space in afile system by moving files from disks (on-linesystem) to tapes (off-line system). This change isFig. 1. The current TJ-II data flow.

J. Vega et al. / Fusion Engineering and Design 48 (2000) 69–75 71

very transparent because data are moved but thefile i-node remains on-line. In this way, the filesare listed in their original directories althoughphysically they can reside on tape. Input/outputinstructions on migrated files cause recall opera-tions and the files are copied back to the on-linesystem. The time delay due to file restoration isthe only difference between an on-line file and anoff-line one. In the specific application to TJ-IIdata, the on-line DMF system will be a UNIXstandard Network File System (NFS), in order toallow data access from the TJ-II centralcomputer.

As can be seen, this design provides full dataavailability, although access time depends onwhether or not the data reside in disk.

Data restoring from tapes can be necessary forthree reasons. One of them is data analysis, inwhich case, information can be deleted from diskonce computations have been completed. A sec-ond purpose is data addition for old discharges.In this circumstance, information must be movedto RAID 0+1 in order to produce back-upcopies with the new data. At this moment, dataare managed in the same way as data comingfrom new shots. The third condition for datarestoring is data qualification. The treatment ofthis situation is the same as the previous one,because we are really adding new information tothe database.

Although the two RAID 5 file systems of theTJ-II central server provide up to 100 Gbytes ofstorage, this amount may not be sufficient foroptimal operation. This capacity has to providespace for three purposes:1. new data coming from RAID 0+1 after

backup2. restoration of old data from tapes3. RAID 5 data redundancy

The last point occupies a 20% of total storagebecause we use RAID arrays consisting of fivedisks. On the other hand, the medium-term stor-age needs are 23 Gbytes/month (125 Mbytes/dis-charge�50 discharges/day�12 days/month�0.3,where the last factor results from saving a 70% ofstorage with compression.). Considering that TJ-II experimental campaigns are planned to be 3months long, 69 Gbytes are required in order to

store all the data from a given campaign. A firstapproximation predicts that about 10 Gbytes ofstorage will be devoted, to data restoring fromtape, ‘off-line’ data integration and data qualifica-tion. Then, the RAID 5 files systems are able tohold the data of entire three-month campaign.Nevertheless, the TJ-II operation requirementswill establish the reasonable amount of storageneeded to maintain data on-line, i.e. in hard disk.The storage size will not be a stationary quantity,because it will have to fit to the requirements ofthe TJ-II experiments.

Therefore, it would be desirable to providemore magnetic disk in order to ensure ‘on-line’access to data from several TJ-II experimentalcampaigns. An easy, but very high cost solution,would be to add storage to the central server.However, the price of this option compels us tolook for other alternatives. We have contemplatedthe possibility of using personal computer tech-nologies with JBOD (Just a Bunch Of Disks)peripheral. This arrangement offers some veryimportant advantages such as low cost (availabil-ity of equipment and no data redundancy re-quired), scalability (multiple file systems ondifferent computers), efficiency (personal com-puter technologies become faster and faster dayby day), reliability (safe and strong systems) andversatility (systems can be devoted to several pur-poses). The file systems created in this way wouldbe configured as NFS file systems in order to beable to mount all of them from the UNIX centralserver and to manage the information in a verytransparent way.

Fig. 2 shows what would be the TJ-II data flowwith the incorporation of the NFS files systemsbetween the RAID 5 and the DMF storagesystems.

The main guidelines of the design have beendescribed, but other remarks must be pointed out:

a) Data migration towards slower file systemstakes place according to the age of theinformation.

b) Data never return to faster file systems, un-less new information is added (‘off-line integra-tion’ or data qualification). In such circumstances,data are returned to the RAID 0+1 file system inorder to back-up the new data.


Fig. 2. The TJ-II data flow after on-line file systems extension.

solution for the TJ-II database. The main draw-back arises from the file size expected for rawdata, over 125 Mbytes/discharge. This hugeamount of data has compelled us to compress it.A compression technique, based on delta com-pression has been developed [7]. After the 2000first TJ-II discharges, the average compressionfactor has been 70%. This compression rate re-duces the ‘discharge file’ size to 38 Mbytes, whichis still a large quantity. Then, an analysis wasperformed in order to diminish the impact of thefile size in several aspects related to TJ-II dataflow:

a) Transfer rates between file systemsData moving (data migration, data addition or

data qualification) would imply the transfer of 40Mbytes files between the several file systems.

b) Misuse of space in the RAID 0+1 file systemAccording to TJ-II data flow scheme, both the

‘Off-line’ data integration and data qualificationwould require the ‘discharge file’ to be copied tothe RAID 0+1 file system. Although the newinformation may represent B2% of the file size, itwould be necessary to use space for the whole filein the mirrored file system. Obviously, this causesa waste of disk space that must be avoided.Therefore, a very desirable aim would be to limitthe information that is copied to the RAID 0+1array. This can be achieved by splitting the ‘dis-charge file’ into several files in accordance withsome practical criteria. With such a set-up, itwould be feasible to handle small parts of thedata rather than deal with the information fromthe whole shot.

c) Inefficient use of data backupBy design, all information contained in the

RAID 0+1 file system is backed up and movedto RAID 5. Reasoning in the same way than inthe previous point, it follows that the whole ‘dis-charge file’ must be saved when actually only asmall fraction of the file contains new informa-tion. Again, if the ‘discharge file’ could be dividedinto smaller parts, it would be possible to managethe modified components, thus avoiding copyingof untouched data.

c) TJ-II experimental information is neverdeleted. In addition, the data are always dupli-cated using one of three possible states: mirroring,magnetic disk plus backup tape or two differenttapes (backup and DMF).

3. General architecture of the TJ-II database

Typically, in fusion device databases, data areclassified according to shot number. This informa-tion is usually merged in the so-called ‘dischargefile’ [8]. This file contains all data from a dis-charge that are indexed according to signal namesor channel numbers. Generally, the file includestwo kinds of records. On the one hand, there areheader records, where general signal or channelparameters are stored, for example, dimensions,units, resolution and time-base descriptions. Onthe other hand, there are data records, whichprovide the data themselves, i.e. the informationcaptured by diagnostics and translated into digitalform.

The use of a single ‘discharge file’ is generally asuitable choice from a practical point of view.However, a single ‘discharge file’ is not a proper


Therefore, it would be suitable to diminish theimpact of a single ‘discharge file’ by splitting theinformation into several files. For this purpose,classification rules have been defined in accor-dance with practical criteria. The data from adischarge are grouped according to the conceptsof signal family, signal work-group and datapriority.

1) Signal familyAs is usual in fusion devices, the TJ-II dis-

charge data are organised as signals. A signal inthe TJ-II database is the simplest object thatencloses some physical meaning. Depending onthe different ways in which diagnostics can trans-late their measurements, signals can be grouped inseveral families: temporal evolution data, profiles,2-D contours, 3-D surfaces, images and, in gen-eral, any kind of information needed for scientificinterpretation of the TJ-II experiments. The exis-tence of different signal families is the startingpoint for dividing the TJ-II ‘discharge file’ intoseveral smaller files. In this way, there would havea ‘discharge file’ for each signal family. However,as most of signals of a fusion device (95+%)belong to the temporal evolution family addi-tional sorting is required.

2) Work-groupIn fusion experiments, people are organised

into work-groups that involve scientists with acommon interest in studying specific topics orapplying particular diagnostic techniques. Whenusers need access to old data, they mainly demandsignals generated by their own work-group, whilethe demand for access to signals from othergroups is much smaller. Consequently, this empir-ical fact provides a second way of classifying theTJ-II data. They are also sorted according to thework-group responsible for the signals, therebyallowing for the creation of smaller data files, asdesired.

c) Data priorityThe data stored in a discharge can provide

either direct information with some physical sense

(temperature, density, confinement time or en-ergy), or access to raw data, that require a com-prehensive analysis in order to extract physicalproperties from them. The first kind of data gen-erally requires small storage space, have very highaccess frequencies and are very useful to all users.On the contrary, raw data with no direct physicalmeaning can occupy large amounts of storagespace, are infrequently accessed and are normallyrequired for specialised analysis. These contrastsmake it easy to differentiate between both kindsof information when establishing optimal datamigration criteria towards slower file systems. Theraw data with no direct physical meaning can bemoved towards the DMF file system sooner thandata with a direct physical sense. In this way, it ispossible to save storage in the faster file systemsfor signals that are more frequently demandedand accessed by a high number of users. There-fore, it is convenient to define a third index fordata classification. This index can be called ‘prior-ity’, whose meaning is to set priorities for datamoving (in connection with data age) between theseveral file systems defined for the TJ-II databases. Presently, a set of seven priorities can bemanaged for the TJ-II data.

4. A multi-layer model for the TJ-II database

After defining the data flow and general archi-tecture of the TJ-II database, the outstanding taskis to define more precisely the way in whichdata are stored. The division of the data froma discharge according to signal family, work-group and priority has already been estab-lished. However, several considerations must bemade.

Data locationDue to the fact that data can reside in several

locations, it is necessary to know its exact locationat any time. This kind of information mustbe maintained permanently on-line, i.e. with ashort access time, and can never be migrated totape.


On-line informationIn addition to the data location, other elements

must be always available in on-line file systems.Discharge date and time are obvious factors.Also, general information (pulse set-up, timingsystem programming and main objectives of theshot) as well as users’ comments on dischargesbelong to this group. The set of signals acquiredin a discharge have to be available at all times aswell. Finally, signal parameters are also included.The term parameter describes a couple ofelements: a label and a float value. The TJ-IIdatabase system provides users with the capabilityof defining an unlimited number of parameters forthe discharge signals. Experimental results,calibration constants, diagnostic geometricalfactors and input/output quantities of a computerprogram are some of the possible values that userscan include.

Shot files structureThe structure of the discharge files must be

carefully defined. Header records and datarecords must be managed with a high degree offlexibility. With regard to this point, severalfactors must be taken into account:

1. Fast and direct access to recordsInput/output operations must be guided by

direct pointers to the records in order to avoidsurfing within the files looking for specialcharacters that identify records.

2. Storage sa6ingThe use of variable length records can save

storage space, which is the critical point.

3. Addition of signalsSignals can be added to the database at any

moment. The synchronisation of resources, inaddition to the completeness and consistency offiles must be ensured.

4. Processed dataThis type of data can be modified, but only the

last data version must remain in the database.The database must ensure this without wasting

disk space while providing file completeness andconsistency.

5. Temporary dataTemporary data must only exist for a short

time. Automatic processes for data deletion mustalso ensure file completeness and consistency.

6. Data compressionCompression is only applied to data records

because header records do not have a properstructure to use compression techniques, and usu-ally do not require a lot of storage space. Differentcompression methods can be applied to signals atthe data record level according to their character-istics.

7. Data accessInternal structures and functions for data read-

ing/writing must follow the same access scheme,independently of signal families.

According to these considerations, we havedefined a multi-layer model for actual data storage.The block diagram shown in Fig. 3 illustrates thismulti-layer structure. Presently, there are threelayers, although the architecture is sufficiently opento allow new layers to be added when required.Each layer is made up of several files whose namescontain encoded the main indexes according towhich the information is classified. In some files,data are structured as a set of fixed length records.In this case, the registers are either indexed accord-ing to a particular field or directly accessed throughrecord pointers located in other files. However, inother cases it is not suitable to store the informationwith a fixed length record structure, for example therecords of compressed data.

The first layer contains the set of files that mustbe permanently on-line. General information aboutdischarges is classified according to shot number.A particular file for each discharge keeps accountof the signals acquired relative to that shot. Therecords of this file correspond to data structureswhose fields are common to all signals: signal name,family, user and work-group, signal qualifier, datapriority, signal category (raw, process or tempo-rary), pointers to user defined parameters andpointers to second layer structures.


Fig. 3. The multi-layer structure of the TJ-II database.

flow they follow between different storage systems.We summarize some relevant characteristics of theTJ-II database:

1. Addition of signals or data qualification arepossible at any time.

2. The database is almost an on-line system,thank to the use of the DMF software.

3. The experimental data of a discharge arestored in several files, grouping data according topractical criteria. A proper division for the signalsis based on the signal family, user work-group anddata priority.

4. The files are distributed into several layers,from more general content to more specific.

5. The TJ-II multi-layer model provides a veryflexible way of data management:

5.1 Transparent use of multiple data family.5.2 Data of the same kind are kept together.5.3 The most accessed data can be maintainedon-line.5.4 Storing variable length records saves storageand avoids limiting the availability of usefulfeatures such as multiple sampling frequencies ortime history of profiles.5.5 Addition of new families can be carried outeasily without modifying data structures of olddata.

References

[1] C. Alejaldre, et al., Plasma Phys. Control. Fusion 41 (1999)1.

[2] J. Vega, C. Cremy, E. Sanchez, et al., Fusion Eng. Des. 43(1999) 309.

[3] C. Cremy, J. Vega, E. Sanchez, et al., Rev. Sci. Instrum. 70(1999) 513.

[4] C.M. Dulya, C. Cremy, J. Vega, et al., Rev. Sci. Instrum.70 (1999) 517.

[5] J. Vega, E. Sanchez, C. Cremy, et al., Rev. Sci. Instrum. 70(1999) 498.

[6] DMF Administrator Guide SG-2135 2.5.[7] J. Vega, C. Cremy, E. Sanchez, et al., Rev. Sci. Instrum. 67

(1996) 4154.[8] P.C. Van Haren, N.A. Oomens, Fusion Technol. 24 (1993)

391.[9] S.J. Sicola, Digit. Tech. J., Vol. 6, No. 4, Fall 1994.

The second layer stores structured files whoserecords provide signal header data. The file namesare encoded with the shot number, signal family,work-group and priority. The record structure isfamily dependent. Presently, two signal families aredefined for the TJ-II database: temporal evolution(amplitude vs. time) and profiles (any magnitude vs.another one).

The third layer is made up of files with a variablerecord length. Each record provides informationabout an individual signal. A record can be com-posed of either an unstructured string of bits or avariable number of data structures of the samekind. Access is achieved by keeping both the fileoffset, where the record begins, and its length in thesignal data header. These records store either thedigital form of the data or the particular structuresthat allow the storage of an unlimited number ofsignal characteristics along the discharge: multiplesampling frequencies or time history of profiles.

The second and third layer files are the ones thatfollow the TJ-II data flow shown in Figs. 1 and 2.

5. Summary

We have described the way TJ-II data are or-ganised and stored in the database as well as the

Documents

Data management in the TJ-II multi-layer database