EDAMIS specialised user guide EDAMIS Web Forms for data ... · EDAMIS EDAMIS specialised user guide – EDAMIS Web Forms for data providers QUG-03-2014-07-30 [QUG] (EDAMIS Web Forms

EDAMIS specialised user guide

EDAMIS Web Forms for data providers

Project EDAMIS Author Eurostat Unit B3, IT for statistical production Document release 1.3 EDAMIS release 3.3 Status Released Reference QUG-03-2014-07-30 [QUG] (EDAMIS Web Forms for data providers) F.doc

EDAMIS EDAMIS specialised user guide – EDAMIS Web Forms for data providers

QUG-03-2014-07-30 [QUG] (EDAMIS Web Forms for data providers) F.doc

TABLE OF CONTENTS

1 INTRODUCTION ........................................................................................................................................ 1

1.1 BACKGROUND INFORMATION ................................................................................................................... 1

2 ACCESSING THE EDAMIS WEB PORTAL .......................................................................................... 2

3 INSTANTIATING WEB FORMS .............................................................................................................. 9

4 WEB FORM FEATURES ..........................................................................................................................15

4.1 HEADER AREA .........................................................................................................................................15

4.2 DATA ENTRY AREA ..................................................................................................................................16 4.3 FOOTNOTE AREA .....................................................................................................................................17

4.4 TABS .......................................................................................................................................................18

4.5 THE CONTROL BAR ..................................................................................................................................18 4.6 VALIDATION WINDOW .............................................................................................................................21 4.7 TASK BUTTONS ........................................................................................................................................22

4.8 COMMENT BOX ........................................................................................................................................23

5 ENTERING DATA IN WEB FORMS ......................................................................................................24

5.1 KEYING DATA ..........................................................................................................................................24

5.2 VALIDATION RULES .................................................................................................................................25 5.3 DELETING DATA ......................................................................................................................................28

5.4 COPY AND PASTE .....................................................................................................................................28

5.5 SPECIAL VALUES AND FLAGS ...................................................................................................................29 5.6 CONFIDENTIAL DATA ...............................................................................................................................31 5.7 SPECIAL VALUES AND VALIDATION RULES ..............................................................................................31

5.8 M ISSING DATA .........................................................................................................................................31

5.9 ZERO BY DEFAULT ...................................................................................................................................31

6 EXPORTING DATA ..................................................................................................................................32

7 IMPORTING DATA ..................................................................................................................................35

8 MAKING AN OFFICIAL TRANSFER ....................................................................................................39

9 OPENING EXISTING WEBFORMS .......................................................................................................41

9.1 LOCKING OF WEB FORMS ........................................................................................................................41

10 MAKING REVISIONS TO DATA .....................................................................................................43

11 GLOSSARY ..........................................................................................................................................45



TABLE OF FIGURES

Figure 1: EDAMIS Home Page ................................................................................................ 2 Figure 2: Self Registration Form ............................................................................................... 2 Figure 3: Welcome Page ........................................................................................................... 3 Figure 4: Web Form Entry Menu .............................................................................................. 4 Figure 5: Web Form Entry Filter............................................................................................... 4 Figure 6: Web Form Entry - Dataset filter ................................................................................ 4 Figure 7: Web Form Entry - Year filter .................................................................................... 5 Figure 8: Web Form Entry - Filter results ................................................................................. 5 Figure 9: Web Form Entry - All datasets .................................................................................. 7 Figure 10: Web Form Entry - Filter changed ............................................................................ 7 Figure 11: Web Form Entry table ............................................................................................. 8 Figure 12: Web Form Entry - Quick filter ................................................................................ 8 Figure 13: Creation of a new Web Form Entry ......................................................................... 9 Figure 14: Web Form Entry Creation Form ............................................................................ 10 Figure 15: Web Form Entry - Period ...................................................................................... 10 Figure 16: Web Form Entry: Language .................................................................................. 11 Figure 17: Web Form Entry - Open a Template ..................................................................... 12 Figure 18: Displaying a Web Form Entry ............................................................................... 12 Figure 19: Security Warning ................................................................................................... 13 Figure 20: Run of the Java Applet .......................................................................................... 13 Figure 21: Second Security Warning ...................................................................................... 14 Figure 22: Web Form Entry Page ........................................................................................... 15 Figure 23: Header area ............................................................................................................ 15

Figure 24: Data Entry area ...................................................................................................... 16

Figure 25: Lock Headers Menu ............................................................................................... 16 Figure 26: Header area hidden ................................................................................................ 17 Figure 27: Footnote area ......................................................................................................... 17

Figure 28: Web Form Entry Tabs ........................................................................................... 18 Figure 29: Control Bar ............................................................................................................ 18

Figure 30: Code added to the Description ............................................................................... 19 Figure 31: Header and Footer hidden ...................................................................................... 19 Figure 32: Hidden row displayed ............................................................................................ 20 Figure 33: Hidden row ............................................................................................................ 20

Figure 34: Hide row menu ...................................................................................................... 20 Figure 35: Validation rules ...................................................................................................... 21

Figure 36: Task buttons ........................................................................................................... 22

Figure 37: Comment button .................................................................................................... 23 Figure 38: Comment Window ................................................................................................. 23 Figure 39: Selected cell ........................................................................................................... 24

Figure 40: "Enter" key action .................................................................................................. 24 Figure 41: Validation rule not satisfied ................................................................................... 25 Figure 42: All satisfied validation rules .................................................................................. 26 Figure 43: "Official Transfer" button disabled........................................................................ 26 Figure 44: Validation rule - Justification needed .................................................................... 27 Figure 45: Validation rule - Justification added ...................................................................... 27 Figure 46: Copy of values ....................................................................................................... 28

Figure 47: Paste of values ....................................................................................................... 29

Figure 48: Values copied and pasted ....................................................................................... 29 Figure 49: Validation of a value .............................................................................................. 30 Figure 50: Flag values ............................................................................................................. 30



Figure 51: Exporting Data ....................................................................................................... 32

Figure 52: Name of the exported data file ............................................................................... 33 Figure 53: Importing Data ....................................................................................................... 35

Figure 54: Data imported ........................................................................................................ 38

Figure 55: Official Transfer .................................................................................................... 39

Figure 56: Confirmation message before Official transfer ..................................................... 39 Figure 57: Official transfer done ............................................................................................. 39 Figure 58: Transferred flag indicator ...................................................................................... 40 Figure 59: Locking Web Forms .............................................................................................. 41 Figure 60: Locked Web Form Indicator .................................................................................. 42 Figure 61: Existing Web Form Entry ...................................................................................... 43 Figure 62: Different versions of a Web Form ......................................................................... 44


QUG-03-2014-07-30 [QUG] (EDAMIS Web Forms for data providers) F.doc 1/45

1 INTRODUCTION This manual was created for data senders using EDAMIS Web Forms. It contains everything a data sender needs to know in order to use EDAMIS Web Forms. It encompasses basic information on the EDAMIS Web Portal, including how to register in order to send data. It illustrates how to access Web Forms, open Web Forms, enter data in Web Forms and make an official data transfer. It also includes additional information on monitoring data transfers that have taken place.

Whilst this manual is specifically intended for data senders, it is also applicable to data preparers with the exception of making an official transfer.

1.1 Background information

EDAMIS : the Electronic Data file Administration and Management Information System is the integrated environment for data transmission and the Single Entry Point for data arriving at Eurostat. It can be accessed through the following tools:

EWA: the EDAMIS Web Application is installed in the National Statistical Institutes and used to send statistical data files to Eurostat. Interactive, semi-automated and fully-automated transmissions are supported.

EWP: the EDAMIS Web Portal is installed at the European Commission and used to manage the dataset inventory, to manage user rights related to transmissions, to send statistical data files to Eurostat (as an alternative to EWA) and to monitor the traffic through its Management Information System.

EDAMIS Web Forms is a component of the EDAMIS Web Portal and can be used to transmit data of low volume. It is built above the basic functions of EWP and does not require any installation on the user’s PC.

EDAMIS Web Forms is typically of interest for the statistical domains where

• data volume is low;

• data collection is carried out via spreadsheet-based questionnaires;

• certain level of data validation before transmission is preferred;

• software installation on client PCs is forbidden;

• a data-sharing agreement with international organisations exists.

Local Coordinators: in each National Statistical Institute (NSI), Local Coordinators (LC) are the main contact point to Eurostat for the effective implementation of electronic data transmission. A Local Coordinator is the national contact point for data transmission issues.

Transmission Coordinators: inside EDAMIS, each data providing organisations can have a Transmission Coordinators (TCO) responsible for the management of users' rights within EDAMIS. The TCOs are in charge of managing and giving rights to users within their organisation. In NSIs the TCO is usually the Local Coordinator. A TCO can be allowed to coordinate users in several organisations of the country, provided that an agreement exists between the organisations concerned.

Domain Manager is responsible for a statistical domain (in Eurostat), taking care of setting up and monitoring the data collection activities in EDAMIS, while being in permanent contact with the data sender organisations.



2 ACCESSING THE EDAMIS WEB PORTAL To access the EDAMIS Web Portal, type in the following URL into the address bar of your web browser and press [ENTER]:

https://webgate.ec.europa.eu/EDAMIS The site appears (see Figure 1):

Figure 1: EDAMIS Home Page

Click on the [ECAS authentication] button; you will be redirected to the ECAS login page.

For the purpose of this guide, it is assumed that the reader (EDAMIS Data Provider) has an ECAS user account in the domain of “External”. If you are not familiar with ECAS, please contact the EDAMIS support for further instructions.

Enter your ECAS username (or email address) and password, click on the [Login] button and after a successful authentication, you are back on the EDAMIS Interface. Different forms can be displayed: � Self-registration

Note If you are not a registered EDAMIS user already, you will have to register, before you can proceed any further, using the automatically displayed Self Registration Page (Figure 2). During registration, you select the datasets, for which you will be sending data, and your role (data sender, in this case).

Figure 2: Self Registration Form



� EDAMIS Welcome page

Note If the user is already registered in EDAMIS, the Welcome Page is displayed (Figure 3):

Figure 3: Welcome Page

If more information is needed: � EDAMIS Help Centre and Support

Note A set of comprehensive documents is available at the EDAMIS Help Centre. These documents cover a wide range of topics, such as the available software, including a number of very technical aspects, advice on choosing the best method to suit your requirements and help with the installation procedures. You can access the EDAMIS Help Centre through the following link:

https://webgate.ec.europa.eu/EDAMIS/helpcenter/website/index.htm The support team will be happy to help if you have any questions or should you want to report any incidents or need to resolve any problem encountered when using the data collection tools, do not hesitate to contact the support team for any problems or questions related to data collection.

Mail to: [email protected]

Tel: (+352) 430133213



The Welcome page presents you a summary table of recent data transmissions via the Web Portal, a number of user menu options (transmission, Reports, Preferences and Logout) and some links (e.g. the EDAMIS Help Centre). Information about these features is provided in the user guides available in the EDAMIS Help Centre. In case of the Web Forms, the only important feature now is the Transmission menu.

Select Transmission in the navigation bar on the left and select Web Form Entry from the drop-down list (see Figure 4).

Figure 4: Web Form Entry Menu

The Web Form Entry page now appears. By default, the Grouped Dataset drop-down box is set to “--- all dataset groups ---”, while the Dataset drop-down box is set to “--- all datasets ---” (see Figure 5).

Figure 5: Web Form Entry Filter

For practical reasons the DMA can divide the data into several datasets. These datasets are grouped and only accessible after selecting the corresponding group in the above-mentioned Grouped Dataset drop-down box.

Generally, search for a specific Web Form on the Web Form Entry table using the Dataset drop-down box (see Figure 6). The Web Form Entry table presents a list of all the datasets, for which you have data sender rights.

Figure 6: Web Form Entry - Dataset filter

Select Web Form Entry



It is possible to filter the Web Form Entry table to show only the Web Forms associated with the selected dataset (see Figure 7) and year.

Figure 7: Web Form Entry - Year filter

Once a dataset is selected in the Web Form Entry table, you can click on [View] to display a list of all Web Forms already prepared or transmitted by your organisation for the selected dataset (see Figure 8) and the selected year.

Figure 8: Web Form Entry - Filter results



The datasets in the drop-down list are grouped by domain and sorted alphabetically. The Web Forms in the Web Form Entry table are sorted by descending order according to their modification date. Each row of information corresponds to a single Web Form instance. The columns contain the following information:

Dataset The name of the dataset in accordance with the EDAMIS naming convention.

Code The full description of the dataset.

Period The reference period. For annual datasets, it is the reference year followed by an "A" (e.g. "2007A" for year 2007). For monthly datasets, it is the reference year followed by the two-digit representation of the reference month (from "01" to "12", e.g. "200709" for September 2007).

Lock A small red lock symbol appears if the Web Form is locked (see section 9).

Modified by The EDAMIS user-ID of the last person who worked on the Web Form.

Modified on The date of the last modification of the Web Form (i.e. created or edited).

Status Draft The Web Form instance has not been transmitted to Eurostat yet.

Estimated A Web Form instance, containing estimated data for the reference period, has been created by Eurostat.

Transferred The Web Form instance has been officially transmitted to Eurostat.

Approved The Web Form instance has been officially approved by Eurostat (note that this explicit approval is no longer used by Eurostat, so only the very first Web Form instances are likely to have this status).

Rejected The Web Form instance has been rejected by Eurostat. A new version of the data will have to be sent.

Country The ISO country code of the country that sent the Web Form instance.

Version The version of the Web Form Instance.

Download List of the full name of the .xml file(s) used by a Web Form. There is one .xml file for each table in the Web Form.

Active The status of the corresponding Web Form template. When the active flag is set to ‘Y’ (Yes), the corresponding Web Form template is the one currently used by the dataset.



It is possible to see all Web Forms for a selected year on the Web Form Entry table using the Dataset drop-down box (see Figure 9) and selecting “ --- all datasets ---“.

Figure 9: Web Form Entry - All datasets

Once another dataset is selected in the drop-down box list, the message “Selection criteria changed, please click on view button to refresh” will appear under [View] in the Web Form Entry. Click on [View] to refresh the Web Form Entry table (see Figure 10).

Figure 10: Web Form Entry - Filter changed

In most cases, the list of available Web Forms will not fit on a single page. You can navigate forward and backward by using the browser’s horizontal and vertical scrollbars.

You can shorten the Web Forms Entry list by using the quick filters available for some columns: Period, Modified by, Modified on, Status, Country and Active columns (see Figure 11).

Web Form Entry table needs to be refreshed after a dataset has been selected.



Figure 11: Web Form Entry table

In the example above, the Modified On Date “05/10/2011” was selected (see Figure 12).

Figure 12: Web Form Entry - Quick filter

Several filters can be applied simultaneously and they operate with “AND” logic.



3 INSTANTIATING WEB FORMS When a data sender wants to send data for a new reference period, a new Web Form instance needs to be created for that reference period. This process is called instantiation.

To instantiate a Web Form, first select the dataset from the drop-down list in the Web Form Entry table (e.g. COSAEA_ALI3_A "Economic Accounts for Agriculture - Labour Input - Final - Annual" – see Figure 13), then click on the [Insert] button below the table.

Figure 13: Creation of a new Web Form Entry

� When not to instantiate

Note There are some circumstances, under which there is no need to instantiate a new Web Form. A draft Web Form for the reference period may already exist: it might have been created either by another data sender in the organisation or, in exceptional cases, by Eurostat upon prior arrangement. Moreover, for some datasets, Eurostat may initially provide data estimates for the data sender. In such cases, Web Forms with Estimated status have already been created by Eurostat and must be completed by the data sender for the reference period in question. In such cases there is no need to instantiate a new draft.



Click on the [Insert] button to open the Web Form Entry – New Form (see Figure 14):

Figure 14: Web Form Entry Creation Form

In the New Form window, the Web Form template field indicates the current Web Form template used by Eurostat for the dataset selected (COSAEA_ALI3_A in the example above). This is for information purposes only. It is predefined by Eurostat and cannot be changed by the data sender.

In the Country field, the data sender's country is shown (LU in the example above). This is also predefined according to the information provided upon registration in EDAMIS and cannot be changed.

Below this, two fields appear side by side in order to define the Reference period. The first field allows the data sender to select the instantiation year. Click on the drop-down arrow in the first reference period field to display the available years (see Figure 15) and select the one.

Figure 15: Web Form Entry - Period



The second field for the reference period allows the data sender to select the frequency (e.g. Annual, Monthly or Quarterly). The list-box is populated with values corresponding to the chosen dataset. In the previous example, only the value “Annual” exists for the dataset.

In case of frequencies shorter than a year, the user can use another list-box to select which period to instantiate.

The language can also be configured. Click on the Available languages field to display the options (see Figure 16). For this Web Form three languages are available. Click on a language to select it (English in the example below).

Figure 16: Web Form Entry: Language

� Languages

Note Although EDAMIS Web Forms can be displayed in several languages, this varies from one domain to another. English is used for most datasets but French and German are also regularly completed.



Having selected the reference period and language, click on the [Accept] button (see Figure 17).

Figure 17: Web Form Entry - Open a Template

The downloading process of the Java applet starts automatically. This Java applet creates the Web Form's structure, formatting and content (including previously-transmitted data, where applicable – see Figure 18).

Figure 18: Displaying a Web Form Entry

� Organisation's IT security settings

Note Depending on your organisation's IT security settings and the Java version, you will be asked to verify the access a potentially insecure website. Click on [Run] (or use the [Enter] key).



Figure 19: Security Warning

As soon as you accept to run the application, the Java Applet starts running and the Java symbol appears (see Figure 20).

Figure 20: Run of the Java Applet

For security reasons, you will be asked whether to block the application (see Figure 21). To continue, click on the [Don’t Block] button. After a short loading time, the Web Form opens and is ready for use.



Figure 21: Second Security Warning

� Java Applet running time

Note The first time a Java Applet is run on a PC, it may take some time to open (20-30 seconds is typical, but it can be longer in some organisations, depending on the IT environment). Some additional security clearance may also be requested (select [Run] to proceed). Once the Java Applet has been run for the first time and the initial software downloaded, the Web Form will open much quicker.



4 WEB FORM FEATURES When a Web Form draft is opened for the first time, a Web Form window will open as shown below (Figure 22). The Web Form window consists of seven distinct parts. These are the header area, the data entry area, the footnote area, tabs, the control bar, the validation window, the task buttons and the comment box. Each of these will be discussed in turn.

Figure 22: Web Form Entry Page

Continuing with the same example, the Web Form above is a draft Web Form instantiated for the annual dataset COSAEA_ALI3_A "Economic Accounts for Agriculture - Labour Input - Final - Annual" by a Luxembourg data sender for the year 2013.

4.1 Header area

The header area (see Figure 23) contains all the applicable reference information for the Web Form instance opened. It typically gives the full dataset description plus other relevant information (e.g. the data unit in this case) in titular format. Below this the dataset name is given on the left hand side. The country code comes next (LU for Luxembourg in this example) – this is always the country of the data sender. This is followed by the reference period (the year followed by "_A" for annual data, a number between 01 and 12 for monthly data or a number between 01 and 04 for quarterly data). Finally, the status (Draft in this case) is shown.

Figure 23: Header area



4.2 Data entry area

The data entry area has a spreadsheet-like appearance (see Figure 24). It consists of a fixed number of data entry cells, each defined by its row heading and column heading. Each cell has a reference, defined by the number of its row and the letter of its column (e.g. in this example, the yellow cell with the value 2.961 has the cell coordinates “B2” ).

Figure 24: Data Entry area

In this example, column A is a nomenclature column. All the various characteristics captured in the Economic Accounts for Agriculture - Labour Input are defined here. Row 0 shows the relevant reference periods. Four reference periods are defined from 2010 to 2013. Each reference period has (in this example) two columns associated with it, a yellow column and a red column. The yellow column is for numeric data (plus a limited number of allowed special characters). The red column is for flag information. The numeric data for 2010, 2011 and 2012 was pre-filled automatically (Note: this in itself is a big advantage over the classic Excel-based data transmissions, where, in most cases, this previous survey data was not directly available for comparison).

In the following example, the Web Form consists of more than 8 rows of information. It extends beyond the height of the data entry area. Therefore, a vertical scrollbar is available on the right hand side to allow the data sender to bring further rows into view. All the columns of this Web Form fit within the width of the data entry area, thus there is no need for a horizontal scrollbar. If this were not the case, a horizontal scrollbar would also appear at the bottom of the data entry area.

There is an option for freezing row and column headings. By right-clicking in the data entry area, a contextual menu will appear (see Figure 25). In this menu, selecting [Lock Headers] will freeze row and column headings.

Figure 25: Lock Headers Menu

Once row and column headings are frozen, only data entry cells will be horizontally or vertically scrolled (see Figure 26).

Contextual menu used to “Lock Headers”.



Figure 26: Header area hidden

4.3 Footnote area

The footnote area contains supplementary information about the Web Form (see Figure 27). It contains typically two separate pieces of information: a “special values” key and a “flags” key (it may also contain additional information Eurostat finds relevant to display on the Web Form).

For this Web Form, three special values are defined (whilst these are not displayed in the screenshot, they can be viewed using the vertical scrollbar on the right hand side of the footnotes area). The special values in this case are "C" for confidential, "NA" for not currently available and "-" for not applicable or zero by default. These special values are the only character values that may be entered in a (non-flag) data entry cell (the yellow cells).

Figure 27: Footnote area

For this Web Form, nine flags are defined (although they cannot be seen in the screenshot). These are "p" for provisional data, "f" for final data, "c" for confidential data, "r" for revised data, "e" for estimated data, "u" for unreliable data and "1", "2" and "3" to be defined by the data sender, as the need arises (an explanation of these user-defined flags can be given in the comment box, see below). Only the predefined flag characters in the footnote can be entered in flag cells.

� No previous reference period data displayed?

Note If Web Forms are used for the first time in order to transmit data for a particular dataset, it may well be the case that no data is displayed for the previous reference periods. This is because there is no record of a previous transmission in EDAMIS, from which it could download the requested data (EDAMIS cannot extract data directly from Eurostat's production databases).

Eurostat will endeavour to conduct some pre-transmissions of the data, solely for the purposes of loading this historical data into EDAMIS. However, given the amount of data involved, this is only likely to be possible in a limited number of cases.

Column headings has been frozen

Row headings has been frozen



4.4 Tabs

Tabs define the various tables that may exist in a Web Form. In the above example, there is only one table in the Web Form (see Figure 28). This was defined as table COSAEA_ALI_FINAL. In other Web Forms, two or more tables may be defined. In such cases, each table is represented by a tab. A data sender can toggle from table to table by clicking on the respective tabs.

Figure 28: Web Form Entry Tabs

4.5 The control bar

The control bar consists of six separate components: two information fields and four function buttons (see Figure 29). The first information field on the left is the reference of the currently selected cell (in the above example, cell D1 is selected). The second field shows the mathematical functions that are defined for the selected cell. (In this example, and throughout the whole guide, no functions are defined.) Functions may, for example, specify that the value of a cell is automatically computed as the sum of other cells.

Figure 29: Control Bar

The first function button is a toggle allowing the data sender to display the applicable

codes in the nomenclature descriptions (column A in the above example). Clicking on

allows to display codes, plus descriptions, and changes the button notation to (see

Figure 30). Clicking on removes the codes again and changes the button notation back

to .



Figure 30: Code added to the Description

The second button is a toggle allowing the data sender to hide the header and footer area.

It expands the visible data entry area (see Figure 31). Once selected, the becomes .

Clicking on will show the header and footer areas again (and change the button back to

).

Figure 31: Header and Footer hidden

The third button is a toggle allowing the data sender to lock/unlock rows and columns headings.

The fourth button is a toggle which allows the data sender to hide hidden rows, as shown in Figure 32 and Figure 33. When clicking on this button, it is replaced by the

button . Selecting this new button allows to display hidden rows.



Figure 32: Hidden row displayed

Figure 33: Hidden row

The function to hide some rows can be accessed using the right click on a row and select “Hide row” in the contextual menu (See Figure 34).

Figure 34: Hide row menu



4.6 Validation window

The validation window displays validation rules for the Web Form. Each row in the validation window corresponds to a single validation rule (see Figure 35). Only validation rules that are currently broken (= not satisfied) appear in the validation window.

Figure 35: Validation rules

Each validation rule begins with a reference. This is the reference of both the table and cell, to which the validation rule applies. In the above example, the four (currently broken) validation rules from the second one to the fifth one have the reference COSAEA_ALI_FINAL!H1, which means that they apply to cell H1 of Table COSAEA_ALI_FINAL (Volume Of Agricultural Labour Input). In principle, there is no limit to the number of validation rules that can apply to a given cell.

After the validation rule reference, the validation rule itself is given. Validation rules come in three different types. There are mandatory rules ("Field is mandatory"), specifying that a given cell must be filled in by the data sender (the second validation rule, in the example above, is a mandatory rule). There are range checks, specifying that the value entered in the referenced cell must be within a predefined range. On the other hand, there are expressions. An example of an expression rule is that one cell must be equal to the sum of two other cells.

After the actual validation rule, the level of the rule is given. Two levels are possible: the basic rules and the critical rules. Basic rules may be broken by the data sender. However, the data sender is required to provide justification before being allowed to officially transmit data. Critical rules must always be obeyed before the official transmission is allowed. Basic rules are highlighted in blue, critical rules in red.

Finally, there is a justification field. Here, for each basic rule broken, the data sender must give a brief account of why the rule should not apply in this case. These justifications will be read by the responsible Eurostat domain manager, who may decide to approve the data or reject it on the basis of the justifications given. The data sender has to click on the field to start editing it. Justification cannot be given to broken critical rules.

The validation window also has a vertical and a horizontal scrollbar to allow navigation across the rules.



4.7 Task buttons

There are nine different task buttons (see Figure 36).

Figure 36: Task buttons

They have the following functions:

Save The button allows a data sender to save a Web Form in draft status at any time. This generates the message "Form saved in Web Form server local repository" and stores the Web Form on the EDAMIS server at Eurostat.

Official Transfer The button starts the data transmission to Eurostat. Clicking on the official transfer button is the final step for the data sender, when all the data has been entered in the Web Form instance (see section 8 for more information). Note that this button is greyed-out in the screenshot above. This is because the official transfer option is disabled until all validation rules are either met or justified (which is not the case above). Once this is the case, the official transfer button will be enabled and the data sender can then proceed with the transmission.

Print The button allows printing the Web Form instance as it is displayed.

Import The button allows the bulk import of data into the Web Form, either in .CSV or .XML format. See also section 7.

Export The button allows the bulk export of data from the Web Form, either in .PDF, .RTF, .XML, .CSV, TEXT or HTML format. The Web Form validation rules can also be exported in .CSV format. See also section 6.

Export DB The button was originally intended to allow the data sender to export the Web Form data to a predefined production database. In the current version of EDAMIS Web Forms, this option is disabled.

Exit The button allows the data sender to exit the application at any time. Once clicked, a dialogue box will appear asking the data sender to confirm that he wishes to exit.

Explanatory Notes File The button allows the data sender to add some

notes to the Web Form by linking it to a text file.



No Explanatory Notes File The button is disabled, when the Web Form is not linked to

a Notes file. In case the template is linked to an Explanatory Notes File, the label of the button contains the name of the Explanatory Notes File. Clicking on this button opens the file.

4.8 Comment box

The comment box (see Figure 37) allows the data sender to write a comment on the data. This comment remains with the Web Form instance and the text can be read by anyone with access to the Web Form. This can be useful for notifying Eurostat of any unusual features of the data contained in this survey instance.

Figure 37: Comment button

To write a comment, click on . This expands the comment box and allows the data sender to enter text (see Figure 38). Simply click in the comment box to start writing. Horizontal and vertical scrollbars will appear if the comment extends beyond the right hand side or the bottom of the comment box. When finished with the comment’s writing, simply click on [OK] and the comment box will collapse back to its former size in the Web Form.

Figure 38: Comment Window



5 ENTERING DATA IN WEB FORMS

5.1 Keying data

Data is entered manually in Web Forms much in the same way as it is entered in an Excel spreadsheet. First, click on the cell, in which you wish to enter data. Take cell H1 for example in the COSAEA_ALI3_A "Volume of Agricultural Labour Input" Web Form (see Figure 39). The cell currently selected is highlighted with a different background colour than the other cells.

Figure 39: Selected cell

Once a cell is selected, key in the desired value and press return (see Figure 40). Please note that the decimal separator is always a point ".". Comma decimal separators "," are not accepted. Pressing [Enter] completes the data entry for this cell and automatically selects the next cell below (D2 in this case) allowing to enter the next value.

Figure 40: "Enter" key action

� Decimal places

Note The number of decimal places for the data is fixed for each Web Form. In the example above, the data is given to two decimal places. As in a formatted Excel worksheet, if less than two digits are given after the decimal separator, zeroes are automatically inserted to make up the two decimal places. If more than two digits are given after the decimal separator, then the entry is rounded automatically to two decimal places.



5.2 Validation rules

As soon as a value is entered for a particular cell, it has immediate consequences for the validation rules applicable to that cell ("validation on the fly"). There are two types of error status when a validation rule is not fulfilled: “critical” or “basic”. If the error status of the broken rule is “critical”, the official transfer cannot be carried out. When the error status of the broken rule is “basic”, the data sender must provide a justification before carrying out the official transfer. In the example above, as soon as the value 1 was entered in cell H1, two validation rules that were initially displayed in the validation window for that cell (as they were initially broken) disappeared. These were:

"Field is mandatory rule"

(which disappears as soon as the data is entered in the cell) and:

“out range +3% year n-1”

(which is now satisfied). See Figure 41.

Figure 41: Validation rule not satisfied

Once all validation rules are satisfied, the validation window will disappear and the [Official Transfer] button will be enabled (see Figure 42).



Figure 42: All satisfied validation rules

If at least one basic rule remains broken, the validation window will not disappear. However, the [Official Transfer] button will be enabled once justification has been provided in the corresponding cell for each broken basic rule (see Figure 43, Figure 44 and Figure 45).

Figure 43: "Official Transfer" button disabled



Figure 44: Validation rule - Justification needed

Figure 45: Validation rule - Justification added

� Validation rules and future periods

Note Some Web Forms have data columns for future reference periods (e.g. Monthly Web Forms always have columns for January till December, irrespective of the current survey instance. Thus, a draft Web Form for February, for example, will have blank columns for March to December as well). Validation rules are automatically disabled for the future reference periods in such Web Forms, so that null data can remain in these data columns without any adverse effect on data transmission.



5.3 Deleting data

Data can be deleted from cells in two different ways. The first way is to double-click on the cell. This will move the cursor to the end of the entered value and then the backspace key may be used to delete the characters. An alternative and often easier method is to select a cell by clicking on it once and delete the whole content by pressing the [Delete] key. It is possible to select a group of adjacent cells by holding down the [CTRL] key, clicking on the first cell and dragging through the remaining cells, and then do a group delete.

5.4 Copy and paste

It is possible to copy and paste data within a Web Form using [CTRL]-[C] and [CTRL]-[V] . Simply select the cell or group of cells you wish to copy, hold down simultaneously the [CTRL] and [C] keys to copy, then select the cell or group of cells you are willing to paste and hold down simultaneously the [CTRL] and [V] keys to paste. It is also possible to copy and paste from an external file (Excel, for example), using [CTRL]-[C] and [CTRL]-[V].

Another way of copying and pasting data is using the contextual menu by right-clicking the selected cell or group of cells and choosing the “Copy” function (see Figure 46).

Figure 46: Copy of values

Then, move to the cell where data must be copied, right-click to open the contextual menu and use the “Paste” function (see Figure 47).



Figure 47: Paste of values

Once the cell or group of cells were copied, they will appear in the form (see Figure 48).

Figure 48: Values copied and pasted

5.5 Special values and flags

The only entries acceptable in the data entry cells are numeric values (including the "." decimal separator) and those character values predefined as Special values. If a data sender enters any forbidden character (e.g. a "%" symbol), then an error message appears (see Figure 49).



Figure 49: Validation of a value

Flags can only be entered in the flag column. To enter a flag, click on the appropriate flag cell. This immediately opens a drop-down list showing the various flags predefined for this Web Form (see Figure 50). The meaning of these flags can be found in the flag key in the footer area. To select a particular flag, simply click on it. Only one flag can be assigned to each flag cell. To remove a flag, click on the flag cell again and click on the empty flag value (the last in the drop-down list).

The flag column is visible only when defined by the Eurostat domain manager.

Figure 50: Flag values



5.6 Confidential data

There are two ways of indicating confidentiality in a Web Form. The preferred way (from Eurostat's perspective) is to provide the data and flag each affected cell with "c". Such data will neither be published, nor forwarded to other parties by Eurostat. Member States may not be willing to provide confidential data even to Eurostat. In such cases, the special value "C" can be entered in the cells, and then there is no need to flag these cells with "c".

5.7 Special values and validation rules

Special values can be problematic for some validation rules. For example, in the validation rule for D2:

"THIS<=(D3+D4+0.2)"

If the special value ":" is entered in cell D3, then the validation rule is broken, as ":" is not a numeric value and cannot be computed in the expression. For this reason, most of the validation expressions are basic rules, i.e. they can be broken (with a justification).

5.8 Missing data

Data, which is missing or not available for some reason, should be marked by ":" in the appropriate Web Form cells. In most cases, it will be mandatory to enter something in each Web Form cell for the current period. Leaving such cells blank is not an option.

5.9 Zero by default

In accordance with the conventions used in Eurostat's dissemination database, data senders are requested to use the hyphen symbol "-" to represent data which is zero by default. Zero by default applies to observations on the Web Form that are non-existent in the given Member State.



6 EXPORTING DATA Exporting data is not a requirement of the transmission process. Nevertheless, it can be a useful functionality of Web Forms that a data sender may occasionally wish to use (e.g. to prepare a file for the bulk import of data). The export function makes it possible to export data from a Web Form table in six different formats: .PDF, .RTF, .XML, .CSV, TEXT or .HTML. In addition, the Web Form validation rules may be exported (in .CSV format). To export data, simply select the file format you wish to create (for example, .CSV) from the [Export] button drop-down list (see Figure 51).

Figure 51: Exporting Data

A dialogue box appears asking you to choose the destination file (see Figure 52). Select the folder, to which you wish to export, enter a file name and click on [Save].



Figure 52: Specifying the file name for the exported data



In this example the exported CSV file appears as follows: VOLUME OF AGRICULTURAL LABOUR INPUT (ALI); Measured in (1000) annual work units (AWU) ; ; Theme :4.04 - Agricultural accounts and prices; Domain :COSAEA; Dataset :COSAEA_ALI3_A; Year :2013 Reference period :A Country :LU Status :Draft; ; ;2010-Annual;2010-Annual;2011-Annual;2011-Annual;2012-Annual;2012-Annual;2013-Annual;2013-Annual; TOTAL;3.726;R;3.617;;3.657;;3.744;; NON-SALARIED;2.961;R;2.831;R;2.850;;2.934;; SALARIED;0.765;R;0.786;R;0.807;;0.810;; ; SPECIAL VALUES KEY C Confidential NA Not available - Not applicable or zero by default FLAGS KEY P Provisional F Final C Confidential R Revised E Estimated U Unreliable 1 User-defined 2 User-defined 3 User-defined;



7 IMPORTING DATA Data may be imported into a Web Form in either .CSV or .XML format. Importing data may be of particular interest to data-providing organisations that already have automated processes established in order to populate the Excel questionnaires used in the classic data transmission process.

Importing data is essentially the mirror operation of exporting data. For this reason, it is good practice to export the required data structure first, either in .CSV or .XML format, to study it, and be able to replicate the very same structure for the data import.

For example, in section 6, we saw the basic structure of a .CSV file exported from the COSAEA_ALI3_A dataset. This is precisely the structure required to import data in .CSV format back into this Web Form. Let us say, for example, that a data sender wishes to flag all the data for 2009 as unreliable. To do so, the data sender may enter "U" in each of the semi-colon separated fields corresponding to these flag cells and import these updated data back into the Web Form (see the following page).

This updated file may then be imported back into the Web Form using the [Import] button. A dialogue box will appear asking the data sender to choose the file to import (see Figure 53).

Figure 53: Importing Data



In this example the updated imported .CSV file appears as follows: VOLUME OF AGRICULTURAL LABOUR INPUT (ALI); Measured in (1000) annual work units (AWU) ; ; Theme :4.04 - Agricultural accounts and prices; Domain :COSAEA; Dataset :COSAEA_ALI3_A; Year :2013 Reference period :A Country :LU Status :Draft; ; ;2010-Annual;2010-Annual;2011-Annual;2011-Annual;2012-Annual;2012-Annual;2013-Annual;2013-Annual; TOTAL;3.726;R;3.617;;3.657;;3.744;U; NON-SALARIED;2.961;R;2.831;R;2.850;;2.934;U; SALARIED;0.765;R;0.786;R;0.807;;0.810;U; ; SPECIAL VALUES KEY C Confidential NA Not available - Not applicable or zero by default FLAGS KEY P Provisional F Final C Confidential R Revised E Estimated U Unreliable 1 User-defined 2 User-defined 3 User-defined;



In this example the imported .XML file appears as follows:



Having selected the updated file in this case, click on [Open], and the data is automatically imported into the Web Form (see Figure 54).

Figure 54: Data imported



8 MAKING AN OFFICIAL TRANSFER When all data is entered into a Web Form and the data satisfies all the validation rules (or at least is justified in case of basic rules), it is possible to make an official transfer. In such cases, the [Official Transfer] button will be automatically made available. To make the official transfer, simply click on the [Official Transfer] button (see Figure 55).

Figure 55: Official Transfer

The following dialogue will appear asking the data sender to confirm the official transfer (see Figure 56). Click on [Yes].

Figure 56: Confirmation message before Official transfer

After confirmation, a message will briefly appear in the pale yellow bar at the bottom of the Web Form (see Figure 57).

Figure 57: Official transfer done



The data sender will subsequently be returned to the Web Form entry window (see Figure 58). Note that the Web Form is automatically unlocked, once the official transfer is made. Note that the status of the Web Form also changes to "Transferred".

Figure 58: Transferred flag indicator

Once a Web Form is officially transferred, the responsible domain manager at Eurostat will automatically receive a notification of the arrival of the Web Form. At that stage, the domain manager will inspect it and, if all appears to be in order, will download its contents into Eurostat's production database. In this case, no further action needs to be taken by the data sender and the data is deemed to be officially accepted. Alternatively, if the domain manager is not satisfied with the data, s/he can reject the Web Form, which automatically triggers a rejection email to the data sender. In this case, the data sender will have to instantiate a new draft Web Form for the reference period in question, re-enter (modified) data and transfer it again. � Data preparers

Note Data preparers cannot make an official transfer, thus the official transfer button is not available for them. Apart from this one distinction, data preparers can do everything a data sender can.



9 OPENING EXISTING WEBFORMS

Section 3 presented how a data sender can create and open a Web Form draft during the instantiation procedure. A data sender may wish to access an already existing Web Form instance. For example, the data sender may have only partially completed a saved Web Form instance, which was instantiated earlier; or may wish to view the data in a Web Form instance transferred earlier to Eurostat.

To open an existing Web Form instance, whether it is an estimated, draft, transferred or rejected instance, the data sender must return to the Web Form Entry table (see Figure 4 to Figure 8) to find the instance in question. The Web Form can then be opened by simply clicking anywhere on its table row. This automatically starts the Java applet. Thereafter, the Web Form opens in Read-only mode. In Read-only mode, data may be viewed and exported. However, it can be neither edited nor officially transferred.

An estimated or draft Web Form instance (but not a transferred instance) can also be opened in Edit mode. In Edit mode, the data sender has the same possibilities of entering and transferring data, as when instantiating a Web Form. However, to edit a Web Form, the data sender must first “Lock” the estimated or draft Web Form instance, bearing in mind that a Web Form instance cannot be locked if it is at the same time locked by another user.

9.1 Locking of Web Forms

The Lock mechanism in Web Forms ensures that only one person can edit a Web Form instance at any given time, thus avoiding problems with parallel updates.

To lock a Web Form instance, select it by ticking the small box at the beginning of the Web Form row. (See the example dataset COSAEA_ALI3_A in Figure 59.)

Figure 59: Locking Web Forms



Next, click on the [Lock] button. A small red padlock symbol will now appear in the Lock cell on the Web Form Entry table (see Figure 60). The Web Form instance is now locked and ready for editing by the data sender, who locked it. Simply click anywhere on the Web Form instance row to open it.

Figure 60: Locked Web Form Indicator

A Web Form instance remains locked until the person, who locked it, unlocks it. At that point, another user can lock the Web Form and make further changes to it. To “Unlock” a locked Web Form instance, click on its tick box and thereafter, on the [Unlock] button.

When a data sender instantiates a Web Form instance, it is automatically locked. Moreover, when a data sender officially transfers a Web Form instance, it is automatically unlocked.

If two or more data senders share the responsibility for a particular dataset, it is important that they make sure to unlock Web Form instances after use (to unlock, simply select the tick box and click on the [Unlock] button). Otherwise, one data sender may inadvertently prevent the other from doing any further work on a particular Web Form instance.



10 MAKING REVISIONS TO DATA It is sometimes necessary to send revisions or updates for data that have already been transferred to Eurostat. This simply entails instantiating a new survey instance for the revised data for the reference period concerned. This creates a completely new version of the Web Form for the reference period. The process of instantiation was presented in section 3. The only difference instantiating for an already existing reference period is that the data sender will be asked to confirm the creation of another instance for the selected period (see Figure 61). Click on [OK] to continue and the new Web Form instance opens as normal.

Figure 61: Existing Web Form Entry

When a Web Form is first instantiated for a given reference period, it is automatically given version number 1 (V1). Subsequent instantiations for the same period will be given the version number V2, V3, V4, etc. There is technically no limit to the number of versions that may be created.

The version number of Web Forms is displayed in the XML column of the Web Form Entry table, as part of the full filename of each separate table with the Web Form instance. In the example below, COSAEA_ALI3_A "Volume of Agricultural Labour Input - Annual", an annual dataset, a Luxembourg data sender transferred data for 2013 (see Figure 62). Subsequent revisions have to be made and a new Web Form is instantiated for 2013. Both Web Form instances can now be seen in the Web Form Entry table with the designations V1 and V2 within the individual filenames of their individual tables.

When a draft is created for a new period, the Web Form is empty when the user opens it; whereas, when a revision of a received Web Form is concerned, a new draft will be automatically created by EDAMIS for the revision period with previous data pre-filled in the Web Form instance.



Figure 62: Different versions of a Web Form

The process of reference period re-instantiations with automatic version control leaves a complete and transparent record of revisions within the EDAMIS system. For this reason, it is very much in the interest of both data provider organisations and Eurostat to follow this process when sending data revisions.

� Revision and validation rules

Note As a general rule, validation rules apply to the earlier reference periods on Web Forms as well as on the current reference period. This is certainly the case with all monthly and quarterly Web Forms, which have a fixed format (i.e. always the same start and end point, Jan to Dec or Q1 to Q4). Therefore, the quality of the data is safeguarded during the revision for these Web Forms.

However, this is not necessarily the case in annual Web Forms, which typically have a floating format (the current reference period is always the last column). The current reference period cells often have validation rules, which depend on the previous reference period (e.g. the current value maximum is within 90-110% of the previous value). These rules cannot be applied to the first reference period. Therefore, revision on the fly can jeopardise the quality control of the data in these cases.



11 Glossary

Concept Definition

CSV Comma-Separated Values file format.

Data file Physical transmitted data file which should normally correspond to an instance of a dataset (a physical instance of a dataset occurrence).

Dataset Dataset structure with a specific periodicity and to which a deadline for transmission is usually linked. The data content of a dataset is homogeneous.

Dataset naming convention

Convention used by all production units at Eurostat in order to name a dataset in a uniform fashion. It is composed of the domain name, the statistical table name and the periodicity, all separated by an underscore. The dataset naming convention is recognised by all third-party applications at Eurostat.

Dataset occurrence Occurrence of a dataset for one country and one period of time series or sequence. For Web Forms, it is called a “Web Form instance”.

Dataset structure Concept of one single data structure associated to one or several statistical tables (closely linked to the same statistical domain).

Domain Consistent group of datasets that are closely linked together (belong to the same statistical area and based on the same legal act(s) or voluntary agreements).

EDAMIS Electronic Data files Administration and Management Information System.

EWA EDAMIS Web Application.

EWP EDAMIS Web Portal.

SDMX-ML XML format for the exchange of SDMX-structured data and metadata. The SDMX format (Statistical Data and Metadata Exchange) is the result of cooperation between the BIS, the ECB, Eurostat, the IMF, the OECD and the UN to explore common e-standards and ongoing standardisation activities that could yield more efficiency in the field of statistical information.

Table Statistical table as defined in a legal act or a voluntary agreement (with fields-made records).

Theme Theme of the Statistical Programme.

Documents

EDAMIS specialised user guide EDAMIS Web Forms for data ... · EDAMIS EDAMIS specialised user guide – EDAMIS Web Forms for data providers QUG-03-2014-07-30 [QUG] (EDAMIS Web Forms