EDAMIS Web Application user guide for data sender€¦ · QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 3 1 Introduction The EDAMIS Web Application (EWA) is a web based

QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 1

EDAMIS Web Application

user guide

for data sender

Project EDAMIS

Author Data and metadata services and standards, Unit B5

Release 1.1

Status Final

Reference QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc


TABLE OF CONTENTS

1 INTRODUCTION ......................................................................................................................................... 3

1.1 PRINCIPLES ................................................................................................................................................ 3 1.2 USER INTERFACE ....................................................................................................................................... 3 1.3 THE EDAMIS MESSAGES ......................................................................................................................... 4

2 WHERE TO START FOR SENDING DATA TO EUROSTAT USING THE EWA? ........................... 4

2.1 GET YOUR USER ACCOUNT......................................................................................................................... 4 2.2 LOGGING IN FOR THE FIRST TIME ............................................................................................................... 4 2.3 MAINTAINING THE USER ACCOUNT ............................................................................................................ 6 2.4 EUROSTAT CONNECTION STATUS ............................................................................................................... 6 2.5 LOGGING ON AND LOGGING OFF ................................................................................................................ 6

2.5.1 Logging on ...................................................................................................................................... 6 2.5.2 Automatic Logging off ..................................................................................................................... 6 2.5.3 Logging off ...................................................................................................................................... 6

3 SEND DATA .................................................................................................................................................. 7

3.1 INTERACTIVE MODE ................................................................................................................................... 7 3.2 SEMI AUTOMATED MODE ........................................................................................................................... 9 3.3 FULL AUTOMATED MODE ......................................................................................................................... 10

4 MANAGE SAVED DATA FILES ............................................................................................................. 12

5 RECEIVE DATA FILES ............................................................................................................................ 13

6 ACTION HISTORY ................................................................................................................................... 14

7 GLOSSARY ................................................................................................................................................. 15


1 Introduction

The EDAMIS Web Application (EWA) is a web based application which allows a user to

send data files to Eurostat. This user guide explains new users how to get started with the

application. Some basic but important issues are addressed in this document beginning from

the registration and the statistical data transmission to the monitoring of the data files traffic.

1.1 Principles

EDAMIS is a complete environment that offers data transmission and management services in

the European Statistical System. It stands for "electronic Data files Administration and

Management Information System". It includes:

1. Advanced data transmission facilities for all Eurostat data providers, mainly based on

three main approaches: the EWA (EDAMIS Web Application), the EWP (EDAMIS Web

Portal) direct upload system and the EWF (EDAMIS Web Form) which looks like a

spreadsheet.

2. Advanced data reception features at the Eurostat "Single Entry Point" (SEP), which is the

EDAMIS server

3. A reporting system that provides reports on traffic

The EWA is a local HTTP server that enables the secured transmission of data from Member

States to Eurostat. The statistical data that needs to be transmitted to Eurostat is made

available to the EWA, which converts it into a standardised message and uses a

telecommunication component (STATEL – hidden for the data sender) to transfer the

message to the EDAMIS server in Eurostat. Based on the metadata associated with the data in

the message, the server knows where and how to deliver the data. The server then sends an

acknowledgement to the EWA to inform the user that the data has been delivered.

1.2 User Interface

The EWA is used through a Web browser such as Internet Explorer or Firefox, with the EWA

server performing all the functions, such as sending data to the EDAMIS server, receiving and

processing acknowledgements, or encrypting confidential datasets. No applets are

downloaded, and the browser remains the visible interface of EWA. As a result, no

certificates must be installed or accepted, and the operation of EWA is easy to master.

EDAMIS Help Centre and Support

Note A set of comprehensive documents are available at the EDAMIS Help Centre. They cover a wide

range of topics like the available software, including very technical aspects for some of them and

the advices to select the best method to fit your requirements as well as the installation

procedures. You can look at the EDAMIS Help Centre in the following link:

https://webgate.ec.europa.eu/edamis/helpcenter/website/index.htm

The support team is at your disposal for any questions, incidents resolutions or you want to report

or to resolve any problem encountered when using the data collection tools. Do not hesitate to

contact the support team for any problem or question related to data collection.

Mail to: [email protected]

Tel: (+352) 430133213

https://webgate.ec.europa.eu/edamis/helpcenter/website/index.htm

mailto:[email protected]


1.3 The EDAMIS Messages

The EDAMIS system is based on sending and receiving messages. Unlike many message

based systems, there is no size limit to EDAMIS messages, so sending or receiving a dataset

of several hundreds of megabytes is just as feasible as sending a 10 kilobyte dataset. Previous

implementations used the EDIFACT standard; this implementation uses XML formatted

messages. The logical contents of these messages are very similar, but the additional

functions offered by EWA require additional information in the messages.

2 Where to start for sending data to Eurostat using the EWA?

2.1 Get your user account

Obtain the URL from the administrator of the local EWA, who usually is your “Transmission

Coordinator” or “the Local Coordinator” in your country. You will find an up-to-date list in

the following link https://webgate.ec.europa.eu/edamis/helpcenter/website/overview/lc.htm

Your administrator will provide with a user-id, which in principle, should identical to your

ECAS user-id, if you have one.

2.2 Logging in for the first time

Once you have both, the URL and your user-id, you can open your web browser and copy the

provided URL in the address filed. This URL is the internal hyperlink under which the EWA

has been installed and it looks like , or it can also be an address –

(depending on your network configuration) -- like http://192.168.1.100/Login as it appears on

the login mask below (See fig 1)

Fig 1: EWA login mask

When logging in for the first time, you must only introduce your User-ID and leave the

Password empty. The registration process will be completed at the first login, where you will

have the possibility to specify your own password (See fig 2).

https://webgate.ec.europa.eu/edamis/helpcenter/website/overview/lc.htm

http://192.168.1.100/Login


Fig 2: User Management screen

In the user management window, the data sender will need to complete the form by

introducing a password which is mandatory as indicated by the asterisk nearby. The data

sender will have to confirm the password (which is hidden) and is invited to complete the rest

of the form if necessary. Once this is done, the data sender will click on the [Update] button to

save the changes (See Fig 3).

Fig 3: User Management details

Lines on short activity report field

Note When the data sender logs in the EWA, a short activity report is displayed. The data sender has

the possibility to specify the number of lines which will be shown by entering the corresponding

number in this field. If left to zero, no short activity report will be displayed.


2.3 Maintaining the user account

Once the registration is completed, the full main menu is made available and a confirmation

that the user management parameters have been updated (See Fig 4).

Fig 4: EWA main menu

Later on, if the data sender wants to update the user data, this can be done via the menu

“ ”. This will bring the user to the “User Management” screen as in fig 2.

2.4 Eurostat connection status

The upper right corner of the EWA window shows the status of the connection to Eurostat. If

the status line is green (See fig. 5), datasets will be transmitted to Eurostat. If the status line is

red, there might be a communications problem and it would be advisable to contact your

EWA system administrator or the Local Coordinator.

Fig 5: EWA connection status

2.5 Logging on and logging off

2.5.1 Logging on

The data sender logs in with its User-ID and password when the logging mask is displayed.

2.5.2 Automatic Logging off

After a configurable period of inactivity (3 hours by default), users are automatically logged

off. It is however recommended to use the “Log off” main menu

http://app2.EDAMIS.net/clientdocs/logoff.htmlentry to terminate the connection with the

EWA once the work has been completed.

2.5.3 Logging off

The data sender can press the “Log off” main menu entry to log off from EWA.

http://app2.edamis.net/clientdocs/logoff.html


3 Send data

3.1 Interactive mode

Sending a data file to Eurostat in an interactive mode is as simple as this:

1. Log on to the EDAMIS Web Application

2. Select “Send Data file”

3. Browse for the file to transmit

4. Complete the metadata as requested

5. Click Perform Transfers

By using the main menu “Send Data File”, the “Send Data File” screen appears (See Fig 6).

Fig 6: Send Data File screen

The screen areas are as described in the table below:

1 File name

This is the name of the file which will be sent to Eurostat. It can be whatever file, file

type, file size. EDAMIS Web Application will automatically complete the “send data

file” input form, provided that the file name is structured according to the dataset naming

convention or that the file is GESMES formatted. This concerns the dataset, the year, the

period and the action fields

2 Dataset

In the dataset drop down box, there is the list of datasets that the user is supposed to send

to Eurostat as configured by the administrator of the EDAMIS Web Application.

3 Year

This is the year of the reporting period, for example “2008”.

4 Period

This is the reporting period and it depends on the periodicity of the dataset. This field is

greyed once it is not relevant. For instance, it is not visible for yearly dataset since the

year itself is sufficient to determinate the coverage of the reported data.

5 Action

The action code is only given as information and has no effect on the data transmission

chain. It can be “New”, “Append”, ”Partial”, “Replace” or “Other”. When used, there

must be an agreement about the meaning of the action code between the production unit

at Eurostat and the data providers in Member States.


6 Encrypt?

In the EDAMIS inventory, when a domain has an encryption key attached to it, the

encrypt flag is automatically proposed to the data sender. When ticked, the data file

which is transmitted will be encrypted with the public encryption key attached to the

domain whose the dataset belongs to. At that time, only the owner of the corresponding

private key (domain manager at Eurostat) will be able to decode the ciphered data file.

7 Valid?

In the EDAMIS inventory, when a dataset has defined validation rules, the valid flag is

automatically proposed to the data sender. It will then be possible to check the coherence

of the data file against the defined validation rules before sending the data to Eurostat.

The [Add/Validate] button will launch the validation and the validation report will

appear in a separate window. If the validation does not report validation errors, then the

[Perform Transfer] button will be enabled allowing the data file to be transmitted. If the

validation fails, then the data sender must correct the corresponding file and reprocess it

again. In any case, the data sender is able to “force” a transmission using the [Force]

button even if the data file does not match with the validation rules.

8 Force

[Force] button to be used to “force” a data file transmission

9 Clear

The [Clear] button allows deleting a row from the “send data file” input screen.

10 Free text comments

This is a place where the data sender can leave a free text message at the attention of its

counterpart at Eurostat. It has a length of 256 characters.

11 File with explanatory notes

When there is a need to provide the used methodology, the explanatory note is used. This

allows the data sender to attach a file which has a maximum size of 2Mb. It will be

delivered to the statistician in charge of the dataset together with the notification

message.

12 Browse

The [Browse] button allows the data sender to “browse” the directories in which the data

files are located.

13 Add/Validate

The [Add/Validate] button has two functions. It allows building up a list of several data

file to be transferred to Eurostat, in case of multiple transmissions and if validation rules

exist for the dataset(s), it performs the validation of the attached data file.

14 Perform transfer

The [Perform transfer] button, used at the final stage, really performs the transmission of

the data file(s) which has/have been selected with the [Add/Validate] button.


3.2 Semi automated mode

The semi automated mode allows a third party application to deliver data files in a particular

directory of the EWA belonging to a specific user. Usually, EWA is installed by default under

the \\EWA directory on a dedicated server. Under the “EWA” directory there is a “user”

subdirectory containing all defined EWA users. Therefore, all the EWA users have their own

user directory where the data files which need to be sent to Eurostat can be placed (See fig 7).

Fig 7: the “user/sender” directory

In the above-example, the EWA application knows three users which are the “Administrator”

and the users “sender” and “u1”.

Once the data files have been dropped in the user directory, for example in the user directory

of the user “sender”, next time this user will log into the EWA, and will go to the “Send Data

file” main menu, a list with all the data files which were dropped in the “sender” directory,

will appear pre-filled in the screen in a way that the data sender (here called “sender”) can see

all of them. The user will then just need to confirm (or not) that data files in the list must be

transferred (or not) by clicking the [Force] button (See fig 8).

Fig 8: The pre-filled “Send Data File” screen of the user “sender”

In this way, the data sender will have more a role of controlling what must be sent since EWA

will have prepare the job.

The third party application which drops the data files in the appropriate directory could be a

robot, a third party application or a data warehouse, but it can simply be done via a windows

browser using the drag and drop technique.

Access rights to the directory \\EWA\user\sender

Note The network administrator must allows the third party application to have write access right to

this directory otherwise the application will not be able to drop files in the special EWA directory

and therefore the semi-automated mode will not be enabled.


3.3 Full automated mode

The full automated mode allows a third party application to deliver data files in a particular

directory of the EWA belonging to a specific user. Usually, EWA is installed by default under

the \\EWA directory on a dedicated server. Under the “EWA” directory there is a “user”

subdirectory containing all defined EWA users. In each user sub-directory, there is an “EDI”

sub-directory. Once data files are placed in this specific directory, EWA considers that these

data files must be automatically be sent to Eurostat (See fig 9).

Fig 9: the “user/sender/EDI” directory

EWA is regularly browsing all EDI users’ sub-directories and whenever a data file is found,

EWA initiates an automatic transmission to Eurostat.

The mechanisms which enable the full automated mode rely on the use of the dataset naming

convention for building the data file name (which is the envelope) or on the use of the

GESMES formatted data files.

Whenever a data file cannot be processed automatically due to some inconsistencies, EWA

will automatically switch to the semi-automated mode, allowing the data sender to complete

the missing information before performing the transmission.

More information about the dataset naming convention are found in the following link

http://circa.europa.eu/irc/dsis/EDAMIS/info/data/website/practice/DSNC.htm (See fig 10)

Fig 10: Extract of the dataset naming convention

http://circa.europa.eu/irc/dsis/edamis/info/data/website/practice/DSNC.htm


Access rights to the directory \\EWA\user\sender

Note The network administrator must allows the third party application to have write access right to

this directory otherwise the application will not be able to drop files in the special EWA directory

and therefore the semi-automated mode will not be enabled.


4 Manage Saved Data files

By using the “Send Data file >> Saved Data files” menu, the data sender is able to manage the

backup copies of the data files that have been sent to Eurostat (See fig 11).

Fig 11: Manage Saved Data File screen


1 Date

This is the date of the sent data.

2 User

This is the user who sent the data.

3 Data file name / notes name

This is the name of the file which will be sent to Eurostat.

4 Domain

This is the name of the domain to which belong the dataset.

5 Dataset

This is the name of the dataset.

6 Year


7 Period

This is the reporting period.

8 Action


chain. It can be “New”, “Append”, ”Partial”, “Replace” or “Other”.

9 Size

This is the file size of the data file sent.


5 Receive Data files

When the EWA is used to receive datasets, the “Receive Data File” menu entry displays a

form with a list of the datasets that have been received. Click the appropriate dataset to

download it to your PC (See fig 12).

Fig 12: Download Received Data Files screen


1 Date

This is the date of the sent data.

2 Data file name

This is the name of the file which will be sent to Eurostat.

3 Domain

This is the name of the domain to which belong the dataset.

4 Dataset

This is the name of the dataset.

5 Year


6 Period

This is the reporting period.

7 Action


chain. It can be “New”, “Append”, ”Partial”, “Replace” or “Other”.

8 Size

This is the file size of the data file sent.

9 Free text

This is the message which has been given by the data sender.

10 Explanatory notes

This is the link to the file which was attached as explanatory notes for the sent data file.


6 Action History

The “Action History” main menu entry gives access to the following commands (See fig. 13):

Fig 13: Action history menu

1. Short: an on-screen report restricted to the default number of lines set by the

Administrator or in the user account (see Maintain Account).

2. Long: produces an on-screen report of all the actions

3. Month: an on-screen report restricted to the entries for the current month

4. CSV: Send: generates a CSV file with all the Send actions

5. CSV: Receive: generates a CSV file with all the receive actions

6. CSV:Admin: generates a CSV file with all the administrative actions

http://app2.edamis.net/clientdocs/maintain.html


7 Glossary

Concept Definition

CSV Comma-Separated Values file format.

Data file Physical data file transmitted which should normally correspond

to an instance of a dataset (a physical instance of a dataset

occurrence).

Dataset Dataset structure with a specific periodicity and to which is

usually linked a deadline for transmission. The data content of a

dataset is homogeneous.

Dataset naming

convention

Convention used by all production units at Eurostat for uniformly

naming a dataset. It composed of the domain name, the statistical

table name and the periodicity, all separated by an underscore.

The dataset naming convention is recognised by all third-party

applications at Eurostat.

Dataset occurrence Occurrence of a dataset for one country and one period or time

series or sequence. For Web Forms it is called a “Web Form

instance”.

Dataset structure Concept of one single data structure associated to one or several

statistical tables (closely linked into the same statistical domain).

Domain Consistent group of datasets that are closely linked together

(belong to the same statistical area and based on the same legal

act(s) or voluntary agreements).

EDAMIS electronic Data files Administration and Management Information

System.

EWA EDAMIS Web Application.

EWP EDAMIS Web Portal.

SDMX-ML XML format for the exchange of SDMX-structured data and

metadata. The SDMX format (Statistical Data and Metadata

Exchange) is the result of cooperation between the BIS, the ECB,

Eurostat, the IMF, the OECD and the UN to explore common

e-standards and ongoing standardisation activities that could

afford more efficiency in the field of statistical information.

Table Statistical table as defined in a legal act or a voluntary agreement

(with records made of fields).

Theme Theme of the Statistical Programme.

Documents

EDAMIS Web Application user guide for data sender€¦ · QUG-01-2016-04-18 [QUG] (EDAMIS-EWA-Send-Data) F.doc Page 3 1 Introduction The EDAMIS Web Application (EWA) is a web based