Sep Usedr Manual

8/19/2019 Sep Usedr Manual

1/14

Evaluation ServiceIndividual Evaluation Report

Evaluation Service - IT User Manual

Individual Evaluation Report

Process and Supporting Tools


2/14

Service Desk : [email protected] / Phone: +32 2 29 92222

Quick Access

Introduction

This document provides evaluators with an overview of the IT support for remote evaluations organized by the

European Commission in the scope of specific Calls for Proposals. The manual is built in two sections: the first two pages

explain the basics, whereas the screen descriptions and annexes give more in-depth information about the system. We

invite you to read at least the first two pages " System in a nutshell " , which contains best practices when using the

system. The system is supposed to be intuitive enough so that you do not have to read the other sections, but you can

refer to them in case you have specific questions when working with the system.

The link to the system and helpdesk reference is provided below.

System Quick Access

Supporting Tool European Commission Evaluation Service

Link to the application https://ec.europa.eu/research/participants/evaluation/

Login used Your ECAS login

Service [email protected] / Phone: +32 2 29 92222

The service desk is available on weekdays between 9:00 and 18:00 CET (Friday 17:00)

Table of content

Quick Access .................................................................................................................................................................................................................. 2 Introduction ........................................................................................................................................................................................ 2

System Quick Access ........................................................................................................................................................................... 2

Table of content .................................................................................................................................................................................. 2

System in a nutshell ........................................................................................................................................................................................................ 3

Principles .............................................................................................................................................................................................. 3

Main screens, tasks ........................................................ ......................................................... ......................................................... .... 3

How to take a good start ...................................................................................................................................................................... 3

Main Screen Description ................................................................................................................................................................................................. 5

Login Screen ......................................................................................................................................................................................... 5

ECAS Login ........................................................................................................................................................................................ 5

Select the proper ECAS domain ....................................................................................................................................................... 5

Am I properly configured? ............................................................................................................................................................... 6

Task List ................................................................................................................................................................................................ 6

"My Active Tasks" versus "All Tasks" ....................................................... ....................................................... .................................. 6

Overview of the Task List ................................................................................................................................................................. 6

Navigate through tasks .................................................................................................................................................................... 7

Search and Filter tasks ..................................................................................................................................................................... 7

Proposal Details ................................................................................................................................................................................... 9

Evaluation Report .............................................................................................................................................................................. 10

Evaluation Report screen ............................................................................................................................................................... 10

Zoom on the Evaluation Form ........................................................................................................................................................ 11

Signing the Evaluation Report ........................................................................................................................................................ 12

ANNEX I. How to create an ECAS account? ................................................................................................................................................................... 13

ANNEX II. Declining a task ............................................................................................................................................................................................. 14

mailto:[email protected]:[email protected]:[email protected]


3/14


System in a nutshell

Principles

The system is designed to allow you, as an evaluator, to provide an electronic Individual Evaluation Report of aproposal, which will later be used to produce a consensus report which is a synthesis/consensus of your evaluation and

evaluations of other experts. In this document, “Evaluation Report” stands for Individual Evaluation Report (IER).

The process is rather simple and consists of:

Checking that you can provide the evaluation (no conflict of interest of any nature)

Filling-in your evaluation report within an electronic Web-based form

Signing your evaluation report, also electronically, within the same form

Main screens and tasks

To help you during the evaluation process, you will navigate in the system through 3 main screens:

Screen Description

Task List Here you can find a task for each report that has to be written, a link to the proposal details and

buttons for specific actions

Proposal Details Here you can find the abstract of the proposal, the composition of the consortium, and access to Part

A and Part B of the proposal. You can also provide comments from this screen.

Evaluation Report

Here you can write and sign the actual evaluation reports. You also have access to Part A and Part B of

the proposal from this screen. Here you can provide task comments, in case a specific action has to be

taken.

The above mentioned screens provide you with a list of tasks, easy access to proposal content, and to an Individual

Evaluation Report. The tasks have a simple life cycle, but the actions you can perform vary in relation with the status of

a task:

To summarize the various statuses:

Assigned : you did not yet open the evaluation report form

Open : you have opened the evaluation report form at least once


4/14


Signed : you finished your evaluation report and can no longer modify it

Declined : you declined the task and cannot access it any more

Cancelled : the Commission dismissed you as evaluator for that proposal

How to have a good start

First look at all proposals allocated to you in order to promptly identify any conflict of interest and thus to notify the callcoordination team that a proposal should be reallocated to another expert. The suggested process is described in detail

on the next page.

1. Connect to the system (using your ECAS account, contact the Service Desk if you have any issue).

Once connected, your active tasks list is displayed.

2. From the Task List, click on either the proposal reference or acronym.

This action opens the task details screen, where you have access to the proposal information: abstract and composition of

the consortium. From that point, you can already check potential conflicts of interest with the partners of the project, and

decline the task if necessary – see point (4).

3. From the task details, click on Part A and Part B.

Then again, if you face a conflict of interest you can decline the task as explained in point (4).

4.

In case of a conflict of interest, click on the Decline button.

A list of official reasons is displayed and you can check if your reason to decline the task falls in one of these categories. You

are also invited to provide a comment to specifically explain the reason.

Once declined, the task still appears in your task list, with a status Declined and you cannot access to the proposal detailsany longer.


5/14


5.

You can also start editing the proposal, from task details, click on the Edit Task button.

This action opens the Evaluation Report. The first time you edit the report the task state changes from Assigned to

Opened . You are not obliged to sign your report; if you wish you may save it, and return to the report at a later time.

6.

Later on, from the Task List, you will be able to open a report by clicking on the Edit button.

This action too opens the Evaluation Report. Please note that there is an auto-save mechanism every two minutes, thus

you will never lose your comments, even if your computer crashes, or you lose your internet connection.

7. When you are satisfied with your comments, you can click on the Sign button from the Evaluation Report or from the Task

List. Once signed, the report task state changes to Signed . The report is no longer editable, but is still accessible from the

"All tasks" List using the View button.

Main Screen Descriptions

Login Screen

This section describes in details how to enter into the system. Your identity is checked by the ECAS system (European

Commission Authentication Service), now widely used to secure most of the European Commission systems. If you do

not have an ECAS account yet, please create one by following instructions provided in the Annex I of this document

" How to create an ECAS account " .

ECAS Login

When you click on the login button, you will be prompted to authenticate through the ECAS login screen:

Please pay attention to the domain which is displayed on the login screen;

your domain should be External as displayed on the screenshot on the

left.

If the domain shown is not External, please click on Change it and select

the correct domain (see below, section "Selecting the ECAS domain")

Once the proper domain is selected, simply provide your ECAS user name

and password to enter the system.

Selecting the ECAS domainECAS is used by the European Commission for authentication of all

staff of European Union bodies (Commission, Council, Parliament,

etc…), and also by you as external users of Commission IT systems.

The ECAS domain that corresponds to your profile is "Neither an

institution nor a European Body".

Please select this domain or else your login will not be accepted.


6/14


Am I properly configured?

Once authenticated, you should see the list of tasks similar to the list shown

on the right (figure a), and not an empty task list (figure b).

If you get an empty task list, it most probably means that you have several

ECAS accounts and that you are not using the account we have used to

configure your access.

In this case, please contact the service desk immediately; provide the login

you are using (but not your password) along with a screenshot of the login

screen to facilitate troubleshooting.

They will configure your account accordingly.

(figure a) a correct task list

(figure b) an empty task list

Task Lists

The task lists are pivotal screens of the application, where you will navigate through your tasks. In addition to basic

navigation functionalities, the task lists also provide advanced search, sorting and filtering facilities.

"My Active Tasks" versus "All Tasks"

The system will propose you two tasks list, as described above:

My Active Tasks: all the tasks that you have to carry out, so all the Evaluation Reports that you have to write

and sign. As soon as a task is signed, it disappears from this tab and can be consulted in the All Tasks tab.

All Tasks: all the tasks that you have finalized or have the right to see. During the Individual Evaluation Report

phase you can see only your tasks. Once the consensus report phase starts, you can see all the other kind of

tasks such as Review Consensus Reports, Write Consensus Report, etc., in this tab.

Overview of a Task List

The screenshot below describes the task list screen:

A summary of your tasks (you can easily check whether all tasks had been performed)

Access to the task details screen in order to consult proposal information

Access to the actions available on a task: edit a report, decline a task, etc…

mailto:[email protected]?subject=SEP%20Evaluation:%20Empty%20Task%20Listmailto:[email protected]?subject=SEP%20Evaluation:%20Empty%20Task%20Listmailto:[email protected]?subject=SEP%20Evaluation:%20Empty%20Task%20List


7/14


At the top of the screen you

can see the account used to

connect to the system; this

might be requested by the

service desk.

The summary applies to all thetasks allocated to you.

You can sort all columns, at any

given time.

To access a proposal's details,

simply click on the proposal

reference or acronym (see

proposal details for a

description of this screen).

The actions available vary depending on the status of the activity. Click here for an explanation of the various states and

actions possible depending on these statuses.

The score displayed in the task list is the computed based on the marks you provided in the report. The way the score is

computed depends on the call, therefore it is not explained in this document, but will be presented during the expert

briefings to which you will be invited.

Navigate through tasks

Navigation through the different tasks is rather simple: tasks are displayed in a number of pages and you can navigatethrough these pages (the size of pages is customizable).

You can choose the number of tasks

to be displayed in the page. Simply

select another value to implement thechange.

On the bottom-left of the task list you

see a summary of the number of tasks

actually displayed in the page.

On the bottom-right of the task list you

will find the links to navigate through

pages.

Search and Filter tasks

The screen provides you with advanced search and filter functionalities to ease your navigation through the proposals.

A full text search on all columns of the list:

o Simply type the text to search (e.g. the proposal acronym to display, the status, the panel, etc.)

o The Task list is automatically filtered with your selection.


8/14


To narrow the displayed list to a more specific set of proposals through the filters: simply click on search to

activate the filters, reset to go back to the regular list and cancel the filtering.


9/14


Proposal Details

The proposal details screen gives you an easy access to:

Abstract of the proposal

The composition of the consortium, some partner information and contact persons' information

Access to Part A and Part B of the proposal

If you want to raise a conflict of interest, you can flag it directly from here by clicking on

the Decline button at the bottom of the screen (see ANNEX II. " Declining a task " for

more details). You can also go back to the task list, or click on the Edit task button to

open the evaluation report.

You can open the evaluation report from the task list at any moment.

Please note that the proposal details screen is not the only screen where you can access to the Part A and Part B; youcan also view these documents directly from the evaluation report form.

Tips

The proposal details screen allows you to quickly identify conflicts of interest, without even opening the

evaluation report. The detection of conflicts of interest is relatively easy thanks to a display of the composition ofthe consortium, and eventually by quick access to the Part A and Part B of the proposal.


10/14


Evaluation Report

The Evaluation Report screen is the screen where evaluators actually write and sign evaluation reports. The report is a

Web form where evaluators are invited to fill-in evaluation criteria defined in the scope of the call. The criteria

themselves are text fields which do not accept any formatting in order to avoid any problems of interpretation later in

the evaluation process.

Evaluation Report screen

The screen as such provides access to all

information necessary to write the

evaluation report: the proposal (Part

A/Part B), task comments provided by

the organizers of the call, actions that

can be performed, and the Report Form

itself.

The Form is detailed in the next section.

An evaluation progress index is

displayed on the top of the form,

computed on the basis of your data

entry.

Besides Sign (explained in a specific section), and Decline (see

ANNEX II. "Decline a Task "), you can Print the report and of

course Save your data entry at any moment. Printing a report

will generate a PDF file that you can then print or save.

You can get explanation on the rankings to apply by clicking on

the help link at the top of the screen. The help provided

depends on the call (the example provided here is just

presented as an illustration).


11/14


Zoom on the Evaluation Form

The evaluation form displays all criteria and specific sections of

the evaluation report. By default, all the blocks are hidden and

only the title for each criterion is displayed. To expand or

collapse the criterion, just click on the banner of the criterion or

section. To expand or collapse all criteria, click on

"Expand"/"Collapse".

On each banner of an evaluation criterion, you will find a

summary of your evaluation of that criterion: the score you

provided, the threshold for the criterion.

If you expand a criterion, you will see the fields you are

expected to complete. Depending on the evaluation, the report

contains only a text input per criterion, or a list of sub-criteria to

fill-in. Some are mandatory, while others are not. An asterisk (*)

indicates a mandatory field.

Your scoring of the criterion is requested at the top of the

criterion section. The precision of the mark is also dependent on

the call (full point, half point of full decimal).

A character counter is shown to help you in your data entry.

After Signing (see next section), the Evaluation report form will become read-

only. You will still be able to read your assessment or print it, but you will not be

able to modify it.

Tips

You may save your form at any moment with the Save button at the bottom

of the screen, but your form will be saved automatically every two minutes.

Warning

If you signed by mistake, or if you wish to reopen

your evaluation report, please contact your

Coordinator, who is the only person authorized

to reopen your task.


12/14


Signing the Evaluation Report

Signing the evaluation report is done by clicking on the Sign button at the bottom

of the Evaluation Report. During this operation, the system will check that you

have provided all the mandatory information.

If you try to Sign a form without filling all the necessary

fields, a red warning will notify you of the missing

information.

The form automatically expands criteria and sections which

contain the errors; simply browse the form to check which

field is pending completion.

On the left are a few examples of common mistakes:

1.

A score which had not been provided

2. A mandatory field without any content

3.

A YES/NO selection which was not performed.

It is not possible to sign a report without resolving all the

errors raised by the system

As soon as all mandatory fields are provided, the system will ask you to confirm

that you indeed wish to sign the report. The system requires that you confirm that

eventual conflicts of interest had been checked (see ANNEX II. " Declining a task "

for more details).

Warning

After signing a report, you no longer have the possibility to modify it. If you signed by mistake, or if you wish to reopen your

evaluation report, please contact your coordinator, who is the only person authorized to reopen your task.


13/14


14/14

Service Desk: DIGIT-EFP7-SEP-SUPPORT@ec europa eu / Phone: +32 2 29 92222

ANNEX II. Declining a task

You can decline a task at any moment in the process. However declining a task exposes you to various choices - se

below:

1.

The Decline button is available from the task list, from the

proposal details screen, or from the evaluation report.

Click on Decline to decline the task.

2.

You are then proposed to choose the main reason for

declining the task.

The Other reasons are basically covering unforeseen

reasons, and are not covered in this section.

3.

A sub-list of reasons is then proposed.

You are invited to select one of these options. In the case

of conflict of interest, you may choose from the official list

of direct and indirect conflicts of interest that will explain

why you feel you cannot evaluate the proposal.

4.

Once you have declined the task, you no longer have an

access to either the proposal, nor to the evaluation report.

Warning

After declining a task, you no longer have the possibility to access the task . If you declined by mistake, please contact your

coordinator, who is the only person authorized to reopen your task.

Documents

Sep Usedr Manual