ESCO (2015) SEC 037 FINAL
Creation date: 05/03/2015
Last update: 17/04/2015
Planning the Delivery
of ESCO v1
Purpose of this document
This document describes the project management aspects of the development and the delivery of
ESCO v1. It is written to support the Maintenance Committee (MAI) to understand the process of
the content development for the remaining 16 sectors (see the Sector Table in Annex A), and the
integration of the content of all sectors. The document describes as well how a dashboard will
soon provide transparency on progress of the work done on the content. (Detail on many of the
processes described in this document is provided in separate documents.)
Scope of this document
Included in this document is a description of the project management for the remaining
development of the ESCO v1 deliverable, including:
Content development for the remaining 16 sectors (see the sector table in annex A)
Content development across sectors
Tasks involved in integrating and aligning all sectors to ensure a consistent ESCO v1
deliverable for all sectors.
This document does not describe:
The development of the ESCO methodology
ESCO supporting processes like, for instance, the ESCO guideline management and ESCO
tool management (e.g. Collaborative Taxonomy Creator (CTC), and the online consulting
platform)
ESCO test processes (including fit-for-purpose testing and mapping pilot)
ESCO continuous improvement processes
ESCO communication processes
The publishing of demo versions of ESCO (ESCO V0.1 and V0.2)
Mapping of the final ESCO v1 to national classifications.
Final Deliverables
The deliverables of the ESCO v1 project are:
The ESCO v1 data model
The ESCO v1 classification containing occupations (OCC), knowledge, skills and
competences (KSC), as well as related concepts covering all sectors of economic activity
The ESCO v1 qualifications
Project Organization
The project organization relevant for the delivery of ESCO v1:
Figure 1. The ESCO Organisation.
Entity Responsibility for
European Commission Managing the overall project and the final deliverables
Secretariat (SEC) Project management, methodology development, quality
control, monitoring, and mediation for the online consultation
process
ESCO Board (BOA) Providing strategic guidance, and supporting the promotion of
developing, implementing and use of ESCO in the labour
market, and in the education and training sector.
ESCO Maintenance Committee
(MAI)
Providing guidance and support for the process to incorporate
the domain knowledge into the ESCO classification
Sector Reference Groups (SREFs) Responsible for domain knowledge and content development
for the first 11 sectors.
Online Community of Experts Providing domain knowledge, expert review and expert
validation for the remaining 16 sectors.
Cross Sector Reference Group
(CSREF)
Development of cross-sectoral knowledge, skill and
competences.
Software Development Partners Development of the software components relevant for
creating, maintaining and publishing the ESCO classification
Taxonomy Expert Group (TEG) Sector research, content development, monitoring and
mediation for the online consultation process
Content Development Partners Responsible for developing the traversal skills
Translators & Terminologists English language correction and the proper use terminology,
Translations
Milestones and High-Level Process
The ESCO project has two core deliverables relating to the ESCO classification itself, i.e. the ESCO
v1 data model, and the ESCO v1 classification. (At the next MAI meeting the SEC will present how
they will integrate qualifications in the classification.)
The work remaining for these core deliverables is:
The ESCO v1 data model: The work is complete. The data model is currently under life-
cycle management.
The ESCO v1 classification containing occupation, knowledge, skill, and competence
concepts relevant for the 27 defined sectors: The SREFs, TEG, and SEC have completed
most of the work for the first 11 sectors. Work for the remaining 16 sectors has started.
Additionally the work to integrate the sectors classifications into a balanced,
comprehensive ESCO classification will start soon.
The main process, i.e. the process that delivers the ESCO classification for the OCCs and KSCs for
the remaining 16 sectors is broken down into 3 phases:
1. Full content drafting (per sector)
2. Online consultation and refinement process
3. Sector finalisation process.
This process will deliver 16 additional subsets of the ESCO classification, one for each new sector.
In parallel to the work done on the content, a reference group is working on the creation of
transversal KSCs with a cross-sector relevancy for future versions of ESCO. (The TEG contextualises
the transversal KSCs already available while drafting of the full content.)
The SEC and the TEG submit the output of the sector finalisation process to the content
integration and clean-up process. This process will also spin-off the translation processes. The
integration and clean-up processes will, in steps, curate, enrich and improve the quality and the
richness of the ESCO classification. When the TEG and SEC conduct this process, the SEC will
regularly update the ESCO management bodies on the progress, the types of issues encountered,
and the improvements and enrichments being implemented.
When the European Commission decides to publish a formal release of the ESCO classification, the
SEC will prepare the release, and the MAI will be asked to formulate their final opinion on the
proposed release prior to a final decision by the European Commission to publish ESCO v1.
Technical Note:
With the formal publication of ESCO v1, the Commission will take responsibility for actively
managing the published URIs used for identifying the concepts of the classification. This means
that the semantics associated with a URI of a concept will stay intact between versions of ESCO. In
practice, this means for a user that a concept-mapping table from a national classification to ESCO
remains valid with new versions of ESCO.
Content development per sector (for the remaining 16 sectors)1
Figure 2. Content development per sector.
The SEC organises the content development for the remaining sectors in three major phases, i.e.:
1. Full content drafting, delivering the content in the Delta version (ready for online
consultation).
2. Online consultation and refinement, delivering the content in the Iota version (refined and
validated by the online community of experts)
3. Sector finalisation, delivering the content in the Nu version (Sector complete, ready for the
integration and clean-up process)
The individual process steps for these 3 phases are described in the following three sections.
1 Detail on the work associated with each individual step will be documented in the ESCO Guidelines v2.
Full content drafting
Figure 3. Full content drafting (per sector).
The ESCO sector content development work entails all the steps necessary to create a draft of the
content for a sector that is appropriate to be presented to the online community for online
consultation.
This first phase is broken down in:
1. A creation step in which the TEG delivers the sector breakdown with all relevant
occupations and occupation groups. The Alpha version is the intermediate deliverable
from this process step.
2. A validation step in which the SEC validates the Alpha version and helps to remediate
outstanding issues. Once validated, the content is set to be in Beta version.
3. A content extension step in which the TEG extends the OCCs captured in the sector
breakdown with all the essential and optional KSC and associated concepts. In this step,
the TEG also ensures that all KSC have all necessary properties defined. This step delivers
the Gamma version.
In parallel to work being done by the TEG, the MAI will be invited to provide comments in
CTC on the sectoral breakdown and list of occupations.
4. A validation step in which the SEC verifies if the Gamma version meets all the appropriate
quality criteria specified. Once validated, we declare the content for the sector to be in
the Delta version.
Online consultation and refinement2
Figure 4. The online consultation process and refinement process (per Sector).
The online consultation and refinement process defines the following five separate process steps:
5. The first online consultation. This is the process in which the online experts evaluate the
sector breakdown and the occupation profiles. The experts can provide suggestions and
comments and can engage in discussions with fellow experts as well as with TEG/SEC. The
TEG/SEC will monitor and mediate the process. In areas where insufficient activity takes
place, the SEC will take action to resolve the issue to ensure proper evaluation of the
content. In parallel to the work done by the Online Experts, the TEG and the SEC, the MAI
can monitor the activity on the platform. In case they observe issues, they can either
discuss that on the provided forum on CTC, or notify the SEC.
The annotated ESCO content that is the result of the first round of online consultation is
referred to as the Epsilon version.
6. On the basis of the first round of online consultation, the SEC will decide which sectors or
subsectors need a second round of online consultation. The SEC will only submit content
to the second round in case content was not properly discussed.
7. In the case, that the SEC decides on a second round of online consultation the TEG will
refine and create the Zeta version of the content on the basis of the comments of the first
online consultation. This process can be monitored by the MAI in CTC.
2 For details on this process see document ESCO (2015) SEC 020.
8. The online community of experts will review the Zeta content in the second round of
online consultation. In parallel to the work done by the Online Experts, the TEG and the
SEC, the MAI can monitor the activity on the platform. This process step will deliver the
Eta version.
9. The TEG refines the content that comes out of the online consultation process. During the
refinement process, the content is updated using the comments and suggestions from the
online consultation process. This delivers the Theta version. This process is monitored by
the SEC as well as by the MAI in CTC.
10. The online community will validate the Theta content and by doing so create the Iota
version. The TEG and SEC monitor this process. In parallel, the MAI can monitor this
process on the online consultation platform as well.
Finalisation of each sector
The Iota versions that are the result of the online consultation and refinement process enter the
process to finalise the content for a sector. During this phase persistent quality issues that are
identified during the last steps of the online consultation process or that surface during the quality
assessment are to be resolved in parallel to sharing the current state of the content for a sector
with the MAI for evaluation.
11. As a first step, the SEC will execute a quality assessment on the actual state of the Iota
content. The SEC will document possible issues and create a Kappa version.
Figure 5. The sector finalisation process.
12. The SEC will present the Kappa content for a sector, the Key Performance Indicators
(KPIs)3 and the associated report that describes the list of issues to the MAI for an opinion.
Possible feedback on the content will be captured to support the SEC and the TEG to
refine the content.
13. In parallel to presenting the Kappa content to the MAI, the TEG will start to address the
pending issues. The additional feedback from the MAI is added to the work as soon as it is
available. These activities will result in the Lambda version of the content.
14. To prepare for the alignment and rationalisation of the KSC concepts in a later stage, the
SEC needs to normalise the concepts. The knowledge structure requires the concepts to
be described as atomic concepts or inseparable concepts (e.g. “serve food and drinks” will
be reformulated as two concepts, i.e. “serve food” and “serve drinks”). Cases for which
normalization is not straightforward will be labeled to be “complex”. The SEC will resolve
these cases at a later stage.
15. The SEC does a final validation check on the Mu version and declares the content work for
the sector to be final with the Nu version. (Please note: The content is final when
evaluated within the scope of a single sector, re-evaluation in the scope of multiple sectors
3 Key Performance Indicators (KPIs): a list of measurable parameters are defined for the content to support Quality
assessment. E.g. average number of skills in a job profile, number of concepts that do not have a description (and links
to these concepts), balance between essential and optional skills.
will mean that the SEC needs to review and edit the content to achieve a balanced fit in
the whole ESCO classification.)
ESCO cross-sector development
The ESCO cross-sector development work for ESCO v1 consists of two major phases, i.e. phase 1 in
which the transversal KSC concepts (e.g. "measure") are identified and described, and phase 2 in
which the transversal KSC concepts are contextualized in context of the defined occupation
profiles.
For ESCO v1, phase 1 of the cross-sector development has been completed. The TEG executes
phase 2 in the context of the content development process.
In parallel, further work on the transversal KSCs will be done in preparation for a major future
ESCO releases (starting with ESCO v2).
ESCO integration and clean-up process4
The ESCO integration and clean-up process is a process that can be executed in an iterative
manner and can start as soon as the first sectors are available in their “Nu” version. The SEC will
test the process with the first three sectors that have been signed-off with their KSCs by the MAI
(AGRI, HOSP and VETE). The SEC uses the outcome of the test to determine how best to
implement it for the remaining sectors.
In order to structure the integration and clean-up process, we group the individual process steps
of the around three themes (see also figure 6), i.e.
1. clean-up of the integrated occupations pillar
2. clean-up of the integrated skills and competences pillar
3. adding the translations
Process step for the clean-up of the integrated occupations pillar:
16. This is the step in which the occupation concepts that have relevancy across sectors are
identified, aligned and rationalised.
4 For details on this process see document ESCO (2015) SEC 026 Final.
Figure 6. The ESCO integration and clean-up process.
Process Steps for the clean-up of the integrated skill and competences pillar:
17. The SEC and the TEG make the vast amount of knowledge, skills and competence concepts
manageable by grouping concepts around common themes. The SEC will decide which
themes to use on the basis of the actual knowledge in the data.
18. The SEC and the TEG will use the common themes to enrich the knowledge structure with
associations (“same as”, “broader than”, “narrower than”, or “related”)
19. During this step, the effect of merging the KSC concepts, which are declared to be the
same by the “same as” associations, will be evaluated. The SEC will merge the identified
concepts in case no undesirable side effects occur.
20. The SEC will investigate and resolve the cases that were labelled to be complex during
previous steps.
21. Correction of the English terminology used in the content. This step can happen as soon as
the collection of concepts is stable, i.e. once the occupational overlaps are addressed, and
the KSC concepts are split. Since there are no dependencies of the knowledge structure
itself, the implementation can occur in a parallel process. However, in order to have the
corrected terms available in the master content, the SEC will organise that the corrected
terms are copied into the core content as soon as they are available.
22. Once the corrected English terms are available there exists a stable basis for all the
translations. These translations can happen per language and independently of the main
process. The SEC will organize that the translations are integrated in the main content set
as soon as they are available.
Project Planning, Quality Control and Monitoring
Project Planning
Based on the process defined and the constraint to have to deliver the ESCO v1 classification in
December 2016, we have set the target dates for the milestone versions Delta, Iota, Pi, Sigma and
Omega. The SEC will use the target dates as a reference for testing if the process steps of the
newly defined integration and clean-up process can be executed in time. The SEC will also use the
target dates as a basis to negotiate the target dates for the other versions.
Target dates for the Key Milestone Versions:
Milestone
version
Description Target date
Delta Sector draft full content ready for online
consultation
September 2015
Iota Refined sector full content, validated by the online
community, and assessed by the SEC
February 2016
Nu Sector content finalised (Evaluated by the MAI,
with issues resolved and KSC concepts normalised)
March 2016
Pi Cleaned occupations, KSC collections created, KSC
concepts linked, English corrected, first set of
translations integrated
August 2016*
Sigma KSC concepts merged, KSC complex cases
addressed, translations integrated
November 2016*
Omega ESCO V1 created, approved and published December 2016**
* Preliminary target dates; the team has no experience with the underlying process; Tests with currently
available content will be conducted to validate assumptions made.
** December 2016: the deadline set by the European Commission.
Quality Control and Monitoring of the ESCO Content Development
The SEC has created the procedures for quality control and monitoring for the drafting for the
sector content development for the remaining 16 sectors using the experience with developing
the content for the first 11 sectors. The SEC has captured the experiences in the guidelines as well
as in the elaborate list of KPIs. The KPIs will soon be available on a dashboard for the different
defined versions of the content sets. This dashboard updates the values of the KPIs daily and will
provide through the KPIs transparent feedback on various aspects of the content. The SEC and the
TEG will use the KPIs as a first indicator for a wide range of issues. Issues like e.g. missing preferred
terms (PT), missing references to ISCO codes, and unbalanced distribution of essential and
optional skills will surface immediately. As a consequence of using the dashboard, the SEC will
save time on checking the statistical aspects of the classification of a sector, and thus have more
time to focus on the semantics and the classification itself. For instance, based on experience, the
SEC knows that it has to check consistency between the description of an OCC and the actual KSCs
associated.
The SEC will also use the same information to update the MAI at their meetings. The dashboard
with the KPIs and notes will provide a basic structure to present and discuss progress and
challenges.
Content development dashboard
The SEC has defined the milestones for the sector content development process around the
maturity of the content. Each version reflects a certain level of maturity of the content of a sector
or for the full ESCO content set. The SEC will describe the defined versions in detail in the updated
ESCO guidelines. The dashboard captures the associated list of KPIs and provides access to a
document in which issues can be captured. The dashboard has also incorporated the target dates
for the various versions. Based on the target dates, the dashboard shows if a milestone is met in
time or not.
The SEC will configure the dashboard around the versions defined for different slices of the
content. The definition for a slice, during the phase in which the sector content is developed, is
directly based on the NACE codes associated with the sectors. Using this definition the sector
content development work will be presented in the dashboard using the sectors as rows and the
versions as columns. In a similar fashion, we will represent the milestones for the integration and
clean-up process. However, during this phase, we will use a different definitions for the slices
represented in the rows. The rows will represent the OCC, the KSC and all the languages for the
group of sectors being integrated. This ensures that the SEC can monitor progress aligned with the
different streams of activity.
Through the dashboard, the team has at all times access to all the key information on the issues
and the progress being made on the content. Progress is reflected by showing in each cell the
target date for a version in a heat map5. (A mock-up of the dashboard is provided in figure 7)
This dashboard will be the leading instrument to guide the TEG and the SEC through the process. It
supports through the KPIs and the notes associated with versions a solid means for collaboration
between the SEC, the TEG and other parties that have a role in the process. It will also support the
weekly SEC-TEG alignment meeting that helps the SEC and TEG to manage and align focus, quality,
issues and priority for the delivery of the ESCO content.
5 Colours will indicate if milestones have been met.
Figure 7. A mock-up of the dashboard currently being developed.
Annex A
Table with the list of ESCO Sectors