Document of The World Bank...06 25‐Oct‐2011 Unsatisfactory Unsatisfactory 9.42 07 19‐May‐2012 Moderately Satisfactory Moderately Satisfactory 10.99 08 07‐May‐2013 Moderately

Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: ICR00003743

IMPLEMENTATION COMPLETION AND RESULTS REPORT

(Loan Number 48490‐RU)

ON A

LOAN

IN THE AMOUNT OF US$50 MILLION

TO THE

RUSSIAN FEDERATION

FOR A

JUDICIAL REFORM SUPPORT PROJECT

August 30, 2018

Governance Global Practice

Europe And Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective December 31, 2017)

Currency Unit = RUR

RUR62.32 = US$1

FISCAL YEAR

July 1 ‐ June 30

Regional Vice President: Cyril E. Muller

Country Director: Andras Horvai

Senior Global Practice Director: Deborah L. Wetzel

Practice Manager: Roberto Adrian Senderowitsch

Task Team Leader: Amitabha Mukherjee

ICR Main Contributors:Amitabha Mukherjee, Olga Albertovna Schwartz, Rajni Khanna

ABBREVIATIONS AND ACRONYMS

AI BEA

Artificial Intelligence Bureau of Economic Analysis

BEEPS Business Environment and Enterprise Performance Survey CAS Country Assistance Strategy CC Constitutional Court of the Russian Federation CDP CEE CGJ

Customs Development Project Comprehensive Economic Effect Court of General Jurisdiction

CIDA Canadian International Development Agency CPS Country Partnership Strategy CQ Selection based on Consultants’ Qualifications DEE DO

Direct Economic Effect Development Objective

EC European Commission EU FER

European Union Foundation for Enterprise Restructuring

FMR Financial Management Report FPS FTAP FTP

Federal Penitentiary Service Fiscal Technical Assistance Project Federal Targeted Program

GTZ German Agency for Technical Cooperation IACC Inter-Agency Coordinating Council IAMC IBRD

Inter-Agency Management Council International Bank for Reconstruction and Development

ICB International Competitive Bidding ICR ICT

Implementation Completion and Results Report Information and Communications Technology

IFC International Finance Corporation IFR IPF

Interim Unaudited Financial Report Investment Project Financing

IRI JET

Intermediate Result Indicator Judicial Education and Training

JP Justice(s) of the Peace JRSP Judicial Reform Support Project LCS Least Cost Selection LRP Legal Reform Project MCC MOED MOEDT

Moscow City Courts Ministry of Economic Development of the Russian Federation Ministry of Economic Development and Trade of the Russian Federation

MOF Ministry of Finance of the Russian Federation MOJ Ministry of Justice of the Russian Federation NCB National Competitive Bidding NDB OE

New Development Bank Overall Effect

PIU Project Implementation Unit PPP QCBS

Public Private Partnership Quality and Cost Based Selection

RF Russian Federation RFP Request for Proposals RSFSR SAC

Russian Soviet Federated Socialist Republic Supreme Arbitration Court

SBD Standard Bidding Document SC Supreme Court of the Russian Federation SIL Specific Investment Loan SSS Single Source Selection TAMP2 TMP USAID VCIOM WB WBG

Second Tax Administration Modernization Project Treasury Modernization Project United States Agency for International Development Vserossiiski Centr Izucheniya Obschestvennogo Mnenia (Russia Public Opinion Research Center) World Bank World Bank Group

TABLE OF CONTENTS

DATA SHEET ............................................................................................................................ 1

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ........................................................ 9

A. CONTEXT AT APPRAISAL ........................................................................................................... 9

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ...................................... 14

II. OUTCOME ...................................................................................................................... 18

A. RELEVANCE OF PDOs .............................................................................................................. 19

B. ACHIEVEMENT OF PDOs (EFFICACY) ........................................................................................ 20

C. EFFICIENCY ............................................................................................................................. 26

D. JUSTIFICATION OF OVERALL OUTCOME RATING ..................................................................... 30

E. OTHER OUTCOMES AND IMPACTS .......................................................................................... 31

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 32

A. KEY FACTORS DURING PREPARATION..................................................................................... 32

B. KEY FACTORS DURING IMPLEMENTATION .............................................................................. 35

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 38

A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................. 38

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 39

C. BANK PERFORMANCE ............................................................................................................. 40

D. RISK TO DEVELOPMENT OUTCOME ........................................................................................ 41

V. LESSONS AND RECOMMENDATIONS .............................................................................. 41

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ............................................................ 43

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 59

ANNEX 3. PROJECT COST BY COMPONENT............................................................................. 62

ANNEX 4. EFFICIENCY ANALYSIS ............................................................................................ 63

ANNEX 5. BORROWER COMMENTS ....................................................................................... 79

ANNEX 6. OVERVIEW OF KEY JUDICIAL AND LEGAL REFORMS ............................................... 84

ANNEX 7. JRSP SURVEYS – SUMMARY OF KEY RESULTS ......................................................... 91

ANNEX 8. 2017 MOSCOW CITY COURTS SURVEYS – SUMMARY ........................................... 105

ANNEX 9. MOSCOW CITY COURTS AUTOMATED SYSTEMS ‐ KEY FEATURES AND RESULTS .. 115

ANNEX 10. SUPREME ARBITRAZH COURT & COMMERCIAL COURTS ICT SYSTEM ................. 123

ANNEX 11. SUPPORTING DOCUMENTS ................................................................................ 133

ANNEX 12. SUMMARY OF RUSSIAN FEDERATION’S ICR ....................................................... 134

The World Bank Judicial Reform Support Project (P089733)

Page 1 of 148

DATA SHEET

BASIC INFORMATION Product Information

Project ID Project Name

P089733 Judicial Reform Support Project

Country Financing Instrument

Russian Federation Investment Project Financing

Original EA Category Revised EA Category

Not Required (C) Not Required (C)

Organizations

Borrower Implementing Agency

Ministry of Finance, Moscow City Courts, Supreme

Court of the Russian Federation

Ministry of Economic Development, Foundation for

Enterprise Restructuring and Financial Institutions

Development

Project Development Objective (PDO) Original PDO

The project development objective is to strengthen judicial transparency and efficiency in courts financed by the JRSP.


Page 2 of 148

FINANCING

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$)

World Bank Financing IBRD‐48490

50,000,000 50,000,000 49,902,853

TF‐90655

1,896,800 1,895,592 1,895,592

Total 51,896,800 51,895,592 51,798,445

Non‐World Bank Financing

Borrower 122,414,000 138,867,800 138,857,600

Total 122,414,000 138,867,800 138,857,600

Total Project Cost 174,310,800 190,763,392 190,656,045

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing

15‐Feb‐2007 15‐Nov‐2007 20‐Sep‐2010 30‐Mar‐2012 31‐Dec‐2017


Page 3 of 148

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions

18‐Sep‐2010 5.71 Other Change(s)

26‐Mar‐2012 10.54 Change in Loan Closing Date(s)

30‐May‐2012 14.24 Change in Results Framework Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Legal Covenants Change in Institutional Arrangements Change in Implementation Schedule

27‐Mar‐2014 22.19 Change in Loan Closing Date(s)

12‐Sep‐2014 24.17 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Legal Covenants Change in Implementation Schedule Other Change(s)

03‐Oct‐2016 41.57 Change in Loan Closing Date(s)

31‐May‐2017 46.68 Change in Loan Closing Date(s) Reallocation between Disbursement Categories

KEY RATINGS

Outcome Bank Performance M&E Quality

Satisfactory Moderately Satisfactory Substantial

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 19‐Jun‐2007 Satisfactory Satisfactory 0

02 30‐Jun‐2008 Satisfactory Satisfactory 2.74

03 29‐Jun‐2009 Unsatisfactory Unsatisfactory 4.22

04 24‐Feb‐2010 Unsatisfactory Unsatisfactory 4.87

05 23‐Oct‐2011 Unsatisfactory Unsatisfactory 9.42


Page 4 of 148

06 25‐Oct‐2011 Unsatisfactory Unsatisfactory 9.42

07 19‐May‐2012 Moderately Satisfactory Moderately Satisfactory 10.99

08 07‐May‐2013 Moderately Satisfactory Moderately Satisfactory 20.58

09 28‐Dec‐2013 Moderately Satisfactory Moderately Satisfactory 20.69

10 18‐Jun‐2014 Moderately Satisfactory Moderately Satisfactory 23.21

11 23‐Jan‐2015 Satisfactory Satisfactory 25.60

12 23‐Sep‐2015 Satisfactory Satisfactory 27.85

13 04‐Apr‐2016 Satisfactory Satisfactory 34.90

14 16‐Nov‐2016 Satisfactory Satisfactory 41.97

15 15‐May‐2017 Satisfactory Satisfactory 46.68

16 22‐Dec‐2017 Satisfactory Satisfactory 50.00

SECTORS AND THEMES

Sectors

Major Sector/Sector (%)

Public Administration 100

Law and Justice 100

Themes

Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Public Sector Management 99

Rule of Law 99

Judicial and other Dispute Resolution Mechanisms

53

Legal Institutions for a Market Economy 33

Personal and Property Rights 13

ADM STAFF

Role At Approval At ICR

Regional Vice President: Shigeo Katsu Cyril E Muller


Page 5 of 148

Country Director: Kristalina I. Georgieva Andras Horvai

Senior Global Practice Director: Cheryl W. Gray Deborah L. Wetzel

Practice Manager: Ronald E. Myers Roberto Adrian Senderowitsch

Task Team Leader(s): Amitabha Mukherjee Amitabha Mukherjee

ICR Contributing Author: Olga Albertovna Schwartz


Page 6 of 148

EXECUTIVE SUMMARY

The World Bank Group’s 2007-2009 Country Partnership Strategy (CPS) for the Russian Federation (RF) addressed key aspects of Russia’s modernization strategy. The CPS supported enhanced transparency and increased public dissemination of information through strategic improvements in public sector management and governance, including judicial reform. The Judicial Reform Support Project (JRSP) was fully aligned with this strategy and vision. The authorities cited the Bank’s international experience in judicial reform and its added value in the development of coherent judicial modernization programs. The project was regarded as a first step in a longer-term judicial modernization process. The project development objectives (which remained unchanged) were to strengthen judicial transparency and efficiency in courts financed by the JRSP. The project had four components: (a) Institutionalizing Judicial Transparency and Accountability; (b) Harnessing ICT for Judicial Transparency and Effectiveness; (c) Strengthening Human Capital; and (d) Project Management and monitoring and evaluation (M&E). The PDOs were not revised because their continued relevance was reconfirmed to Bank management by the judicial leadership, MOE and MOF all through implementation, at the mid-term review (MTR) and at each restructuring. The original total project cost was US$172.4 million, comprising US$50 million from IBRD (29 percent) and US$122.4 million (or 71 percent) from the Russian Federation (RF). In 2017, the RF added almost US$19 million to the project, boosting the total project cost to US$188.86 million (of which IBRD comprised 26.5 percent). By project closing, US$188.76 million had disbursed, including US$49.90 million from the Loan proceeds. The overall outcome is rated Satisfactory. The relevance of PDOs and Efficacy are rated Substantial, while Efficiency is rated High. The quality of Monitoring and Evaluation (M&E) is rated Substantial while Bank Performance is rated Moderately Satisfactory. The detailed Efficiency Analysis is at Annex 4, while Annexes 7-12 provide information and context about activities, results and achievements. The ICR ratings are summarized below:

Relevance of PDOs

Efficacy Efficiency Overall Outcome

Quality of M&E

Bank Performance

Substantial Substantial High Satisfactory Substantial Moderately Satisfactory

While component names were not revised, some activities were added or deleted at the 2012 and 2014 restructurings to respond to evolving RF judicial reorganization and modernization priorities and address JRSP implementation challenges. These restructurings (i) updated the scope of Component B: the Moscow City Courts (MCC) and Court of Intellectual Rights (CIR) were added as project beneficiaries, and unimplemented Judicial Department (JD) activities were cancelled and (ii) other activities were added to enable the Supreme Court (SC) to review the results of the modernization and plan an information and communications technology (ICT) strategy for the future which could in future accommodate the judiciary’s vision to move towards a single integrated judicial ICT system. A new Project Implementation Unit (PIU) – the Foundation for Enterprise Restructuring (FER) - was engaged with effect from December 2011, in place of the original PIU. FER was instrumental in turning the project around by working closely with all courts covered by the project, and with the Ministry of Economic Development (MOED – the


Page 7 of 148

implementing agency), the Ministry of Finance (MOF) and the World Bank. The inclusion of the MCC at the 2014 restructuring increased the strategic relevance of the project and the PDOs. The MCC is Russia’s largest regional court system, with 35 district courts in Moscow City, with the Moscow City Court (with more than 500 judges – the largest in Russia and in Europe) at the top. The MCC court system annually handled more than twice the caseload of the 900 courts of general jurisdiction (CGJ) originally included in the project, significantly increasing the JRSP footprint and impact. Several factors affected preparation and implementation. Positive factors included realistic objectives; strong supervision and implementation support; high counterpart capacity; strong commitment of the judicial and executive leadership to the project and a highly professional PIU from December 2012 which helped turn the project around; and readiness on the part of the Bank and the RF to restructure the project to address evolving circumstances and take deft advantage of opportunities to achieve project objectives and results. Risk factors included complex project design (due to the RF judiciary having three highest-level courts at the time); several beneficiary entities within the judiciary; shortcomings of the first PIU in stakeholder engagement, procurement and results reporting (which delayed implementation of some – but not all – activities); and some results framework weaknesses. The overall quality of M&E is rated Substantial. There were significant strengths in M&E design, implementation and utilization, but shortcomings in the articulation of some results indicators and the first PIU’s inability to obtain results information in a timely manner somewhat diluted overall M&E performance. These issues were addressed at the 2012 restructuring and M&E finished strongly. In addition, significant elements of the JRSP M&E framework are being used by the MCC even after project closing to track its own performance. Bank performance is rated Moderately Satisfactory (MS). The MS rating reflects some shortcomings at entry on Results Indicators and implementation arrangements, which were overcome during implementation. The Bank team understood and noted the project’s high risks and the potential high rewards. The quality of supervision is rated Satisfactory because the stable and technically strong Bank team worked closely with courts and executive agencies (Ministries of Economy and Finance) to support implementation, help counterparts restructure the project to restore its achievements and results, and provide technical support all through to the courts covered by the project. The Borrower’s ICR, received in July 2018, was quite detailed and comprehensive and, most importantly, reinforced the Bank team’s analysis on project performance. The project design and implementation experience generated four sets of lessons:

a) Commitment works when coupled with the creation of conditions for success. JRSP achievements stemmed from a combination of top-level commitment from the executive and judicial leadership, technical-level commitment from the judges and staff of the courts financed and high technical competence in Borrower and Bank teams. These together created an environment in which sequenced change, underpinned by the transformative power of ICT, could be induced, nurtured and sustained. A key element of such commitment included shedding lagging counterparts and the original PIU, the sequencing and phasing of ICT implementation to build up commitment, and addressing institutional


Page 8 of 148

issues beyond ICT hardware and software (e.g. handling the cessation of SAC functioning and the changes to the structure of the RF judiciary, the induction of the MCC and the exit of the JD). The Bank trusted the commitment of the judicial and executive leadership. Rushing significant institutional change in courts in a country as complex as Russia would have created obstacles to change and led to the convergence of interests opposed to modernization. Discussions on how to use restructuring to effect change continued during implementation, and more importantly through peer-based knowledge-sharing events (e.g. JUSTPAL conferences) and study visits (e.g. to European and US courts). Where the authorities’ commitment is clear, such an approach can yield positive and sustainable results.

b) Complementary reforms and change management increase the impact of institutional reforms. For example, as the survey findings demonstrate, in addition to improving court-wide ICT infrastructure and systems for the SC, SAC, CC and MCC, it was critical to include actions to support collaborative activities for different groups – judges, judicial staff, academia, experts of methodological centers, and other users of the court system – whose common interests in enhancing judicial transparency and efficiency could converge around specific reforms (e.g. publication of judicial decisions, or automated complaints handling mechanisms). Such convergence through, for example, peer-based knowledge-sharing (e.g. peer-to-peer learning from other judiciaries and JUSTPAL) facilitated mindset changes and boosted collaboration. In addition, in ICT-intensive projects, managing change over the medium term is as important as technical competence. This project proved - again - that ICT in institutional modernization is more than installing computers and software: they are one element of modernization and can have a systemic impact only if the content of business processes and the organization of courts into cooperating networks of service delivery institutions can generate efficiency, transparency and trust. This is the largest Bank-financed project in ICT-intensive judicial modernization: it has played a significant role in pioneering such an approach.

c) Successful teams on both sides, which generate trust and promote smooth implementation in complex high-risk operations, should be maintained regardless of organizational shifts or requirements in the Bank or the government. This is because stable and successful teams on the Borrower and Bank sides can together address implementation risks and challenges, as the Borrower’s ICR points out. The Bank team remained constant throughout the project life – an unusual feature that contributed to meaningful support to project beneficiaries and a trusted relationship between the Bank and Russia’s courts, the PIU, MOE and MOF. The Borrower’s ICR also makes this point. In addition, the Bank team’s candid reporting, with ratings downgrades, helped counterparts turn the JRSP around.

d) This project is a good example of the value-added the Bank brings to middle-income countries. The loan amount constituted a very small element of successive RF Federal Targeted Programs (FTPs) for judicial modernization. The project strategy was aligned with the RF judicial modernization strategy. The Bank’s main role was to provide implementation support, policy advice, technical feedback and a menu of alternative approaches for beneficiaries. The JRSP experience reconfirmed that counterparts were more interested in the intellectual support, exchange of knowledge and the implementation discipline associated with a Bank-financed project, rather than in the financing itself. The courts used the JRSP to access Bank support (through the know-how that comes bundled with Bank financing) to conceive, design and implement complex, high-risk, ICT-intensive systems. This is also evident more broadly in the Bank’s relationship with other middle-income countries, where fee-based services and knowledge exchange are becoming increasingly important.


Page 9 of 148

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL

1 The Judicial Reform Support Project (JRSP) became the fifth public sector modernization project in the RF supported by the World Bank. Four projects – the Second Tax Administration Modernization Project (TAMP2), the Customs Development Project (CDP), the Treasury Modernization Project (TMP) and the Fiscal Technical Assistance Project (FTAP) – were under implementation. The JRSP followed the Bank-financed Legal Reform Project (LRP), which closed on December 31, 2005. 2 The February 15, 2005 CAS Progress Report included the JRSP as an investment loan to address the CAS priorities of strengthening public sector governance and service delivery and improving the business environment. 3 The Constitution specifies an independent judiciary (Article 10) and guarantees judges’ independence and subordinates them only to the Constitution and federal laws (Article 120). 4 The LRP, which closed December 31, 2005 supported, among other things, the drafting of several key federal laws. 5 For example, the Ministry of Economic Development and Trade estimated in 2004 that for CGJ buildings, 719 (or 34 percent) were unfit for use while 162 (or 7.8 percent) needed repairs.

Context

1. The World Bank Group’s 2007-2009 Country Partnership Strategy (CPS) for the RF addressed key aspects of Russia’s modernization strategy. It focused on selected areas where it felt it could provide solutions complementing those of the country. The CPS was grouped around four pillars: (a) economic diversification for sustainable development and growth (Pillar 1); (b) improved public sector management and governance including stepping up engagement in judicial reform1 (Pillar 2); (c) improving the delivery of social and communal services (Pillar 3); and (d) enhancing Russia’s global role, as a donor and as a producer of global public goods (Pillar 4). The CPS supported enhanced transparency and increased public dissemination of information through strategic improvements in public sector management and governance, including judicial reform: the JRSP was fully aligned with this strategy and vision2.

2. By 2006, the RF had undertaken significant legal and judicial modernization steps.3 Substantive and procedural laws had been enacted4. Trial by jury was re-introduced. A trio of judicial structures, employing more than 120,000 personnel, comprised the RF court system in 2006: the Supreme Court (SC) overseeing Courts of General Jurisdiction (CGJs) and military courts; the Supreme Arbitration Court (SAC) overseeing arbitrazh (commercial) courts; and the Constitutional Court (CC). A RUR45 billion Federal Targeted Program (FTP) for the judiciary for 2002-2006 had financed substantial increases in judges’ salaries, new positions (including 2,500 court administrators and 14,000 judicial assistants), courthouse renovations and equipment. The justice system at appraisal is depicted at Box 1.

3. Court caseloads were rising. The 2005 CGJ annual case load comprised about 6 million civil cases, 1 million criminal cases and 4 million “administrative” cases (petty offences). The commercial court system heard 1.5 million cases annually; case inflows were rising rapidly as economic activity expanded - cases filed with arbitration courts nearly doubled between 1995 and 2000. New laws and amendments were adding to the commercial court case inflow. The use of court information and communications technology (ICT) to improve efficiency and transparency was evolving, but many courthouses were in unsatisfactory condition and some courts had no premises of their own5.


Page 10 of 148

4. The Judicial Department (JD - see Box 1) is responsible for oversight of, and provision of ICT and physical infrastructure, for all CGJs. It was created by spinning off these functions from the Ministry of Justice (MOJ) in 1998, so that courts no longer had to depend on the executive branch for operational and capital needs. A key actor in the RF judicial system, the JD’s 2005 budget was about US$1 billion, growing to about US$6 billion by 2017. It is responsible for all CGJ recurrent and capital expenditures, and for oversight of CGJs’ performance. Its annual work program is approved by the Council of Judges, the highest RF judicial self-governance body. The JD vision was for the JRSP to finance the roll-out and implementation of court automation systems to 900 out of 2,500 CGJs.

5. Several RF programs at the time addressed judicial constraints, the most significant being the 2002-2006 FTP and its successor 2007-2011 FTP. The 2007-2011 FTP had a budget of RUR48.4 billion. Among other things, it aimed to expand courts’ use of ICT, including the creation of websites for CGJ and arbitration courts for publishing judicial decisions and other public information. The FTP envisaged preparation of new legislation to increase transparency and judicial accountability. The FTP also financed security of court premises and housing for judges. At the time, some development partners financed programs to support different elements of judicial reforms, mainly training of judges and some pilot ICT systems in a few first-instance courts. In 2005, the JD awarded a contract to a state-owned enterprise (NII Voskhod) to develop and pilot an ambitious ICT system envisaged to be deployed to all CGJs across the RF. The system, called GAS Pravosudiye (state automated system for justice), would comprise, when

Box 1. Justice System of the Russian Federation

C ourts of G eneral JurisdictionCourts of Arbitration InstitutionsExec + Leg Branches

President

Parliam ent

LA W S Constitutional Court: Federal Law “On the Constitutional Court in the RF” – 1994 General Courts: C ivil Procedure Code of the RF - 2003

Crim inal Procedure Code of the RF - 1996Federal Law “On Justices of the Peace in the RF” – 1998

Courts of Arbitration: Arbitration Procedure Code of the RF - 2002

Institutions: Federal Law “On Office of the Public Prosecutor in the RF” – 1992 Notes:Regulations of the M inistry of Justice in the RF – 1999 M ilitary courts are included in the courts of general jurisdiction.Regulations of the M inistry of the Interior in the RF – 1996 The Judicial Departm ent (JD) oversees functioning of the courts Regulations of the Federal Security Services in the RF – 2003 of general jurisdiction & provides m aterial and technical support.

Governm ent D istrict courts*

Justices of the Peace

SC

Oblast Courts*

C ourt of appeals

A rbitration court

SAC

Circuit court

Ministry of Justice

Constitutional Court

C onstitutional Court

General Prosecution Service

Ministry of the Interior

Federal Security Services

Human Rights Commission of the

President

Ombudsm an

law yers, attorneys, barristers, solicitors, students, post-graduate students, notaries

Federal law “O n Justice system in the RF” – 1998

JD


Page 11 of 148

completed, an integrated suite of 27 applications for case management, court administration and all other judicial and ‘housekeeping’ (e.g. human resources and budgeting) functions of courts. The 2007-2011 FTP continued financing for GAS Pravosudiye implementation, except for the 900 CGJs where JRSP resources would finance implementation.

6. Rationale for Bank involvement. The authorities cited the Bank’s international experience in judicial reform and its added value in the development of coherent judicial modernization programs. The project was regarded as a first step in a longer-term judicial modernization process. The JRSP was envisaged to contribute to increased transparency and efficiency in court functioning and help improve the business environment by increasing the efficiency of the arbitration court and CGJ systems.

Theory of Change (Results Chain)

7. The PAD did not have a results chain as this was not required at the time. Nevertheless, the PAD is clear enough for the results chain to be constructed (Figure 1). The PDO has been unpacked into its 2 elements and the links to PDO indicators, IRIs and component/activities depicted. Some critical assumptions were made, such as: (a) the courts had the capacity to design and implement technically difficult ICT-based reforms - this held true for all courts, especially the SC, MCC and SAC which implemented technically and institutionally challenging ICT-based reforms; (b) the MOEDT (later MOE) was able and willing to coordinate JRSP implementation as the implementing agency responsible for use of JRSP loan proceeds – this assumption held true; (c) the courts would collaborate closely with the Bank on the design and implementation of JRSP activities – this held true for all courts. The abolition of the SAC in 2014 did not affect the reasoning underlying the results chain since SAC functions were transferred to the SC.

8. The project results chain focused on two project-level outcomes – transparency and accountability – in courts financed by the JRSP; the longer-term outcome sought was increased public trust and confidence in courts. Transparency was improved through several activities supported by the project, such as: (i) drafting a law to mandate publication of all judicial decisions, (ii) information kiosks and screens in public-access locations in courts financed, to enable citizens, litigants and lawyers to access their cases, obtain information about court functioning in general and about the specific court in particular, (iii) inclusion of transparency-enhancing elements in the MCC ICT system (such as reporting on complaints received and resolved). The enactment of Law 262 (mandating publication of judicial decisions) greatly improved transparency, with court ICT systems becoming the constraining factor (because the existing ICT systems did not have the capacity and bandwidth to handle such massive amounts of data, until JRSP-financed systems addressed this constraint). With this enactment, the original transparency outcome was met early in the project: the original PDO indicator was therefore deleted at restructuring and replaced with a new one (see Table 1). In addition, private sector and other actors could now access information on cases, while survey questions on transparency provided more information on transparency. Efficiency was mainly improved through streamlining business processes and automating them through ICT systems in the SC, SAC, CC, MCC and the commercial courts. Almost US$169 million of the US$189 million disbursed was invested in such activities, greatly enhancing efficiency, as the economic analysis demonstrates.


Page 12 of 148

Figure 1. JRSP Results Chain

Components/

ActivitiesIRIs PDO level indicators PDO Components

Longer Term

Outcomes

A. Surveys

PDO Indicator 1. Availability of data from periodic surveys of users

of judicial services

B. Information services for users (kiosks, video-

conferencing, etc)

Reduction in complaints by

litigants in MCC

Reduction in cases with

contested court records, as a

percentage of all cases in MCC

A modern Data Processing Center

for Moscow courts is

functional at the MCC

Time taken by Moscow lower

courts to forward criminal and civil cases to courts of

appeal

Component C - training activities

Number of Manpower

Trained under the Project

Incr

ease

d pu

blic

trus

t and

con

fiden

ce in

cou

rtsB. ICT system for

MCC

PDO Indicator 2. Decisions of MCC in civil

cases enforced, as a percentage of all decisions by MCC in civil

cases

PDO indicator 4.

New video‐

conferencing

facilities (to

enable remote

participation in

court

proceedings)

are functional in

courts financed

by the project

PDO Indicator 3. Reduction in time taken by

Moscow courts to respond to

citizen applications

1. T

o st

reng

then

ju

dici

al

trans

pare

ncy

in

cour

ts fi

nanc

ed b

y th

e JR

SP

2. To streng

then

efficien

cy in

cou

rts fin

anced by

the JRSP

B. ICT systems for SC, CC, SAC, AC,

MCC, CIR

Percent of courts financed by the JRSP in

which new information services are available for

users


Page 13 of 148

Project Development Objectives (PDOs)

9. The original project development objectives were to strengthen judicial transparency and efficiency in courts financed by the JRSP.

Key Expected Outcomes and Outcome Indicators

10. The two key expected JRSP outcomes were to: a) Strengthen judicial transparency in courts financed by the JRSP, measured by 2 outcome indicators:

i. Periodic surveys show improved private sector and public ratings for judicial transparency and efficiency;

ii. Judicial decisions in all arbitrazh courts, all courts of general jurisdiction financed by the JRSP, the Constitutional Court and the Supreme Court are routinely published and publicly accessible.

b) Strengthen judicial efficiency in courts financed by the JRSP, measured by the outcome indicator: i. Periodic surveys of judges and staff in courts covered by the JRSP show increasing satisfaction with

use of new information systems compared to the 2006 baseline.

11. The first PDO indicator was meant to track a longer-term outcome – increased user satisfaction with transparency in judicial functioning, while the other PDO indicators and intermediate indicators were meant to measure project-specific results from the ICT modernization of JRSP-financed courts. Together, the indicators covered activities valued at more than 90 percent of the actual total project cost of US$188.76 million (project costs by component are at Annex 3).

Components

12. The JRSP financed four components whose estimated and actual costs (indicated below) add up to the estimated and actual total project costs:

A. Institutionalizing Judicial Transparency and Accountability (total estimated cost US$7.15 million, actual cost US$4.78 million). This component financed: (i) periodic surveys of users of judicial services such as individuals, businesses and professional users (e.g. lawyers, bailiffs, and notaries) and dissemination of survey results; (ii) research and analysis on development of transparency, openness and accessibility of judicial decisions, processes and practices; (iii) the creation and implementation of common case management standards, guidelines and processes; (iv) the development and piloting of judiciary performance criteria and indicators; and (v) development and implementation of communications and change management strategies for judicial reform.

B. Harnessing ICT for Judicial Transparency and Effectiveness (total estimated cost US$146.10 million, actual cost US$168.76 million). This financed activities to: (i) modernize, develop, and deploy integrated information systems for courts to facilitate document flow, record management, information collection, and internal knowledge sharing; (ii) improve public access to and availability of judicial information through online publication of judicial decisions and other relevant information; and (iii) more effectively link the SC, the SAC, lower courts, JD offices, and other relevant justice entities (such as prisons) through integrated information systems and video-conferencing.

C. Strengthening Human Capital (total estimated cost US$10.10 million, actual cost US$7.09 million). This component financed: (i) information technology-related education and training for judges and court


Page 14 of 148

personnel for the Constitutional Court, the CGJ, the SAC and the Judicial Department; and (ii) knowledge exchange, including seminars and workshops for the judiciary.

D. Project Management and M&E (total estimated cost US$9.06 million, actual cost US$8.13 million). This component financed: (i) logistical and secretarial support for the Inter-Agency Coordination Council (IACC – later renamed the Inter-Agency Management Council or IAMC) and the operating costs of the project implementation unit (PIU) – the Bureau of Economic Analysis (BEA); (ii) expert technical ICT support for the judiciary; and (iii) implementation of the project results framework.

Figure 2 depicts the authorities’ overall strategy to modernize Russia’s courts, using a mix of instruments and stakeholder coalitions, and the stakeholder coalitions developed during JRSP implementation.

Figure 2. Modernizing Russia’s Courts – Strategy and Stakeholder Coalitions

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE)

Revised PDOs and Outcome Targets

13. The PDOs were not revised because the continued relevance of the original PDO was reconfirmed to Bank management by the judicial leadership, MOE and MOF all through implementation, at the mid-term review and at each restructuring. The table below shows the changes made to outcome targets and PDO indicators at the 2012 restructuring (when US$14.24 million of the IBRD loan had disbursed) and the 2014 restructuring (when US$24.17 million had disbursed).

Revised PDO Indicators

Table 1. Revisions to PDO Indicators and Outcome Targets

Original PDO Indicator and

outcome target

Revised PDO Indicator and

Outcome Target

Date of revision/

drop

Reason for Change

Component A


Page 15 of 148

1. Periodic surveys show improved private sector and public ratings for judicial transparency and efficiency.

Outcome target: Increased satisfaction with judiciary, measured by 75% increase over baseline figure.

Availability of data from periodic surveys of users of judicial services

Outcome target: Yes

Revised May 30, 2012

The baseline for the original outcome target was derived from the 2010 user survey.

(Please see paragraphs 25-29 of the ICR and Annexes 7 and 8 for key survey findings)

Component B

2. Judicial decisions in all arbitrazh courts, all courts of general jurisdiction financed by the JRSP, the Constitutional Court and the Supreme Court are routinely published and publicly accessible.

Outcome target: 100% of above courts publish their decisions

Dropped

Dropped May 30, 2012

The original indicator was adopted when Russia’s courts were not required to publish their decisions. A few courts did so; the vast majority did not. This changed with the passage of law no. 262 of 2010: from July 1, 2012 all courts were legally required to publish all decisions. Hence at the 2012 restructuring the indicator as originally worded became moot: the Bank’s support to the drafting of law 262 contributed to the mandatory publication and public accessibility of judicial decisions, the SC and CC were publishing all their decisions, and the indicator was considered to have been substantially achieved by removing the legal impediment that most impeded publication: the only constraint to publication became courts’ lack of ICT systems, infrastructure and bandwidth to handle high data volumes and storage. Hence this original indicator was dropped. At the time of the drop, most CGJs (which comprised about 80 percent of the total number of RF courts) were unable to publish their decisions only because of ICT constraints (e.g. lack of software to address privacy issues, lack of ICT systems and equipment, and insufficient ICT bandwidth).

Decisions of Moscow City Courts in civil cases enforced, as a percentage of all

Added September 12, 2014

This indicator measures a key element of judicial efficiency, namely the extent to which


Page 16 of 148

decisions by Moscow City Courts in civil cases

Outcome target: 40 percent

the MCC’s judicial decisions in civil cases are actually enforced.

Reduction in time taken by Moscow courts to respond to citizen applications, as a percentage of 2016 baseline



This indicator was added to measure the efficiency of response to citizen applications addressed to the MCC. In 2014, Moscow courts took 30 days on average to respond to citizen applications, including complaints. This was targeted to be reduced to 24 days (i.e. 20 percent reduction) by the end of the project. Reports on this indicator are part of the automated reports generated by the JRSP-financed MCC ICT system.

New video-conferencing facilities (to enable remote participation in court proceedings) are functional in courts financed by the project



This indicator was originally subsumed under the ICT applications for each court system and hence covered the SC, CC, SAC, CGJs and the commercial court system. At the 2014 restructuring, with the exit of the JD and the CGJs from the JRSP, the SAC having been abolished, and the MCC added to the project, this indicator covered the SC, the commercial courts and the MCC system.

Component C

3. Training of judges, court personnel and Judicial Department demonstrates increased technological proficiency.

Outcome target: 5,000 judges and 5,000 judicial staff trained

Dropped

Dropped September 12, 2014

By the 2014 restructuring, much of the component C budget had been spent and about 3,000 judges and staff trained. The JD and CGJs were exiting the project. It was agreed to drop this indicator and include new ones that would capture the results of the ICT modernization of the MCC.


Page 17 of 148

Revised Components

14. The original component names were not revised; some activities within components were added or deleted at the May 2012 restructuring to respond to evolving RF judicial reorganization and modernization priorities and address JRSP implementation challenges:

a) Activities were added to reflect major changes in the organization and functioning of Russia’s courts: The passage of landmark Law no. 262 of 2010 mandated all courts to publish their decisions online – an

activity was added to Component A on research and analysis on publication of judicial decisions. In 2014 the SAC ceased to function as a result of a Constitutional amendment and associated laws; its

functions were transferred to a newly-created Commercial Division in the SC. The commercial court system below the SAC remained in place, with the SC becoming the apex court in place of the SAC. This was a major structural change in the RF judiciary. It was recognized in the Loan Agreement and an activity added to analyze the impact of the abolition of the SAC on commercial courts’ processes and ICT systems.

b) A new covenant (a “circuit-breaker”) was added to mitigate implementation risks: it provided that (A) the RF, through the PIU, should: (i) by not later than October 31, 2015 or such other agreed date, sign the contracts for remaining procurement of goods with contract duration of 12 months or more, and (ii) by not later than June 30, 2016 or such other agreed date, sign all other contracts for activities included in the procurement plan agreed upon between the Borrower and the Bank; and (B) If the project failed to sign any of the contracts referred to above by the respective due dates, the Bank might, at its discretion, determine that the amount of the Loan pertaining to the unsigned contracts would not be required and, without further notice, cancel such amount from the Loan Account. 15. At the 2014 restructuring: The scope of Component B was updated: the MCC and Court of Intellectual Rights (CIR) were added as

project beneficiaries along with relevant activities and the remaining unimplemented JD and CGJ activities were deleted; and

Other activities were added to enable the SC to review the results of the modernization and plan an ICT strategy for the future which could – in the right circumstances – accommodate the vision of the MCC and other courts to move towards a next-generation suite of ICT applications that, together, could integrate into a single judicial ICT system.

16. The 2014 restructuring increased the strategic relevance of the PDOs. The JD sub-component to modernize the ICT systems of 900 CGJs (estimated cost US$42 million) was cancelled at the 2014 restructuring. The MCC was added – the JRSP was to finance the design and implementation of a next-generation suite of ICT applications for the MCC. The substitution of the JD and the 900 CGJs by the MCC was significant. The 900 CGJs designated by the JD for investment under the JRSP comprised about a third of the total number of RF general jurisdiction courts. Although large in number, they were scattered across Russia’s territory and mostly comprised small courts with one, two or three judges. Their total caseload was less than 10 percent of the total RF caseload volume. The Bank team had expressed concerns on different occasions about (a) the Bank’s ability to verify implementation across so many locations with a limited supervision budget and (b) implementation issues with the JD-developed GAS Pravosudiye System intended for implementation in the 900 CGJs. The MCC was Russia’s largest regional court system, with 35 district courts in Moscow City, with the Moscow City Courts (with more than 500 judges – the largest in Russia and in Europe) at the top. The MCC court system annually handled more than twice the 900 CGJs’ caseload,


Page 18 of 148

increasing the JRSP footprint and impact. The MCC wanted to partner with the Bank to design and implement a next generation suite of applications integrated into a single ICT system, which could serve as a pilot to improve the GAS Pravosudiye system. The high MCC caseload volume (about 15 percent of the total RF caseload volume at the time) made this change a potentially high-impact one. In addition, the strong MCC leadership team, the high-capacity MCC ICT team and the compact MCC territory made it likely that this ambitious approach could work. Hence, while the decision to cancel the investment in the 900 CGJs could have seemed like a lost opportunity, adding the MCC and the CIR made the PDO more relevant and JRSP efficiency and impact higher (see also discussion on this issue under Efficiency).

Other Changes

17. At the May 2012 restructuring other changes included: (a) recognition of the Foundation for Enterprise Restructuring (FER) as the new JRSP PIU, (b) preparation and adoption of a revised Operational Manual by November 30, 2014 as a result; and (c) reallocation of loan proceeds so that the FER began its role as the PIU with a clean slate: the costs incurred by the BEA till the date of termination of its contract were frozen (at US$726,248.06) and depicted separately.

18. Other restructurings summarized in the Data Sheet extended the closing date, revised implementation timelines and reallocated loan proceeds to take account of changed circumstances on the ground.

Rationale for Changes and Their Implication on the Original Theory of Change

19. The rationale for dropping the JD and the 900 CGJs has been explained earlier, and more detail is in the Annex. The addition of the MCC and CIR more than compensated for this, and increased project ambition and potential achievement due to the very heavy MCC caseload. And it allowed the MCC to design, test and deploy an improved ICT system that began to strengthen transparency, efficiency and trust – and possibly lead to an improved court ICT system across Russia. These changes therefore did not affect the original theory of change. Upon the cessation of functioning of the SAC in 2014, its functions were transferred to the SC which became the apex court for the commercial courts – hence this change did not affect the original theory of change. It actually strengthened the theory of change because with one less apex court out of the original three, the SC was now in sole charge of the CGJs and commercial courts, and began to look for ways to increase its efficiency and transparency. As a result, the JSRP financed a study commissioned by the SC to examine the potential to integrate commercial and CGJ ICT systems in the future, building on JRSP achievements. Overall, therefore, the changes that occurred during JRSP implementation strengthened the reasoning behind the original theory of change.

II. OUTCOME

Relevance of PDOs Achievement of PDOs (Efficacy) Efficiency Overall Outcome Substantial Substantial High Substantial


Page 19 of 148

A. RELEVANCE OF PDOs

Assessment of Relevance of PDOs and Rating

Rating: Substantial

20. The PDO was relevant to RF priorities throughout the project life. Improved transparency and efficiency of the RF court system was and remains critical to the RF. This was reiterated by RF authorities and the judicial leadership at mid-term review and through successive restructurings.

21. The Bank’s implementation support evolved in relation to changing needs on the ground: the Bank reviewed in detail the pilot of the GAS Pravosudiye and provided feedback through discussions and mission Aides Memoire on the design, integration and implementation risks. This led to the exit of the JD and the inclusion of the MCC and the CIR in the project. The Bank team provided examples of, and experts in, international good practice during the drafting in 2008-2009 of Law 262 (which mandated publication of all judicial decisions). Intensive consultations between European and US judges and experts on the one hand, and Russian judges and drafters on the other, were arranged by the Bank team under its supervision budget, with strong support from Bank management. And when the SAC ceased to function in 2014, the Bank helped counterparts to minimize the impact on the commercial court ICT system, and supported the SC in developing a strategy for a future integrated ICT system for CGJs and commercial courts.

22. The PDO was consistent with the Bank’s CPSs for the RF till 20166. The 2017 Systematic Country Diagnostic (SCD) reviewed justice achievements and constraints as part of the scan of the governance horizon: the JRSP provided generally sufficient information to show alignment of the objectives to the preceding CPSs. In addition, when circumstances changed, the outcome indicators were revised in 2012 and 2014 to keep the objectives relevant. In 2017, the New Development Bank financed a $460 million follow-on judicial modernization project. The PDO and project design were also consistent with the approach to judicial reform in the World Bank’s Europe and Central Asia (ECA) Region, which was to support transparency- and efficiency-enhancing reforms. The JRSP, which focused on the potential of ICT to facilitate such change, was aligned with this approach. Lastly, the May 2018 Presidential Decree on national programs and priorities mentions digitalization of the economy and of state functions and services to be facilitated by a new Ministry of Digital Economy.

23. For the reasons above, the relevance of PDOs is rated Substantial.

6 In the CPS for 2012-2016 (report no. 65115-RU), the JRSP is aligned with and contributes to the CPS results under the cross-cutting theme of “Improving Governance and Transparency”.


Page 20 of 148

B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome

Rating: Substantial

24. Three of four PDO-level indicators, and five of seven intermediate results indicators, were achieved by project closing. The achievements are assessed below.

Outcome 1. Increased judicial transparency in courts financed by the JRSP

25. PDO Indicator 1. Availability of data from periodic surveys of users of judicial services – achieved7. Surveys from 2010-2017 show that (i) trust in courts slowly increased and (ii) more entrepreneurs felt that courts protected their property rights. The 2010 survey was published by the BEA and Levada Center while the MCC published the 2017 survey results on its public website. The indicator has been useful in tracking transparency and trust in courts. Five rounds of surveys (2010, 2011, 2012, 2013 and 2017) tracked user satisfaction with, attitudes to, and actual experiences with, courts (including enforcement of judicial decisions). The surveys succeeded in tracking some longer-term trends and outcomes, as the extracts below show. The successful conduct of five rounds of user surveys (in addition to two more 2017 surveys specially commissioned and conducted for the MCC) and the availability of survey data for research and publication was key to achievement of PDO indicator 1. All survey results were shared with the judicial and executive leadership and used by courts to improve their functioning. Even if the wording of the original PDO indicator is considered, the surveys show increasing trust and confidence in courts (Figure 3).

Figure 3. Trust and Confidence - 2010-2017 Survey Feedback from General Public and Entrepreneurs

Table 2. Responses to the question on trust and confidence in the courts: Can an ordinary Russian citizen have his/her case considered fairly and without prejudice by the court? (General public, percent answering yes)

2006 2007 2008 2009 2010 2012 2013 2017

Percent 46 55 56 46 64 73 51 76 Table 3. Responses to the question: What issues in judges’ work are the most acute from your point of view?

30

40

50

60

70

80

2009 2010 2012 2013 2017

Can An Ordinary Russian Citizen Have His/Her Case Considered Fairly & Without Prejudice By The Court? (General Public:

% Responding "Yes")


Page 21 of 148

(General public, percent choosing various options) 2010 2011 2012 2013 2017 Bribery, corruption of judges and court personnel 56 59 55 48 45

Judges’ dependence on the authorities and influential groups 46 45 50 40 39 Absence of equality of parties in court proceedings, e.g. public officials have priority over ordinary citizens 25 27 30 25 27 Red tape, delays, unproductivity 29 26 26 25 20 Low public standing of judges 17 24 17 25 14

Table 4. Responses to the question: Are your private property rights protected? (Entrepreneurs, percent responding)

2011 (n = 812)

2012 (n = 823)

2013 (n = 802)

2017 (n = 800)

Yes/rather yes 43 48 52 51 No/rather no 54 50 45 44 Hesitate to answer 3 2 4 6

Table 5. Responses to the question: How high is the public standing of the following institutions in the community? (General public)

2011 2013 2017 2011 2013 2017 2011 2013 2017 Very high + Rather high Very low + Rather low Hesitate to answer

Church 72 63 61 19 30 29 9 7 10 Government 65 49 62 29 46 32 7 5 6 Special services (FSB etc.)

63 65 69 21 26 19 16 9 13

Army 46 55 73 47 41 20 7 5 6 State Duma 45 38 50 48 57 42 7 5 8 Businesses 43 44 52 47 46 36 11 9 13 Courts 41 44 54 49 50 37 9 6 9 Human Rights Organizations

41 40 43 43 48 38 16 12 19

Public Chamber 37 35 40 36 49 37 27 16 22 Law enforcement 34 38 53 58 56 37 8 6 10

26. JRSP activities increased courts’ transparency at all levels. They increased the availability of information about citizens’ and businesses’ legal rights, about court functioning, about how to lodge complaints about judges and courts – each of these also increased the protection of parties’ rights in court proceedings. Details can be seen in Annex 4 (Efficiency Analysis). SC, SAC and MCC information kiosks and public information systems created positive transparency effects. The RF SC building was equipped with self-service information terminals (output B.2.2.5 ‘Purchase of information referral system equipment for RF SC visitors and delivery of related services for system implementation’). Through such activities, citizens gained comfort and confidence (difficult to measure in monetary terms) in obtaining information on their

7 The original PDO indicator 1 read as follows: “Periodic surveys show improved private sector and public ratings for judicial transparency and efficiency”. As explained earlier, it was modified to its present form at the 2012 restructuring.


Page 22 of 148

own. A similar positive effect arose from providing the MCC with self-service terminals and panels through JRSP activity B.3.1.2. All of this improved courts’ transparency and openness and citizens’ confidence in approaching courts with their disputes and complaints.

27. Survey findings on transparency – Levada surveys. In the Levada surveys two questions for the general public were directly related to openness and transparency of the court system: “From your point of view is information on court activities accessible to citizens, public at large and to what extent?” and “From your point of view how transparent court proceedings are for 1) interested persons, their relatives; 2) mass media; 3) general public”. On the first question, respondents stating that information is fully accessible increased by one percentage point between 2010 and 2017 (from 3 in 2010 to 4 in 2017), respondents stating that information was mostly accessible stayed stable (20 percent) and respondents stating that judicial information was limited fell from 51 percent in 2010 to 44 percent in 2017. On the second question, respondents stating that court proceedings are not transparent enough for interested persons and relatives fell from 31 percent in 2010 to 28 percent in 2017, and respondents stating that court proceedings are not transparent enough for mass media fell from 41 percent in 2010 to 38 percent in 2017. Respondents stating that court proceedings are not transparent enough for the public fell from 48 percent in 2010 to 45 percent in 2017. In the 2017 survey, 63 percent of entrepreneurs responded that they use the database of arbitrazh court decisions, and 24 percent used templates of court documents. On the question “Is it difficult to get information on arbitrazh court jurisprudence?”, 47 percent of entrepreneurs answered that it was not difficult and 27 percent that it was rather easy. Commenting on arbitrazh court websites, 71 percent were satisfied with the amount of information, 66 percent were satisfied with the search features and 64 percent were satisfied with the regularity and timeliness of information updates.

28. Survey findings on transparency – MCC surveys. Satisfaction with functioning of all information systems rose by 10-20 percentage points between 2014-2017 and by 13 percentage points on average. Satisfaction with access to video recordings of court hearings and live-streaming of court sessions rose by 13 percentage points. Professional users were highly satisfied with the MCC ICT system’s information and analytical applications (57 percent), the external internet portal of the Moscow courts (57 percent), the data exchange system between courts (56 percent) and video-conferencing with detention centers (55 percent). Satisfaction of the general public with all systems rose by 11 percentage points on average. The most appreciated were the external web portal (65 percent, an increase of 17 percentage points), information kiosks (63 percent, an increase of 8 percentage points), and public access to the e-Justice system (58 percent, an increase of 9 percentage points).

29. Additional sources of information for users about the courts. According to the Levada surveys, newspapers and TV programs remain the most important sources of information about the judicial system for the general public. The importance of internet sources is rising (from 4 percent in 2010 to 15 percent in 2017). Among firms, 49 percent of respondents named arbitrazh courts’ websites among their main sources of information about the judicial system (against 32 percent in 2010).

30. Numerous project outputs contributed to the achievement of the transparency outcome, including: Study and assessment of effectiveness of interaction between courts and mass media; Publishing of the reporter "Decisions of the Constitutional Court of the Russian Federation" in English


Page 23 of 148

Consulting Services for Analysis of the Legal and Regulatory Framework of the RF and development of proposals for improvement to ensure legal provision for the use of ICT in case management and interaction between courts and other government agencies of the RF and their territorial bodies (in part related to live-streaming of court hearings and public access to records of court hearings);

Supply and Installation of the integrated document flow management system and information posting in the internet portal of the Constitutional Court of the RF;

JRSP/1/B.2.2.5. Procurement of equipment of information system for SC visitors and installation; JRSP/1/B.3.1.2. Turnkey creation of the Integrated Information System of Moscow city general

jurisdiction courts (in part related to installation of information kiosks, live-streaming of court hearings and public access to records of court hearings);

JRSP/1/B.4.2.1. Supply and Installation of information mini-booths for the arbitrazh courts; JRSP/1/B.4.2.2. Development, implementation and support of uniform portal of the arbitrazh

(commercial) courts of Russia; JRSP/1/B.4.2.3. Supply and Installation of systems of storage of electronic copies of judicial

documents for the arbitrazh courts; and JRSP/2/C.1.11. Professional development of public relations specialists of the Constitutional Court of

the RF, commercial courts, courts of general jurisdiction and divisions of the Judicial Department.

Outcome 2. Increased efficiency in courts financed by the JRSP

31. PDO Indicator 2. Decisions of Moscow City Courts in civil cases enforced, as a percentage of all decisions by Moscow City Courts in civil cases: not fully achieved (81 percent achieved). This indicator was important because MCC decisions in civil cases needed to be enforced by public bailiffs for the successful party to actually obtain the benefit of the court decision. Bailiffs’ non-enforcement of such decisions was difficult for the courts to track. The new system, which was linked to the bailiff ICT system, made it more for bailiffs to delay or avoid enforcement, and provided information to the courts on the progress of enforcement of each court decision. Against the end-project target of 40 percent of MCC civil decisions enforced, the MCC achieved 37 percent. The 2014 baseline was 24 percent. The under-achievement was due to challenges in initially implementing the technically complex MCC integrated ICT system (which required interfacing with the bailiff ICT system). The achievement was significant but fell short of the ambitious target.

32. PDO Indicator 3. Reduction in time taken by Moscow courts to respond to citizen applications, as a percentage of 2016 baseline - Fully achieved (265 percent of target). This indicator was added to measure the efficiency of response to citizen applications addressed to the Moscow City Courts. This indicator, as a published report automatically generated by the MCC ICT system, increases transparency by providing users/litigants information on the MCC’s response to citizen applications. In 2014, the MCC took 30 days on average to respond to citizen applications, including complaints. This was targeted to be reduced to 24 days (i.e. 20 percent reduction) by the end of the project. The actual achieved by end-project was 16 days, i.e. a 53 percent reduction in time from the baseline of 30 days (in other words, 265 percent of the targeted 20 percent reduction). Reports on this indicator constitute part of the numerous automated reports being generated by the integrated ICT system installed in the MCC under the largest JRSP ICT contract.


Page 24 of 148

33. PDO Indicator 4. New video-conferencing facilities (to enable remote participation in court proceedings) are functional in courts financed by the project - fully achieved (100 percent). This indicator originally covered the SC, SAC, 900 CGJs and the commercial court system. It was already achieved for the SC, CC, SAC and all commercial courts by the time of the 2012 restructuring. At the 2014 restructuring, with the exit of the JD and the 900 CGJs from the JRSP, and with the SAC having been abolished, the revised indicator covered the SC, CC, commercial courts, MCC, and half of all penitentiaries. It was fully achieved by project closing.

34. Achievement of key IRIs underpinned success as measured by the PDO results indicators: a) For achievement of PDO 2 as measured by PDO indicators 2 and 3, the achievement of IRI “percent

of courts financed by the JRSP in which new information services are available for users” was key to success. Underlying this IRI were several outputs under component B, summarized in the Key Outputs section of Annex 1;

b) PDO indicators 2, 3 and 4 were dependent on the achievement of a key IRI (a modern data processing center for the Moscow Courts is operational at the MCC);

c) For PDO 4, the establishment of the VC systems was preceded by development of technical specifications and functional requirements that would be compatible with interoperability of ICT systems (i.e. which could work with court and prison ICT systems at the very least); and

d) Underlying each IRI were the outputs indicated by component in Annex 1 (Key Outputs section).

35. Some IRIs were not preconditions for PDO indicators: they were included for specific reasons: a. They were key elements of the MCC automated system and needed to work satisfactorily for the

overall achievement of PDO 2 – the IRIs “time taken by Moscow lower courts to forward criminal and civil cases to courts of appeal” and “reduction in cases with contested court records” are examples. Together they ensured that cases sent to other/higher courts were complete in terms of documentation and time was not wasted in getting missing documents or evidence; or

b. They contributed to citizen engagement and complaint handling, e.g. “reduction in complaints by litigants in MCC”.

36. Numerous project outputs contributed to the achievement of the efficiency outcome, including: Development of new criteria and indicators of efficiency of functioning of the judicial system; The scientific analysis and systematization of decisions of the Constitutional Court of the RF; Analysis and generalization of the practices of the European Court on Human Rights in the context of

the RF Constitution and Russian legislation; Analysis of efficiency and recommendations to improve video-conferenced hearings in CGJs; JRSP/1/B.1.1.2. Modernization of ICT infrastructure of the Constitutional Court of the RF; JRSP/1/B.2.1.1. Retrospective conversion of RF SC decisions and data files into electronic format; JRSP/1/B.2.2.2. Procurement of equipment for workplaces of judges and employees of the SC; JRSP/1/B.2.2.3. Modernization of the SC Information System; JRSP/1/B.2.2.4a. Procurement of hardware and software for virtualization of computing resources of

the SC and related services for implementation of the virtualization system; JRSP/2/B.2.2.8 (i). Information Systems Audit of the SC and the SAC of the RF; JRSP/2/B.2.2.8 (ii). Development of IT Concept of the SC of the RF;


Page 25 of 148

JRSP/2/B.2.2.9. Development and implementation of the information system "Case Processing and Judicial Documents Flow" of the SC of the RF, based on modern ICT;

JRSP/1/B.2.3.1. Procurement of equipment and system of communication channels for the system of remote court proceedings of the SC of the RF;

JRSP/1/B.2.3.2. Supply of additional video-conferencing equipment to the SC of the RF and Federal Penitentiary Service (FPS) offices;

JRSP/1/B.3.1.2. Turnkey creation of an Integrated Information System of Moscow city CGJs; JRSP/1/B.4.1.1. Supply and installation of modern internal phone communication for the arbitrazh

court system; JRSP/1/B.4.1.3. Development of a uniform system of corporate e-mail in commercial courts; JRSP/1/B.4.1.5.1. Analysis of the needs of the CIR in terms of the specialized functionality of its IT

systems, and development of the concept and technical requirements for the new specialized automated case processing system;

JRSP/1/B.4.1.5.2. Development of new functional modules of the Arbitrazh Case Management System, reflecting process-specialization of the CIR and based on current legislation;

JRSP/1/B.4.1.9.1.1. Upgrade of the CIR data storage system to enable processing and storage of live court hearings broadcast; and

JRSP/B.4.1.9.3. Procurement of an automated system for document flow management in the CIR.

37. Significant poverty and social effects also arose from the JRSP-financed videoconference (VC) system installed in the Supreme Court, which linked it with courts and penitentiaries8. Witnesses and detainees could now remotely provide testimony and be heard from different locations without needing to travel vast distances, stay overnight, take leave from work, or to leave their minor children or sick relatives alone. The state saved money on transportation and security of detainees. With the implementation of the VC system, the ability of citizens to access justice services improved greatly. Court practice showed that people serving sentences or those in pre-trial detention are sometimes witnesses in other court proceedings in other regions. The convicts or accused used to be transported to the place of hearing by special transport – such round trips could take several months, leading to high expenditures in addition to security risks and logistical complications. This project design accommodated this important access intervention. Providing prisons with VC facilities also produced other positive effects: for example, prisoners’ relatives, no matter how far they live from the place of imprisonment and regardless of whether they possessed money for transportation to the prison, now have the right to video-conferencing.

38. The MCC integrated information system allows citizens – including low-income citizens, women and other vulnerable groups – and businesses to lodge complaints with the court by electronic means and remotely track the status of the complaint. The social effect of such an opportunity is significant. To lodge a complaint in person requires a citizen’s presence in court during working hours. Consequently, by engaging with the court remotely, she or he not only significantly saves travel time (considerable for a city of Moscow’s size and spread) but the person does not have to miss work for a personal visit to the court or to spend working hours on telephone calls to obtain information on the status of the case. Online services – accessible through public internet access sites or from home or other internet connections - make applicants more confident and comfortable as they can access the information at any time or from any location.

8 VC systems were installed in all regional courts, district courts of general jurisdiction and in more than half the penitentiaries.


Page 26 of 148

39. Non-JD project activities proceeded rapidly during 2007-2012, including modernization of SC, SAC and CC ICT systems, and identification and promotion of good practices in Russia’s judicial reforms, supported in part through the ECA Justice Sector Peer-Assisted Learning (JUSTPAL) Network. JRSP-financed analytic work (e.g. a survey of user experiences with courts, performance indicators) was taken into account by the judicial leadership in designing further reforms, and in promoting a culture of increasing transparency, efficiency and user feedback9. The SAC Chief Justice visited the World Bank in April 2009. The RF courts sent 2 delegations to the April 2011 Athens launch conference for the ECA JUSTPAL Network, and participated actively in subsequent JUSTPAL events, sharing Russian courts’ innovations and good practices. Importantly. The Bank team organized a 2008 visit of a high-level delegation from Russia comprising bankruptcy judges from the SAC, Moscow City Arbitrazh Court, Appeal Arbitrazh Court for the First Circuit, SAC staff members and MOE to the United States (Sacramento and Boston) to study consumer bankruptcy in the US: best practices identified by them contributed to the preparation of the draft law on consumer bankruptcy in Russia, which in turn played an important role in enhancing Russian commercial courts’ ability to deal efficiently with the influx of such cases in the post-2008 crisis period.

Justification of Overall Efficacy Rating

40. The project’s overall efficacy is rated Substantial. The operation substantially achieved its objectives and intended outcomes, as explained above. Outcome 1 was substantially achieved with the robust 5 rounds of surveys and the additional 2 surveys specially designed for the MCC. The MCC published its survey results on its website. Significant findings from the surveys are at Annexes 7 and 8: these support the conclusion that the significant project investment on modern automated systems and electronic access was beneficial for accessing courts as well as information about courts. Revised PDO indicator 1 was achieved, as were two of three other PDO indicators. Outcome 2 was also achieved, as indicated by the outcome indicator, IRIs and other evidence summarized above.

C. EFFICIENCY

Assessment of Efficiency and Rating

Rating: High

41. Economic analysis at completion, based on conservative assumptions, shows a Net Present Value (NPV) of RUR7.3 billion (or US$121.8 million equivalent based on the prevailing exchange rate) and an economic internal rate of return (EIRR) of 63 percent, in line with satisfactorily-performing public sector institutional modernization projects in the RF and elsewhere. Details of the economic analysis, including methodology, underlying assumptions and calculations, is at Annex 4. The analysis demonstrates that project efficiency objectives were achieved.

42. Summary of analysis. Project economic analysis at appraisal followed Bank and ECA Region practice at the time for public sector institutional modernization projects. The project was designed to provide

9 In addition, a Japan Social Development Fund (JSDF) Grant, implemented by a policy research institute, the Institute of Law and Public Policy, had achieved unanticipated policy impact and results on the ground. The Grant Management Council included a representative from the Presidential Administration and was chaired by a retired Justice of the Constitutional Court Ms. Morschakova. The RF law on free legal aid was revised to include non-state providers such as lawyers and NGOs, based in part on results from JSDF-financed activities in the 2 participating regions.


Page 27 of 148

economic benefits from improvements in judicial performance and was expected to improve judicial transparency and efficiency. Positive economic effects arose from savings on staff time of judges and other court officials; lower transactions costs between courts, other governmental institutions and users of courts such as citizens and businesses; and reduced cost of processing information and providing it to citizens, businesses and governmental agencies. The estimated direct economic effects (DEE) of JRSP activities considerably exceeded the expenditures incurred on the JRSP. In addition to the DEE, project implementation positively affected the quality and timeliness of the administration of justice and improved access to judicial services for citizens, businesses and professional users. It also contributed to improving actual experiences with, and perceptions about, the judicial system as a whole. This in turn contributed to increasing satisfaction of citizens and businesses with the judiciary. JRSP activities also had social effects such as increasing the transparency of courts at all levels; increasing the availability of and access to court decisions, judicial information and judicial processes; and increasing protection of the rights of litigants and businesses, as the survey data cited in the ICR demonstrate.

43. Taking account of the overall economic effect, the comprehensive economic effect (CEE) demonstrated efficiency in JRSP implementation. The CEE value obtained for the period up to and including 2022 is RUR40 billion (equivalent to US$665.9 million at the end-2017 exchange rate or US$614.7 million based on the average weighted exchange rate provided by the RF Central Bank for the corresponding years). In addition, a stress test applied to assess the JRSP results showed high stability. Under the stress test, all key assumptions underlying CEE estimations were simultaneously varied from +10% to -10%. This facilitated examination of system behavior under stringent conditions.

Methodology

44. The JRSP was implemented within a larger context of major judicial reforms initiated by the government and mostly financed by Federal Targeted Programs (FTPs). Efficiency was calculated as a combination of direct economic effects of project activities, and as part of the overall effect of reform of Russia’s judicial system, of which the JRSP was an element. JRSP effects are divided into 2 categories: direct and indirect. Direct effects were calculated separately for each activity. Indirect effects were calculated for the entire JRSP as part of the effect of the entire judicial reform. The combined effect is the sum of all direct effects for JRSP activities and of the overall effect of the entire JRSP. The methodology used was to (a) identify JRSP activities with direct effects from project components and calculate the sum of all direct effects (i.e. the cumulative direct effect or CDE), (b) calculate the overall effect (OE) of the JRSP as part of the effect of the entire judicial reform using appropriate methodology summarized in the Annex (based on the detailed project economic analysis) and (c) deduct the amount of JRSP financing from the sum of CDE and OE to arrive at the total comprehensive economic effect (CEE).

45. JRSP activities with direct economic effects. Table 6 below shows the most important JRSP activities with direct economic effects with a summary of the effects identified.

Table 6. Activities with direct economic effects

Activity number and description Description of direct economic effects B.2.1.1. and B.2.1.2. Conversion of RF SC Reduction in staff time and cost associated with searching


Page 28 of 148

decisions and data files into electronic format (Stages 1 and 2)

archival documents

B.2.2.3 Modernization of SC Information System

Reduction in staff time and cost associated with scanning cases and other files/documents Reduction in staff time and cost of IT specialists in repairing/replacing computers

B.2.2.4.a Procurement of hardware and software for virtualization of SC computer resources and provision of related virtualization system implementation services

Savings from procurement, deployment and maintenance of physical servers (PS) from creation of required operational infrastructure

B.2.2.9 Development and Implementation of the SC Information System “Case Processing and Judicial Documents Flow

Savings by citizens and organizations as they (and/or their representatives) do not need to be physically present to file procedural requests

B.2.3.1 Procurement of equipment and development of communication channels for organization of remote proceedings

Cost savings on prisoner transfers, procedural fees, relatives of prisoners, training and investigation activities

B.2.3.2 Supply of Additional VC Equipment to SC and FPS Offices

Saving on procurement of additional MGTS telephone numbers through digital telephone station (DTS)

B.3.1.2. Turnkey creation of Integrated Information System of Moscow city general jurisdiction courts

Saving of labor cost of judicial staff associated with filing of cases in courts of higher instance and passing of judgement by courts of higher instance

Saving of labor cost of judicial staff associated with handling of complaints

Saving of labor cost of citizens and organizations’ representatives associated with submission of claims and applications for examination of case materials through the Internet

Saving of staff cost and time associated with maintenance of databases of new employees

Saving of material and postage costs using electronic exchange with FBS

B.4.1.1. Development of corporate telephone network of commercial courts

Savings on long-distance calls


Page 29 of 148

B.4.1.5.2. Development of new functional modules of arbitration case management system specific to the CIR

Labor saving associated with preparing the state statistical data

B.4.1.9.1 Procurement of servers and active network equipment

Savings on international calls and on buying telephone numbers from telephone company

B.4.1.9.3 Procurement of Automated System for Document Flow Management

Reduced material expenses and labor costs of administrative personnel associated with maintenance of non-judicial paperwork

В.4.1.9.4 Procurement of videoconferencing equipment for the Presidium of the CIR

Savings for citizens and organizations as they (and/or their representatives) do not need to be physically present at court sessions

B.4.2.2. Development, implementation and maintenance of a single portal for Russian commercial courts

Reduced time spent by citizens and organizations to obtain information

B.4.2.3/i-iv Supply and Installation of systems of storage of electronic copies of judicial documents

Saving of labor cost associated with registration of and providing access to arbitration cases

Saving of labor cost of IT specialists associated with equipment maintenance

Saving of funds for procurement, deployment and maintenance of physical servers (PS) and for creation of the required operational infrastructure

B.4.2.4.1 Audio Recording System of Court Sessions

Saving on reduced number of appeals against judges’ orders and decisions

46. Some activities can have longer-term impact even if their direct effect is difficult to identify or measure at this point. Typical examples include training and skills upgrading of judges and judicial staff, knowledge exchange and access to international good practice for judges. It is difficult to assess the direct economic effects of such activities, yet they do impact Russia’s court system.

47. Overall effect of JRSP. The JRSP contributed to the overall effect of the system-wide judicial reform over the years. The total overall effect of the JRSP is the sum of annual effects. The comprehensive economic effect (CEE) of the JRSP is the sum of the direct economic effects and the overall economic effect of the Project, less cumulative actual JRSP expenditures. Inflation estimates used data from the Federal State Statistics Service, while inflation forecasts up to 2022 used data from the MOE10. Details are in Annex 4 (Efficiency Analysis).

10 http://economy.gov.ru/minec/about/structure/depmacro/2017271001


Page 30 of 148

48. Aspects of design and implementation contributed to efficiency. First, the exit of the JD and CGJ activities enabled more efficient utilization of freed-up project resources for the MCC, the CIR and the SC, further enhancing the efficiency of use of JRSP funds. Second, the engagement of the FER as the PIU from December 2011 resulted in more efficient utilization of PIU resources due to faster procurement and project implementation. Third, as Annex 3 shows, project resources were utilized substantially in line with estimates at approval, with the 2017 additional allocation of US$15 million from the MOF helping to further modernize SC ICT systems and reduce ICT operations costs over the medium term.

Box 2. Switching Financing from 900 CGJs to the MCC Increased JRSP Efficiency and Impact

Many of the 900 CGJs whose ICT systems were originally envisaged to be modernized under the JRSP were merged or reorganized during 2013-2017. Typical small CGJs in Russia (with 1-5 judges) resolve about 500 cases annually, while CGJs with 3-5 judges sometimes resolve up to 1,000 cases annually. Using an average of 800 cases annually resolved per small CGJ (which is on the high side), the total number of cases annually resolved by the 900 small CGJs originally included in the project would be 720,000. In contrast, the number of cases annually resolved by the Moscow regional court system is 1.5 million. The exit of the 900 CGJs from the JRSP and the inclusion of the MCC more than doubled the effect of JRSP investment in Russia’s court system.

Following the exit of the JD and the 900 CGJs from the JRSP in 2014, budgetary resources were allocated to the JD for ICT automation of small CGJs, and in 2014-2015 budget-financed contracts were signed to install ICT equipment for the 900 CGJs. This equipment ran on the GAS Pravosudiye system. No budget funds were allocated to the MCC for ICT modernization. The MCC obtained access to resources for ICT modernization only at the 2014 restructuring through inclusion in the JRSP. The MCC and the Bank took this opportunity to design and implement a more efficient ICT system for the MCC, which is now helping Russia’s judicial authorities to further improve GAS Pravosudiye functionality and performance.

49. The project’s Efficiency is rated High for the reasons above.

D. JUSTIFICATION OF OVERALL OUTCOME RATING

50. The overall outcome is rated Satisfactory. Relevance of objectives is rated substantial. Achievement of objectives is rated substantial, and efficiency is rated high based on the economic and financial analysis. In addition, (a) the PDO remained substantially relevant and its strategic importance grew with the entry of the MCC and the CIR into the project, and with the issuance of the May 2018 Presidential Decree on national priorities and programs, (b) 3 of 4 PDO indicators were achieved, (c) 5 of 7 IRIs were achieved, (d) the project economic analysis demonstrates efficiency, (e) project performance since the 2012 restructuring has been remarkable (reflected in project ISRs) and the MCC’s implementation of its integrated ICT system is an example of global good practice in design and implementation of complex ICT systems through a rapid (seven-month) 2-stage bidding process, and (f) intensive and constructive Bank implementation support was possible due to a technically and operationally strong and stable team with no turnover of team members. This is substantiated by Annexes 7 and 8 which depict key survey findings, Annex 9 which depicts the key features and results of the MCC ICT system, Annex 10 which does likewise for the SAC and commercial courts’ JRSP-financed ICT system and Annex 12 (the executive summary of the Borrower’s ICR). The Borrower’s detailed ICR (which was received in July 2018 and filed) provides detailed additional evidence for a Satisfactory outcome rating.


Page 31 of 148

E. OTHER OUTCOMES AND IMPACTS

Gender

51. The project over-achieved the target for women beneficiaries, who constituted the women judges and staff of JRSP-financed courts. This indicator was added at the 2012 restructuring as a mandatory indicator, following Bank policy. The target for female beneficiaries was 40 percent (or 12,800) of the target for total direct project beneficiaries (32,000). By end-project, the actual achievement was 39,579 total direct beneficiaries (124.25 percent of target) of whom 26,541 (or 66.75 percent) were women. Hence the actual achievement for women beneficiaries was 112.56 percent of the target. In addition, the user-friendly features of the JRSP-financed ICT systems facilitated access to courts for women and other vulnerable groups.

Institutional Strengthening

52. The SC, SAC and MCC improved their capacity to design, tender, contract and implement sophisticated ICT systems. This was demonstrated by (a) the SC, the SAC and the commercial courts in installing, operating and maintaining their JRSP-financed ICT systems (preceded by business process streamlining), and (b) the SC and MCC in rapidly designing, during 2016-2017, the US$460 million follow-on judicial modernization project presented by the RF to the New Development Bank for financing, approved by the NDB on August 31, 2017. In addition, the MCC designed, procured and successfully implemented a suite of next-generation integrated court ICT systems at an estimated cost of US$42 million within 3 years – a significant achievement in terms of complex ICT procurement, system integration, testing and going live. The MCC now has the capability to augment this system in the future and help other courts to do so.

Mobilizing Private Sector Financing

53. Not applicable.

Poverty Reduction and Shared Prosperity

54. The MCC integrated information system allowed citizens – including low-income citizens, women and other vulnerable groups - to lodge complaints with courts electronically and remotely track the status of the complaint; so did the VC connectivity and remote hearings between the SC, lower courts and penitentiaries. To be heard, give testimony or lodge a complaint required a citizen’s presence in court during standard working hours. By remotely engaging with the court, citizens not only significantly save travel time but do not have to miss work or to spend working hours on telephone calls to obtain information on the status of the case. Online services can make applicants more confident and comfortable as they can access the information at any time or from any location. Many state institutions working with citizens and businesses (such as the Federal Tax Service, the Pension Fund and entities of the Ministries of Health and Education) provide users with online interaction. This is already considered to be the norm by citizens. This capability of the judiciary has improved the poverty and shared prosperity impacts. It has also improved the service-friendliness of the SC, commercial courts, the MCC and the CIR in the eyes of citizens and businesses.

Social development

55. As stated earlier in the PDO achievement section, the operation had significant social effects such as enabling citizens to access courts and penitentiaries electronically.


Page 32 of 148

Other Unintended Outcomes and Impacts

56. None.

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION

57. JRSP preparation and design took account of lessons learned from the World Bank-financed LRP and from the then-ongoing ICT-intensive TAMP2, TMP and CDP operations.

Focus on reforms that had significant high-level commitment and support from policy-makers and ownership from technical levels. Implementation of TAMP2, TMP and CDP, which had been designed with these lessons in mind, were progressing despite changes in top management in 2005 in the tax and customs entities, because civil servants and technical working groups in these entities ‘owned’ the key interventions. This model was followed in JRSP design.

Intensive attention to the design, procurement and integration aspects of ICT interventions. Learning from TAMP2, CDP and TMP at preparation, major JRSP ICT interventions were sequenced so that analytics came first (which informed the development of user and technical specifications – see key outputs at Annex 1), followed by procurement of equipment and integration of systems. Such sequencing at the design stage was important given the technological and procurement risks associated with ICT systems11.

Focus only on the courts in view of the judiciary’s special status – a lesson learned from several justice reform projects around the world. The RF courts wanted to be responsible for selecting and implementing project activities, to avoid sensitivities between the executive and judicial branches. This lesson was also learned from the LRP, which financed activities for the courts, executive entities and non-state actors (such as universities) but was overseen and implemented by the Presidential Administration through a PIU, leading to implementation problems when disagreements arose between the courts and the executive on project issues.

58. Project risks were identified and the JRSP categorized as a high-risk, high-reward project. While judicial reform is always challenging, it was more so in the RF at that time, with its complex and changing relationships between its apex courts, evolving organization of the judicial and executive branches and the shifts in responsibilities between entities. Nevertheless, it was felt that the potential gains were so large that these were risks worth taking. The key risks summarized in the PAD, and what actually happened, are shown below.

Table 7. Key Risks and What Actually Happened

11 At the time, the World Bank’s ECA Region was at the forefront of ICT procurement risk mitigation in the World Bank: all ICT procurement documents were reviewed by ICT procurement experts in addition to the task team’s own review, even though the task teams for TAMP2, CDP and JRSP included experts in ICT systems design, procurement and implementation.


Page 33 of 148

Risk Stated in the PAD Risk Mitigation Measure Rating What Happened

Political risks are unavoidable. These arise from the complex interplay and sometimes conflicting agendas of major actors: different elements of the judiciary, the executive and the legislature. Such conflicts may be manifested in inter-agency turf battles, coordination gridlocks, backtracking on reforms, and delays in project design and implementation.

Consultative design, clear communications, explicit inter-agency agreements and transparent implementation.

H

This rating was appropriate in hindsight. This risk came to pass. It resulted in the exit of the Judicial Department and CGJs and the addition of the MCC and the CIR to the project.

Reputational risk. The ability of RF reformers and the World Bank to influence the modernization agenda is limited. This creates a high reputational risk for both12.

Appropriate stakeholder representation on IACC. Posting IACC meeting minutes on the official websites (accessible to the public) of the participating project entities. Transparent procurement. Adaptation of global lessons and good practice to project design and implementation.

H

This rating was appropriate in hindsight. Implementation challenges during 2007-2012 arose mainly due to a combination of these three factors: the limited ability of the Bank, the executive branch and some judicial reformers to influence the JD (responsible for the JRSP’s CGJ investments totaling almost $80 million of the $172 million project cost) and the conflict of interest and procurement issues that arose in the original PIU (the BEA). These led, in 2014, to cancellation of further JD-related project activities and a restructuring that resuscitated the project and paved the way for substantial achievement of the PDOs by project closing.

Multiple agencies, differing priorities. Implementation and sustainability risks increase with the number of agencies involved, more so when agencies have different priorities: this lesson was reinforced in the LRP. This risk is high, given the number of agencies involved in the JRSP13.

Clarity on activities to be financed, implementation arrangements and assurance of quality control.

H

Lack of effective cooperation and communication between the main implementing agencies and beneficiaries under the Project.

Extensive discussions and explanation of roles under the JRSP with all beneficiaries leading to a clear definition of responsibilities between all

H

12 The implementation challenges that led to the 2012 and 2014 restructurings were foreseen in the November 2006 Minutes of Negotiation, which recorded agreement on the approach to ICT modernization of the 900 CGJs under JRSP. 13 The SC, the JD, the SAC, the CC, the MOE and the MOF.


Page 34 of 148


stakeholders with regard to particular project components, to be reflected in a Project Operations Manual. An IACC will be established with representatives of the relevant agencies.

Ineffective ICT investment. The project will finance substantial ICT-related expenditures. This investment could prove ineffective.

Rigorously determined technical specifications derived from business needs and accompanied by relevant investments in ICT-related training and practical use of ICT-based business processes.

M

This risk did not materialize because the JD and CGJs exited from the project. All other beneficiary entities strongly owned their ICT investments.

Weak implementation capacity in beneficiary entities may adversely affect the speed and quality of implementation.

Applying the lessons learned from the implementation experience of the LRP and other projects in the RF. BEA project implementation experience will mitigate this risk somewhat. Capacity augmentation in the IT units of the JD, SAC and CC. This is to be achieved through expert technical support and through IT-related training and peer-to-peer learning to be financed through the proposed JRSP, FTP, and other government budgetary resources.

H

This risk did not materialize. All project beneficiary entities – except for the JD - established strong technical working groups. In the case of the JD, the issue turned out to be governance concerns rather than insufficient implementation capacity.

Lack of sufficient budgetary Government funding for the operation, maintenance and support of new or upgraded ICT systems.

Adequate funding requirements will be discussed with RF authorities as part of the ongoing dialogue. Government commitment to provide adequate budgetary resources for operation and maintenance of the ICT systems installed under the project.

L

The rating was appropriate. Budgetary support was very strong throughout. An additional government contribution was provided in calendar 2017.

Ignoring lessons learned from Applying key lessons from The rating was appropriate:


Page 35 of 148


the implementation experience of complex multi-beneficiary projects straddling different branches of the state.

judicial reform projects and the Russia-specific LRP implementation and ICT procurement experiences to JRSP design and implementation arrangements (outlined in section B.5 above).

M close attention to procurement issues by MOE, MOF and the Bank prevented untoward procurement outcomes throughout the life of the project.

Overlap and/or lack of coordination between JRSP activities, FTP activities and those of development partners.

A series of coordination meetings have been held with development partners during project development and design. Continued close coordination and collaboration between the RF authorities, the Bank and development partners to closely align JRSP support with ongoing and planned efforts. An MOEDT-chaired IACC will monitor coordination efforts.

M

The rating was appropriate. RF authorities – especially MOE, MOF and the courts – took care to avoid overlap of JRSP and FTP activities.

Overall Risk Rating H

B. KEY FACTORS DURING IMPLEMENTATION

59. In 2010, all projects in the Bank’s RF portfolio – including the JRSP - underwent a restructuring to align the wording of the PDO in the Loan Agreement with that in the PAD.

60. The government enacted a landmark law in 2010 (Law number 262) – supported by the Bank14 – mandating publication of judicial decisions by all courts regardless of level or jurisdiction. This immediately made the PDO more strategically relevant and the JRSP – which supported implementation of court ICT systems to provide the capability and ‘bandwidth’ for courts to store and publish their decisions online – indispensable for Russia’s judiciary to achieve the law’s objective.

61. An early JRSP analytical work – on judicial performance measurement – resulted in the JD and courts refining their system of and indicators for performance measurement. This assignment was requested by the JD and resulted in a review of policies and practices relating to measuring judicial

14 Versions of the draft law prepared by the SC and SAC – which have the right of legislative initiative in the RF and can submit draft laws to the legislature on issues affecting the justice system – were reviewed by the Bank at the request of the RF judiciary. In addition, at the request of the judiciary, the Bank requested eminent judges and experts from other jurisdictions to comment on the drafts. Passage of this law was a landmark event in enhancing judicial transparency in the RF while addressing privacy concerns as many OECD countries have done on this issue.


Page 36 of 148

performance in Russia and internationally. Based on the analysis (which was reviewed by the World Bank team) and following extensive consultation with the community of judges and with Russia’s judicial leadership, the JD refined the performance measurement indicators for CGJ judges in 2010. Along with the passage of Law 262 referred to earlier, this was an early win for JRSP. In addition, the refined performance measurement criteria included measuring satisfaction of court users and personnel. The Russian NGO Leonid Nikitinsky's Center for Legal Initiatives received grants (first from the US-Russia Foundation and thereafter from the Presidential Administration) to pilot this element (i.e. criteria related to satisfaction of court users and personnel) and conducted three rounds of surveys in Volgograd, Yaroslavl and Nizhny Novgorod oblast courts in 2010-2012.

62. A 2-stage restructuring was approved in March 2012, to change the PIU and address the reluctance of the JD to consider Bank advice to improve the design of the ICT system proposed to be deployed to the 900 CGJs. At the May 30, 2012 restructuring, US$14.24 million (28.5 percent of the loan amount) had disbursed. Implementation delays were also caused by a slow-down in decision-making prior to parliamentary and presidential elections and also somewhat due to the abolition of the SAC.

63. The inclusion of the MCC in the project made the PDO more strategically relevant, since the MCC system is the largest regional court system in the RF in terms of caseload volume. The high MCC caseload volume meant that the JRSP – by the end of the project - had more impact than would have been the case had it financed ICT systems modernization of 900 small CGJ courts across the country. The introduction of the innovative next generation ICT systems in the MCC appears to have provided a paradigm for improving ICT modernization across RF CGJs.

64. The approach of the FER, MOE, MOF and the Bank team to look for opportunities for intervention consistent with the PDO resulted in JRSP support for ICT modernization of the MCC and the Court for Intellectual Property Rights (CIR) in Moscow. This court has all-Russia jurisdiction as a first instance court for all IP-related issues. These interventions had a significant impact on PDO achievement and results. Figure 4 below shows the scale and scope of some of the user-oriented achievements from the MCC ICT system implementation.

Figure 4. Some MCC ICT implementation results


Page 37 of 148

Source: MCC

65. Knowledge-sharing between the RF and other judiciaries, supported by the Bank team within and outside the JRSP project framework, was a constant feature since 2011. Knowledge sharing under JRSP was quite extensive. High-level teams of judges from Croatia’s commercial courts visited Russia in 2011 to learn from the SAC experience of ICT modernization. Delegations from the SC, SAC and JD participated in the launch conference of the JUSTPAL network in April 2011 in Athens and in subsequent JUSTPAL peer-to-peer knowledge sharing conferences in The Hague and Brijuni (Croatia). In October-November 2017, the MCC President and a delegation visited Azerbaijan to study the implementation experience of the US$200 million World Bank-financed Azerbaijan Judicial Services and Smart Infrastructure Project. Apart from participation of the SC, SAC and JD representatives in JUSTPAL events, several knowledge sharing visits for representatives of the RF courts were organized under the project. In 2008 a delegation comprising arbitrazh bankruptcy judges and staff of different level arbitrazh courts, as well as MOE representatives, visited Federal Bankruptcy Courts in Sacramento and Boston to learn how personal bankruptcy functioned in other jurisdictions. The experience shared contributed to the preparation of the draft consumer bankruptcy law. In 2009 a Russian Constitutional Court delegation visited Federal Constitutional Court of Germany, Constitutional Court of the Republic of Austria and European Court of Human Rights to study new ICT applications in those courts. A SAC delegation in 2009 visited Magistrate and District Courts of Singapore and the Supreme Court of the Republic of Korea to familiarize themselves with new information technologies and their use in the named courts. In 2010 a CC delegation visited the Federal Court of Australia and the Supreme Court of Australia and, in 2011, the European Court of Human Rights and the Union Court of the Swiss Confederation to study both their approaches to adjudication of certain constitutional issues and new ICT systems. In 2011 an SC delegation familiarized itself with the ICT modernization experience of the French and Austrian judicial systems.

66. International experience was also analyzed through several analytics under the project. In 2009, while developing performance criteria for the RF judicial system, the consultant LLC "GC Consultum" (Russia) studied criteria and performance indicators developed in the Netherlands, Finland and the USA, as well as at the international level (e.g. the criteria and indicators developed by the International Consortium for Court Excellence, CEPEJ). In 2010-2011 a consortium of Agritim Canada Consulting Ltd (Canada) and Legal Culture Development Foundation (Russia) analyzed examples of international good practices in organizing interactions between the judiciary and the media, including the work of court spokespersons (press secretaries) or specialists with a similar function, and conducted a sociological survey on the organization of interaction with the media in Russian courts. Based on the results of the studies, recommendations were made to improve judiciary-media interaction, and to improve the skills of court spokespersons. In 2017, when preparing draft laws on introduction of ICT tools in court proceedings, international good practices on ICT in case management and related issues were studied.

67. Sustainability of JRSP gains through a follow-on project. The JRSP – in the view of the RF authorities – achieved significant success and results. The RF authorities also appreciated the stable Bank team and its value added. The RF authorities designed a US$460 million follow-on justice modernization project financed by the New Development Bank (NDB) – the loan was approved by the NDB on August 31, 2017. The follow-on project is designed to build on and enhance the sustainability of JRSP gains and results.


Page 38 of 148

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION (M&E)

M&E Design

68. The PAD envisaged a comprehensive annual evaluation of progress against project indicators, as set out in the Results Framework. The results of the evaluation were to be used to analyze intermediate project results and adjusting activities or priorities, if needed. To the extent possible, the results of monitoring were intended to be widely publicized within the judicial community and for the general public.15

69. The nature of this multi-beneficiary project for the judiciary and prevailing RF project implementation practices influenced the M&E design. The JRSP beneficiary entities at approval comprised the CC, SC, SAC, JD, the commercial courts and 900 CGJs. The MOEDT was the implementing agency and the BEA (which operated as a Foundation under the MOEDT’s administrative control) was the PIU. Overall project supervision and guidance was provided by the IAMC, whose membership included representatives from each high judicial entity (i.e. from SC, CC, SAC and the JD) as well as from MOEDT and MOF. It was the practice in the RF at the time for each PIU to have a strong M&E team. Accordingly, the BEA established such an M&E team. Each beneficiary entity identified a technical team (usually from the statistical unit and the units responsible for project activities) responsible for project-related data collection and reporting. These entity M&E teams would provide M&E data to the BEA M&E team, which would compile and share the periodic project M&E reports with the MOEDT, MOF, IAMC and the Bank. The BEA M&E team became the focal point for internal and external project relations and was responsible for benchmarking and tracking agreed project indicators, tracking day-to-day progress in project implementation, and reporting to the IAMC, MOEDT, MOF and the Bank.

M&E Implementation

70. During 2007-2012, M&E implementation was moderately satisfactory. The BEA’s reporting on JRSP results focused exclusively on process and not on substantive or qualitative information. The results information provided by the BEA did not mention qualitative issues – the 2010 Mid Term review agreed that these would be addressed by Bank implementation support going forward.

71. At the 2012 restructuring - when the FER became the PIU – the M&E arrangements were strengthened, results reporting and utilization of M&E information began to improve, and the project closed with a strong M&E performance in its last 3 years. Actual project implementation entailed the execution of a large number of project activities in many functional areas of the work of three supreme courts, numerous lower courts and the JD. Accordingly, the PIU bore a heavy burden in helping the IAMC to coordinate implementation and obtain results information. Results reporting was done by beneficiary entities and their cross-cutting technical working groups with members drawn from relevant business units with each entity (to internalize and institutionalize modernization processes) but these groups needed handholding support from the PIU and the Bank. Once the FER became the PIU, these arrangements worked satisfactorily throughout the rest

15 This was mostly achieved: apart from the internal reporting within the judicial management system, project results were also routinely communicated to the executive agencies overseeing the project (MOED and MOF). In addition, project results were also shared with visiting delegations from other countries and at international conferences such as the JUSTPAL Community of Practice meetings (to which the RF regularly sent delegations representing the SC, SAC and the JD).


Page 39 of 148

of project implementation. The courts strengthened their capacity to sustain operational inputs and outcomes from the project through networks built during project implementation - despite disruptions caused by organizational restructuring such as the abolition of the SAC.

M&E Utilization

72. The IAMC and judicial leadership used the M&E framework to track project progress. Each court used the indicators relevant to its part of the project. The slow implementation of project activities from 2007 to 2010 and the associated BEA inability to work with the courts to obtain results information meant that, till 2010 (when the mid-term review was held), results information was sporadic (as recorded in mission Aide Memoires and ISRs). The MOE began to exercise its oversight authority over the BEA from 2010, when M&E utilization began to improve. The success of the 2012 and 2014 restructurings was based in part on close attention to M&E utilization for project decisions and restructuring discussions. M&E utilization continued to improve from 2012 till project closing, as depicted in the project results framework at Annex 1.

Justification of Overall Rating of Quality of M&E

73. The quality of M&E is rated Substantial overall: project M&E design was largely satisfactory though with some weaknesses. The M&E framework was adjusted to address weaknesses and evolving priorities during the 2012 and 2014 restructurings. From 2012, M&E significantly improved and finished strongly by project closing. In addition, the client (e.g. MCC) continued to use elements of the project M&E system to track results of their activities (e.g. using various elements of the new ICT system).

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE

Safeguard Compliance

74. The project was classified as Category C for safeguard purposes as it did not finance any civil works. As a result, no environmental or social safeguard policies were triggered. No safeguard issues arose during implementation.

Fiduciary Compliance

75. Project financial management (FM) was satisfactory throughout the project. There were no financial management issues. Audit reports were received on time. FM was rated satisfactory at closing.

76. Project procurement was mostly rated satisfactory except for the period 2008-2011 when there were delays in ICT procurements when the BEA was the PIU. Procurement improved significantly when the FER became the PIU in December 2011. A key FER and MCC achievement was the rapid 2-stage ICT procurement for the US$42 million contract for the MCC ICT system. The professionalism of the MCC and FER teams and equally rapid responses from the Bank enabled the process to be completed in eight months from concept to contract signing. Procurement was rated satisfactory at closing.

77. Implementation challenges were resolved through the 2012 and 2014 restructurings. First, performance and governance issues pertaining to the first PIU – the Bureau of Economic Analysis (BEA) - during 2008-2010 slowed down JRSP implementation, especially the procurement process for the US$42 million ICT systems contract for the JD and 900 CGJs and ultimately led to its cancellation. As a result,


Page 40 of 148

by June 2009 the JRSP was rated “Unsatisfactory”. Second, JD-related ICT activities for the 900 CGJs posed technical and procurement risks, due to which implementation of JD activities was slow from 2008 to 2011 (one effect of which was that as of June 30, 2011, 4.5 years after effectiveness, only $7.23 million of loan proceeds - 14.5 percent - had disbursed). All these issues were resolved through the 2012 and 2014 restructurings.

C. BANK PERFORMANCE

Quality at Entry

Rating: Moderately Satisfactory

79. Bank performance for Quality at Entry is rated Moderately Satisfactory for two reasons: (a) in hindsight, the risk to implementation arising from the design and implementation challenges relating to GAS Pravosudiye could have been more starkly highlighted, so that stronger risk mitigation measures could have been included; and (b) the Bank over-estimated the BEA’s ability to communicate effectively with JD management on procurement of large ICT packages – this was left to the Bank even though the BEA team included procurement specialists. For these reasons, a rating of Moderately Satisfactory is appropriate.

Quality of Supervision

Rating: Satisfactory

81. The quality of Bank supervision is rated Satisfactory.

The strong technical, operational and fiduciary strengths in the stable Bank task team enabled the Bank to help technically sophisticated court managements and technical counterparts across the RF judicial system for a considerable period of time. The team’s technical and operational strengths, which included an RF legal expert who was part of the core Bank team throughout, enabled it to provide effective implementation support. Due to the inability of the BEA (the first PIU) to engage on issues of substance with courts’ technical working groups, to obtain results information from courts, and to help courts to prepare satisfactory terms of reference and technical specifications, this function fell on the Bank team which was in constant contact with the judicial leadership on issues of project scope and results gathering, and with technical counterparts to help them prepare terms of reference and technical specifications.

The Bank team helped the RF judiciary to identify good practices from JRSP implementation, and share this knowledge with other countries through participation in international knowledge exchange events, conferences and workshops.

The team’s candor in reporting, along with ratings downgrades, facilitated a shared approach to restructurings. The team’s proactive approach to restructuring and close coordination with executive agencies, courts and the FER resulted in the restructurings being able to address the JRSP implementation challenges pointed out earlier. Early handling of PIU performance and governance issues enabled the project to effectively address risks and keep the focus on implementation and results.


Page 41 of 148

There were some missed opportunities, e.g. cancellation of JD activities at an earlier stage of implementation could have improved project performance and results.

Justification of Overall Rating of Bank Performance

82. Bank performance is rated Moderately Satisfactory overall, for the reasons above. Though (a) the team was technically strong and stable and possessed all needed technical skills, (b) most of the results indicators were achieved, and (c) the project achieved an impressive turnaround due to the 2012 restructuring, the missed opportunities justify the Moderately Satisfactory rating.

D. RISK TO DEVELOPMENT OUTCOME

83. The risk to development outcomes is rated Moderate. On the positive side, the ICT systems in the CC and SC are being further modernized with their own budgets. Both Courts publish their decisions and other information on web portals. In addition, a US$460 million loan for a follow-on judicial modernization project, approved by the New Development Bank on August 31, 2017, is taking forward key JRSP gains and achievements: it will expand ICT modernization of the MCC, other courts and the RF Federal Bailiff Service’s ICT system. It includes investment in ICT and physical infrastructure. The commercial courts’ ICT system, following the abolition of the SAC, is now supported by the JD. The Presidential Decree of May 2018 lays out the national priorities and programs for the RF - which includes progressive digitalization of the economy and of all state functions and services – including the courts.

V. LESSONS AND RECOMMENDATIONS

84. The project design and implementation experience generated four sets of lessons:

a) Commitment works best when coupled with the creation of conditions for success. JRSP achievements stemmed from a combination of top-level commitment from the executive and judicial leadership, technical-level commitment from the judges and staff of the courts financed and high technical competence in Borrower and Bank teams. These together created an environment in which sequenced change, underpinned by the transformative power of ICT, could be induced, nurtured and sustained. A key element of such commitment included shedding lagging counterparts and the original PIU, the sequencing and phasing of ICT implementation to build up commitment, and addressing institutional issues beyond ICT hardware and software (e.g. handling the cessation of SAC functioning and the changes to the structure of the RF judiciary, the induction of the MCC and the exit of the JD). The Bank trusted the commitment of the judicial and executive leadership. Rushing significant institutional change in courts in a country as complex as Russia would have created obstacles to change and led to the convergence of interests opposed to modernization. Discussions on how to use restructuring to effect change continued during implementation, and more importantly through peer-based knowledge-sharing events (e.g. JUSTPAL conferences) and study visits (e.g. to European and US courts). Where the authorities’ commitment is clear, such an approach can yield positive and sustainable results.

b) Complementary reforms and change management increase the impact of institutional reforms. For example, as the survey findings demonstrate, in addition to improving court-wide ICT


Page 42 of 148

infrastructure and systems for the SC, SAC, CC and MCC, it was critical to include actions to support collaborative activities for different groups – judges, judicial staff, academia, experts of methodological centers, and other users of the court system – whose common interests in enhancing judicial transparency and efficiency could converge around specific reforms (e.g. publication of judicial decisions, or automated complaints handling mechanisms). Such convergence through, for example, peer-based knowledge-sharing (e.g. peer-to-peer learning from other judiciaries and JUSTPAL) facilitated mindset changes and boosted collaboration. In addition, in ICT-intensive projects, managing change over the medium term is as important as technical competence. This project proved - again - that ICT in institutional modernization is more than installing computers and software: they are one element of modernization and can have a systemic impact only if the content of business processes and the organization of courts into cooperating networks of service delivery institutions can generate efficiency, transparency and trust. This is the largest Bank-financed project in ICT-intensive judicial modernization: it has played a significant role in pioneering such an approach.

c) Successful teams on both sides, which generate trust and promote smooth implementation in complex high-risk operations, should be maintained regardless of organizational shifts or requirements in the Bank or the government. This is because stable and successful teams on the Borrower and Bank sides can together address implementation risks and challenges, as the Borrower’s ICR points out. The Bank team remained constant throughout the project life – an unusual feature that contributed to meaningful support to project beneficiaries and a trusted relationship between the Bank and Russia’s courts, the PIU, MOE and MOF. The Borrower’s ICR also makes this point. In addition, the Bank team’s candid reporting, with ratings downgrades, helped counterparts turn the JRSP around.

d) This project is a good example of the value-added the Bank brings to middle-income countries.

The loan amount constituted a very small element of successive RF Federal Targeted Programs (FTPs) for judicial modernization. The project strategy was aligned with the RF judicial modernization strategy. The Bank’s main role was to provide implementation support, policy advice, technical feedback and a menu of alternative approaches for beneficiaries. The JRSP experience reconfirmed that counterparts were more interested in the intellectual support, exchange of knowledge and the implementation discipline associated with a Bank-financed project, rather than in the financing itself. The courts used the JRSP to access Bank support (through the know-how that comes bundled with Bank financing) to conceive, design and implement complex, high-risk, ICT-intensive systems. This is also evident more broadly in the Bank’s relationship with other middle-income countries, where fee-based services and knowledge exchange are becoming increasingly important.

.


Page 43 of 148

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS

A. RESULTS INDICATORS A.1 PDO Indicators

Objective/Outcome: To increase judicial transparency of courts financed by the JRSP

Indicator Name Unit of Measure Baseline Original Target Formally Revised

Target

Actual Achieved at Completion

Availability of data from periodic surveys of users of judicial services

Yes/No N Y Y Y

31‐Dec‐2007 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 100% of the revised target and 65% of the original target. The indicator text was revised at 2012 and 2014 restructurings. The original text of this PDO indicator was "Periodic surveys show improved private sector and public ratings for judicial transparency and efficiency". The 2006 baseline was "46% of respondents expressed confidence in the courts in 2006" according to data from Levada‐Centre, an independent polling entity in Russia. The original end‐project target was "Increased satisfaction (actual experience and perceptions) with judicial transparency and efficiency, measured by 75% increase over baseline." Successive surveys showed the following evolution of responses to this question: 46% (2006), 64% (2010), 73% (2012) and 76% (2017). The change from 46% to 76% represented an increase of 65% over the 2006 baseline against the original target of 75% increase over the baseline. This PDO indicator was revised in 2012 to its present form. The first (2010) survey was published by the Levada Center and by the BEA (the project implementation unit) and the last 2 rounds of surveys conducted in 2017 were published by the Moscow City Court on its website. The results of each survey were made available to the judicial leadership, participating courts, and to the Ministries of Economy and Finance. The courts used the results to determine priorities for improving their work. Equally importantly, the surveys were intended to capture and track, over time, longer‐term outcomes of judicial reforms such as public and businesses' trust and confidence in the judiciary. The survey results over time showed an interesting picture (see ICR Annex 7). For example, (a) 54% of businesses in 2010 felt that their property rights were not protected, but this fell to 43% by 2017, showing greater trust


Page 44 of 148

and confidence in the judiciary and (b) in 2006, 46% of individuals felt that an ordinary Russian citizen could have his/her case considered fairly and without prejudice by the court; this rose to 76% in 2017. Hence, whether seen from the original indicator wording or the revised wording, this PDO indicator was achieved (see ICR text).


Target


Decisions of Moscow City Courts in civil cases enforced, as a percentage of all decisions by Moscow City Courts in civil cases

Percentage 24.00 40.00 40.00 38.00

30‐Jun‐2014 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 95% of target. This indicator measures a key element of judicial efficiency, namely the extent to which the MCC’s judicial decisions in civil cases are actually enforced. This information is automatically generated by the new JRSP‐financed ICT system of the MCC. Due to implementation challenges of this technically complex integrated ICT system beyond the control of the MCC (e.g. this indicator required the MCC system to interface with the federal bailiff service information system), the achievement, while significant, fell somewhat short of the ambitious target. However, the upward trend continues after project closing: in February 2018, the percentage rose to 39.

Objective/Outcome: To increase the efficiency of courts financed by the JRSP


Target


Decisions of Moscow City Courts in civil cases enforced, as a percentage of all decisions by Moscow City Courts in civil cases

Percentage 24.00 40.00 40.00 38.00

30‐Jun‐2014 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017


Page 45 of 148

Comments (achievements against targets): Achievement was 95% of target. This indicator measures a key element of judicial efficiency, namely the extent to which the MCC’s judicial decisions in civil cases are actually enforced. This information is automatically generated by the new JRSP‐financed ICT system of the MCC. Due to implementation challenges of this technically complex integrated ICT system beyond the control of the MCC (e.g. this indicator required the MCC system to interface with the federal bailiff service information system), the achievement, while significant, fell somewhat short of the ambitious target. However, the upward trend continues after project closing: in February 2018, the percentage rose to 39.


Target


Reduction in time taken by Moscow courts to respond to citizen applications

Percentage 0.00 20.00 20.00 53.00

30‐Jun‐2014 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 150% of target. This indicator was added in 2014 to measure the efficiency of response to citizen applications addressed to the Moscow City Courts. In 2014, Moscow courts took 30 days on average to respond to citizen applications. This was targeted to be reduced to 24 days (i.e. 20 percent reduction) by the end of the project. This indicator is an important element of the package of automated reports generated by the integrated ICT system installed in the MCC under the JRSP.


Target


New video‐conferencing facilities (to enable remote participation in court proceedings) are functional in courts financed by the project

Percentage 0.00 100.00 100.00 100.00

31‐Dec‐2007 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017


Page 46 of 148

Comments (achievements against targets): Achievement was 100% of target. This indicator originally covered the SC, the SAC and the commercial court system and was already 100% achieved at the time of the 2012 restructuring. At the 2014 restructuring, with the SAC having ceased functioning from 2014, this indicator covered the SC, the commercial courts and the Moscow City Courts ‐ it was 100% achieved by completion.

A.2 Intermediate Results Indicators

Component: A. Institutionalizing Judicial Transparency And Accountability


Target


Reduction in complaints by litigants (about violations of rights of parties in judicial proceedings) in Moscow courts as a percentage of 2014 baseline

Percentage 0.00 40.00 40.00 30.60

30‐Jun‐2014 31‐Dec‐2014 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 76.5% of target. This indicator measures a key transparency and efficiency‐enhancing result from the JRSP‐financed ICT modernization of the MCC. Due to challenges in implementation of the technically complex MCC integrated ICT system, the achievement, while significant, fell somewhat short of the target at project closing because of the time taken to complete the normative laws applicable, and to complete the interagency linkages with the MCC system. The actual level of achievement is commendable, given the complexity of the system and the number of interagency linkages involved. The upward trend in this result continued after project closing: by February 28, 2018, the figure rose to 31.1, up from 30.6 on December 31, 2017.


Target



Page 47 of 148

Reduction in cases with contested court records (i.e. comments and complaints about transcription), as a percent of all cases in Moscow courts

Percentage 0.00 40.00 40.00 35.20

30‐Jun‐2014 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 88% of target. This indicator measures another key transparency and efficiency‐enhancing result from the JRSP‐financed ICT modernization of the MCC.


Target


Time taken by Moscow lower courts to forward criminal and civil cases to courts of appeal

Months 2.00 1.50 1.50 1.50

30‐Jun‐2014 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 100% of target. This indicator was added at the 2014 restructuring as an intermediate indicator of the process efficiency of the MCC. In 2014, Moscow lower courts took, on average, two months to forward criminal and civil cases to courts of appeal, which advised the MCC to shorten the time taken to forward case records. The JRSP‐financed ICT modernization of the MCC enabled the target to be met.

Component: B. Harnessing ICT for Judicial Transparency and Effectiveness


Target



Page 48 of 148

Percent of courts financed by the JRSP in which new information services are available for users

Percentage 0.00 100.00 100.00 100.00

31‐Dec‐2007 31‐Dec‐2012 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): (Note ‐ this is erroneously included. Please see correct version of this indicator below)


Target


Percent of courts financed by the JRSP in which new information services (information kiosks and electronic access to judicial decisions) are available for users

Percentage 0.00 100.00 100.00 100.00

31‐Dec‐2007 31‐Dec‐2012 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 100% of target. This indicator was revised in 2014 to reflect (a) the retention of the SC, the CC and commercial courts in the JRSP at restructuring, (b) the inclusion of the Moscow City Courts and the Court of Intellectual Rights at that restructuring, and (c) the deletion of the JD and the 900 CGJs.


Target


Reduction in complaints by litigants (about violations of rights of parties in judicial proceedings) in Moscow

Percentage 0.00 40.00 40.00 30.60

30‐Jun‐2014 31‐Dec‐2014 31‐Dec‐2017 31‐Dec‐2017


Page 49 of 148

courts as a percentage of 2014 baseline

Comments (achievements against targets): Achievement was 76.5% of target. This indicator measures a key transparency and efficiency‐enhancing result from the JRSP‐financed ICT modernization of the MCC. Due to challenges in implementation of the technically complex MCC integrated ICT system, the achievement, while significant, fell somewhat short of the target at project closing because of the time taken to complete the normative laws applicable, and to complete the interagency linkages with the MCC system. The actual level of achievement is commendable, given the complexity of the system and the number of interagency linkages involved. The upward trend in this result continued after project closing: by February 28, 2018, the figure rose to 31.1, up from 30.6 on December 31, 2017.


Target


Reduction in cases with contested court records (i.e. comments and complaints about transcription), as a percent of all cases in Moscow courts

Percentage 0.00 40.00 40.00 35.20

30‐Jun‐2014 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 88% of target. This indicator measures another key transparency and efficiency‐enhancing result from the JRSP‐financed ICT modernization of the MCC.


Target


A modern Data Processing Yes/No N Y Y Y


Page 50 of 148

Center for Moscow courts is functional at the Moscow City Court

30‐Jun‐2014 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 100% of target.. A functional and data‐secure Data Processing Center is key to the MCC's e‐processing, e‐services and other transparency‐ and efficiency‐enhancing capabilities financed under the JRSP. The Center was completed in 2016 and demonstrated to the World Bank Country Director and the Governance Global Practice Director.


Target


Time taken by Moscow lower courts to forward criminal and civil cases to courts of appeal

Months 2.00 1.50 1.50 1.50

30‐Jun‐2014 31‐Dec‐2016 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 100% of target. This indicator was added at the 2014 restructuring as an intermediate indicator of the process efficiency of the MCC. In 2014, Moscow lower courts took, on average, two months to forward criminal and civil cases to courts of appeal, which advised the MCC to shorten the time taken to forward case records. The JRSP‐financed ICT modernization of the MCC enabled the target to be met.

Component: C. Strengthening Human Capital


Target


Number of Manpower Trained under the Project (number of people)

Number 0.00 4000.00 4300.00 7703.00

31‐Dec‐2007 31‐Dec‐2014 31‐Dec‐2017 31‐Dec‐2017


Page 51 of 148

Comments (achievements against targets): Achievement was 192.57% of the original target and 179.14% of the revised target. This was added as a Core Indicator at project restructuring in 2012. It pertains to Component C and relates to the training of judges and judicial staff of commercial courts, the MCC and the Supreme Court to increase their ICT proficiency (to complement the ICT systems financed by the JRSP).


Target


Direct project beneficiaries Number 0.00 32000.00 32000.00 39759.00

31‐Dec‐2007 31‐Dec‐2012 31‐Dec‐2017 31‐Dec‐2017

Female beneficiaries Percentage 0.00 40.00 40.00 66.75

31‐Dec‐2007 31‐Dec‐2012 31‐Dec‐2017 31‐Dec‐2017

Comments (achievements against targets): Achievement was 124.25% of target for direct project beneficiaries and 112.56% of target for female beneficiaries. This is a mandatory indicator. Direct project beneficiaries comprise the judges and judicial staff of the Supreme Court, the Constitutional Court, the Supreme Arbitrazh Court, the commercial courts, Moscow City Courts and the Court of Intellectual Rights Protection. The original target for total project beneficiaries was 32,000, while the original target for female beneficiaries was 40% of the total number of direct project beneficiaries, i.e. 23,580 female beneficiaries. Actual end‐project achievement was 39,759 total direct beneficiaries (124.25% of target) and 26,541 direct female beneficiaries (112.56% of target).

Component: D. Project Management and M&E


Page 52 of 148

B. KEY OUTPUTS BY COMPONENT

Objective 1: To strengthen judicial transparency in courts financed by the JRSP

Outcome Indicators 1. Availability of data from periodic surveys of users of judicial services 2. Decisions of Moscow City Courts in civil cases enforced, as a percentage of all

decisions by Moscow City Courts in civil cases

Intermediate Results Indicators

1. Percent of courts financed by the JRSP in which new information services (information kiosks and electronic access to judicial decisions) are available for users

2. Reduction in complaints by litigators (about violation of rights of parties in judicial proceedings) in Moscow courts, as a percentage of 2014 baseline

3. Reduction in cases with contested court records (i.e. comments and complaints about transcription), as a percentage of all cases in Moscow courts

Key Outputs by Component (linked to the achievement of Objective 1)

Component A 1. “Surveys of users of the judicial system”. Five rounds of surveys were

conducted (2010, 2011, 2013, 2014 and 2017). The surveys were technically robust and representative at the national level. The reports and findings were disseminated to the courts, judicial leadership, MOE, MOF and the IAMC.

2. “Development of new criteria and indicators of efficiency of functioning of judicial system”. This October 2009 report proposed new performance criteria and indicators for the judicial system. The criteria and indicators were discussed by stakeholders at two roundtables on May 20, 2009 and August 12, 2009. The final recommendations were reviewed by the judicial leadership: while some were adopted for the courts of general jurisdiction, other (user feedback-based) indicators were applied to the surveys of court users.

3. “Study and Assessment of Interaction between the Courts (CGJs and commercial courts) and Mass Media”. The contract was awarded in August


Page 53 of 148

2010 and led to recommendations for improving interactions between the SAC/commercial courts and the media.

4. “Training in effective interaction between courts and mass media for PR specialists of arbitrazh courts and courts of general jurisdiction”. This output was a direct result of the previous output and opened the door for such joint training of court spokespersons and members of the media for the first time under the JRSP. It later led to a strategy for improved communication between the media and SAC/commercial courts.

5. “Analysis and systematization of decisions of the Constitutional Court of the Russian Federation”. This 2010 contract led to an important outcome: the CC implemented a modern system for taxonomy and classification of CC decisions so that they were more easily searchable and accessible by judges, researchers, lawyers and the public.

6. “Translating and editing services for publishing the reporter "Decrees of the Constitutional Court of the Russian Federation". This 2010 contract led to the translation and publication of important CC decisions, which had been pending for some years.

7. “Analysis and Generalization of Practices of the European Court of Human Rights in the Context of the Constitution of the Russian Federation and the Russian Legislation”. This arose out of a joint request from the CC and the SC to address a key issue – the high and increasing numbers of Russian judicial decisions being appealed to the ECHR (of which the RF was a member) and the non-implementation of many ECHR decisions (mainly by the federal government) which created a growing political, judicial and financial issue. The report examined why so many ECHR decisions were going against the Russian state and whether any Russian constitutional or legal provisions were part of the problem. The result of this assignment was that such issues were analyzed for the first time in an integrated manner. As a result, the courts and executive pinpointed issues that could be addressed by the Russian authorities.


Page 54 of 148

Component B 8. “Modernization of the SC integrated system for document flow management,

development of electronic bank of court decisions, information system, E-library and Internet-portal; development of information system for SC visitors”. This important contract resulted in the modernization of the stated information systems critical to greater transparency of SC functioning. It improved the ability of the SC to upload information and documents on the internet portal and to provide needed information to visitors to the SC and to judges, lawyers and researchers online.

9. “Procurement of hardware and software for virtualization of Supreme Court server resources.” This output was linked to the previous output. It facilitated more efficient and more secure use by the SC of its electronic resources.

10. “Supply and installation of systems of storage of electronic copies of judicial documents for the SAC”. This output enabled the SAC to electronically store, search for and retrieve judicial documents including case files and judicial decisions – increasing its transparency and efficiency.

11. “Development of court proceedings’ audio and video recording system (for the arbitration courts)”. This output enabled commercial courts’ proceedings to be electronically recorded and stored, facilitating transparency: parties could obtain copies and judges could review them when preparing their decisions. It also had reduced complaints about behavior of judges, staff and lawyers since video and audio tapes could be examined when such complaints were lodged.

12. “Supply and Installation of Hardware Facilities and General Software to Build a Physical and Technical Platform for Implementing Main Subsystems of GAS “Pravosudie” in Small Courts of General Jurisdiction” – this activity, proposed by the JD, was not agreed to by the Bank: it proposed to procure significant amounts of ICT equipment for CGJs before the software and integration issues had been resolved to the Bank’s satisfaction. This activity is one of the follow-on results of the output from package D.4 below.


Page 55 of 148

Component D 13. (Package JRSP/2/D.4 – the ‘gateway contract’ referred to in the ICR text.)

“Technical review for technical design and preparation of technical specifications for JD”. In September 2009 this contract was signed with NII “Voskhod” (Russia). The report on Phase 1 of Stage 1 was submitted in October 2009; on Phase 3 of Stage 1 in November 2009; Phase 2 of Stage 1 in December 2009. Stage 1 was fully completed in May 2010. On August 4, 2010 the IAMC approved the results of Stage 1 together with the technical requirements for hardware and software procurement for 900 CGJ under proposed JRSP package B.3.1

Objective/Outcome 2: To strengthen efficiency in courts financed by the JRSP

Outcome Indicators

1. Reduction in time taken by Moscow courts to respond to citizen applications, as a percentage of 2014 baseline 2. New video-conferencing facilities (to enable remote participation in court proceedings) are functional in courts financed by the project

Intermediate Results Indicators

1. Percent of courts financed by the JRSP in which new information services (information kiosks and electronic access to judicial decisions) are available for users 2. Reduction in complaints by litigators (about violation of rights of parties in judicial proceedings) in Moscow courts, as a percentage of 2014 baseline 3. Reduction in cases with contested court records (i.e. comments and complaints about transcription), as a percentage of all cases in Moscow courts 4. A modern Data Processing Center for Moscow courts is functional at the Moscow City Court 5. Time taken by Moscow lower courts to forward criminal and civil cases to courts of appeal 6. Number of Manpower Trained under the Project

Key Outputs by Component (linked to the achievement of Objective 2)

Component B 1. “Supply and installation of an integrated document management system and an


Page 56 of 148

automated system of information posting on the internet portal of the Constitutional Court of the Russian Federation” This output improved the Constitutional Court’s electronic document management and uploading of documents to the internal and external portals.

2. “Retrospective conversion of the decisions and data files of the RF Supreme Court into electronic format”. The contract was signed in July 2008 and completed in March 2010. A high-speed planetary scanner, a magnetic tape library, a server and other equipment were installed in the SC. More than 1.5 million pages of documents and more than 1.1 million cards were scanned, indexed, converted to electronic format and archived for environmentally-friendly and secure storage and easier search and retrieval.

3. “Procurement of equipment for workplaces of judges and employees of the RF Supreme Court”. The contract was signed in February 2008 and completed in August 2008. Modern equipment for 720 work-places, 12 servers, 1 data storage system, 25 notebooks, and more than 800 multi-functional devices (printer/scanner/copier) and printers was supplied and installed. (Some of these would be replaced in 2017 with the 2017 JRSP government contribution from the MOF.)

4. “Development of Uniform System of Corporate E-Mail of Arbitration Courts”. This July 2010 contract provided the SAC and commercial courts with a modern and more secure corporate email system.

5. “Supply and installation of information kiosks for the arbitrazh court system”. This September 2009 contract information kiosks to the SAC and all commercial courts, accessible to the public. The kiosks provided information on court functioning, rules and procedures, how to search for cases, a databank that could pull up court decisions, and whom to contact for problems and complaints.

6. “Development, implementation and support of uniform portal of arbitration courts of Russia”. This August 2010 contract helped design, implement and support the launch and continuation of a modern publicly accessible web portal for the arbitrazh court system of Russia (in Russian and English).


Page 57 of 148

Component C

7. “Exchange of experience with Federal Constitutional Court of Germany” – led to a September 2008 ICT knowledge-exchange visit to Germany between the ICT teams of the RF Constitutional Court and the Constitutional Court of Germany, where actual issues of further ICT development of constitutional courts was discussed.

8. “Knowledge Exchange Visits to Constitutional Court of Austria and European Court of Human Rights in Strasbourg October 2009”. RF Constitutional Court judges and staff visited the Constitutional Court of Austria and the European Court of Human Rights in Strasbourg for knowledge-sharing on ICT implementation.

9. “Knowledge Exchange Visits to Federal and High Courts of Australia February 2010”. RF Constitutional Court judges and staff visited Federal and High Courts of Australia for knowledge exchange on ICT implementation.

10. “Study tour of SAC delegation to the Supreme Court and other courts of Singapore April 2009”. This knowledge sharing visit to the Singapore Supreme Court, magistrates’ courts and district courts gathered information on current practice in settlement of commercial disputes and on use of ICT in justice. The RF delegation also discussed achievements and issues in the developing an ICT network for Russian arbitration courts.

11. “Knowledge Visit of SAC delegation to Supreme Court and other courts of South Korea May 2009”.

12. “Training for employees of arbitrazh courts in use of modern information technologies”. The contract was completed in April 2009: 178 persons were trained under an initial program, 94 under a basic training program, and 6 senior IT officials were trained under a special program for IT leaders.

13. “Training of employees of information divisions of arbitration courts of Russia in use of modern information technology”. The contract was signed in April 2010. 97 IT staff members of arbitration courts were trained.


Page 58 of 148

14. “Training of employees of courts and managers of CGJ and JD to information technologies (standard training courses, logistics)”. About 3000 ICT staff of courts of general jurisdiction and Judicial Department were trained under this contract, which was linked to the ‘gateway contract’ (activity D.4) and to the envisaged implementation of the ICT system for 900 CGJs. Component D

15. “Technical design documentation and preparation of technical specifications for SAC and assessment of the needs of arbitration courts for improved automated judicial record management using advanced ICT”. This output provided a medium- to longer-term vision and draft technical specifications for further modernization of SAC and commercial courts’ ICT capabilities for improved efficiency. The recommendations were never acted upon because the SAC was abolished in 2012 and it took about two years for the new commercial division in the Supreme Court to be staffed and begin functioning. By this time the recommendations had become obsolete due to the judicial reorganization and technological change.


Page 59 of 148

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS

Name Role

Preparation

Olga Schwartz Team Member – Judicial and Legal Reform

Nikolai Soubbotin Country Lawyer

Alexander Balakov Procurement Specialist

Galina S. Kuznetsova Financial Management Specialist

Olga A. Gubareva Team Member ‐ Safeguards

Ljudmilla V. Poznanskaya Team Member ‐ Operations

Ramesh Sivapathasundram Team Member – E‐Governance and ICT

Svetlana G. Golubeva Social Safeguards Specialist

Craig Neal Team Member – E‐Governance and ICT

Friedrich Peloschek Team Member – Judicial Reform

Amitabha Mukherjee Task Team Leader – Institutional and Justice Reform

Supervision/ICR

Amitabha Mukherjee Task Team Leader – Institutional and Justice Reform

Olga Schwartz Team Member – Judicial and Legal Reform

Nikolai Soubbotin Country Lawyer

Alexander Balakov Procurement Specialist

Galina S. Kuznetsova Financial Management Specialist

Susana M. Padilla Team Member – Administrative and Client Support

Ronald N. Hoffer Social Safeguards Specialist

Rajni Bajpai Team Member ‐ ICR

Olga A. Gubareva Team Member ‐ Safeguards

Ljudmilla V. Poznanskaya Team Member ‐ Operations


Page 60 of 148

Ramesh Sivapathasundram Team Member – E‐Governance and ICT

Pratheep Ponraj Team Member – E‐Governance and ICT

Arcadii Capcelea Environmental Safeguards Specialist

Svetlana G. Golubeva Social Safeguards Specialist

Eva Maria Melis Team Member – Judicial Reform

Runyararo Gladys Senderayi Team Member – Judicial Reform

Zohreh Bahreh Bar Team Member – Voice Secondee

Kateryna Elishyieva Team Member – Voice Secondee

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation

FY05 15.313 92,179.84

FY06 53.734 368,879.68

FY07 38.339 286,353.17

FY08 0 ‐1,092.46

FY09 0 0.00

Total 107.39 746,320.23 Supervision/ICR

FY07 7.540 89,971.22

FY08 16.765 98,093.65

FY09 36.773 227,917.55

FY10 39.870 252,772.52

FY11 41.476 252,214.76

FY12 22.740 137,320.16

FY13 20.685 172,952.14

FY14 24.900 148,643.01

FY15 19.567 118,584.00


Page 61 of 148

FY16 11.450 89,989.71

FY17 13.626 99,219.06

FY18 12.050 111,527.34

FY19 .200 1,680.64

Total 267.64 1,800,885.76


Page 62 of 148

ANNEX 3. PROJECT COST BY COMPONENT

Components Amount at Approval

(US$M) Actual at Project Closing (US$M)

Percentage of Approval (US$M)

A. Institutionalizing Judicial Transparency and Accountability

7.15 4.78 66.85

B. Harnessing ICT for Judicial Transparency and Effectiveness

146.10 168.76 115.51

C. Strengthening Human Capital

10.10 7.09 70.20

D. Project Management and M&E

9.06 8.13 89.74

Total 172.41 188.76 109.48


Page 63 of 148

ANNEX 4. EFFICIENCY ANALYSIS

Positive economic effects arose from savings on staff time of judges and other court officials, reduced transactions costs between courts and other governmental institutions, reduced cost of processing information and its provision to citizens and businesses. The estimated direct economic effects (DEE) of JRSP activities considerably exceeded the expenditures incurred on the JRSP. In addition to the DEE, project implementation positively affected the quality and timeliness of the administration of justice and improved access to judicial services for citizens, businesses and professional users. It also contributed to improving actual experiences with, and perceptions about, the judicial system as a whole. This in turn contributed – and will continue to do so over time - to increasing satisfaction of citizens and businesses with the judiciary. JRSP activities also had social effects such as increasing the transparency of courts at all levels; increasing the availability of and access to court decisions, judicial information and judicial processes; and increased protection of rights of litigants and businesses, as the survey data cited in the ICR demonstrate. Taking account of the overall economic effect, the comprehensive economic effect (CEE) demonstrated efficiency in JRSP implementation. The CEE value obtained for the period up to and including 2022 is RUR40 billion (equivalent to US$665.9 million at the end-2017 exchange rate or US$614.7 million based on the average weighted exchange rate provided by the Russian Federation Central Bank for the corresponding years). The NPV and EIRR values for the Project are: NPV: RUR7.3 billion (equivalent to US$121.8 million at the end-2017 exchange rate) EIRR: 63 percent The dropping of the 900 small CGJs from the project and the 2014 addition of the MCC to the project also significantly enhanced the efficiency and impact of JRSP resources. Many of the 900 CGJs originally included under the project were merged and reorganized during the last 5 years. These small courts (with 1 to 5 judges) handle about 500 cases annually (for courts with 1-3 judges) and sometimes up to 1,000 cases annually (for courts with 3-5 judges). Taking an annual average of 800 cases per small court (on the high side), the total number of cases considered annually by 900 CGJs originally included in the project would be 720,000. In contrast, the number of cases annually considered by the Moscow region courts is about 1.5 million. Therefore the exclusion of the 900 CGJs and the inclusion of the MCC practically doubled the effect of the project investments into Russia’s CGJs. In addition, no funds for ICT modernization were allocated to the MCC under the FTP – the JRSP resources helped the MCC to significantly improve the efficiency of their operations. Therefore, not only were there no negative impacts of dropping 900 CGJs, adding the MCC with its higher case volume conferred significant additional efficiency to Russia’s overall judicial system and provided an opportunity to make significant improvements to the ICT system currently deployed across other CGJs with Russian budgetary financing.


Page 64 of 148

A stress test was employed to assess the sustainability of JRSP results: all key assumptions underlying the CEE estimations were simultaneously varied from +10% to -10%. This facilitated examination of system behavior under stringent conditions, while the probability of the mistakes in all indicators at the same time is highly unlikely). The stress-test showed high sustainability of JRSP results. Methodological summary. The JRSP was implemented within a larger context of major judicial reforms initiated by the government and mostly financed under Federal Targeted Programs (FTPs). Efficiency was calculated as a combination of direct economic effects of project activities, and as part of the overall effect of reform of Russia’s judicial system, of which the JRSP was an element. JRSP effects are divided into 2 categories: direct and indirect. Direct effects were calculated separately for each activity. Indirect effects were calculated for the entire JRSP as part of the effect of the entire judicial reform. The combined effect is the sum of all direct effects for JRSP activities and of the overall effect of the entire JRSP. The methodology used was to (a) identify JRSP activities with direct effects from project components and calculate the sum of all direct effects (i.e. the cumulative direct effect or CDE), (b) calculate the overall effect (OE) of the JRSP as part of the effect of the entire judicial reform using appropriate methodology summarized in the Annex (based on the detailed project economic analysis) and (c) deduct the amount of JRSP financing from the sum of CDE and OE to arrive at the total comprehensive economic effect (CEE). JRSP activities with direct economic effects. The table below shows the most important JRSP activities with direct economic effects with a summary of the effects identified (Table 64.1).

Table 4.1. Activities with direct economic effects

Activity number and description Description of direct EE

B.2.1.1. and B.2.1.2. Conversion of RF SC decisions and data files into electronic format (Stages 1 and 2)

Reduction in staff time and cost associated with searching archival documents

B.2.2.3 Modernization of SC Information System

Reduction in staff time and cost associated with scanning cases and other files/documents

Reduction in staff time and cost of IT specialists in repairing/replacing computers

B.2.2.4.a Procurement of hardware and software for virtualization of computer resources of the SC and provision of related virtualization system

Savings from procurement, deployment and maintenance of physical servers (PS) from creation of required operational infrastructure


Page 65 of 148

Activity number and description Description of direct EE implementation services

B.2.2.9 Development and Implementation of the SC Information System “Case Processing and Judicial Documents Flow"

Savings by citizens and organizations as they (and/or their representatives) do not need to be physically present to file procedural requests

B.2.3.1 Procurement of equipment and development of communication channels for organization of remote proceedings

Cost saving on prisoner transfers Cost saving on procedural fees Cost saving for relatives of prisoners Cost saving on training

Cost saving on investigation activities

B.2.3.2 Supply of Additional VC Equipment to SC and FPS Offices

Saving on procurement of additional MGTS telephone numbers through the use of digital telephone station (DTS)


Page 66 of 148


B.3.1.2. A turnkey creation of Integrated Information System of Moscow city general jurisdiction courts

Saving of labor cost of judicial staff associated with filing of cases in courts of higher instance and passing of judgement by courts of higher instance


Saving of labor cost of citizens and organizations’ representatives associated with submission of claims through the Internet

Saving of labor cost of citizens and organizations’ representatives on submission of applications for examination of case materials through the Internet

Saving of staff cost and time associated with maintenance of databases of new employees


B.4.1.1. Development of corporate telephone network of commercial courts

Savings on long-distance calls

B.4.1.5.2. Development of new functional modules of arbitration case management system specific to the CIR based on current legislation

Labor saving associated with preparing the state statistical data

B.4.1.9.1 Procurement of servers and active network equipment

Savings on international calls


Page 67 of 148


Savings on buying telephone numbers from telephone company

B.4.1.9.3 Procurement of Automated System for Document Flow Management

Reduced material expenses and labor costs of administrative personnel associated with maintenance of non-judicial paperwork

В.4.1.9.4 Procurement of videoconferencing equipment for the Presidium of the CIR

Saving of funds of citizens and organizations as they (and/or their representatives) do not need to be physically present at court sessions

B.4.2.2. Development, implementation and maintenance of a single portal of Russian commercial courts

Reduced time spent by citizens and organizations to obtain information

B.4.2.3/i-iv Supply and Installation of systems of storage of electronic copies of judicial documents

Saving of labor cost associated with registration of and providing access to the arbitration case

Saving of labor cost of IT specialists associated with the maintenance of equipment

Saving of funds for procurement, deployment and maintenance of physical servers (PS) and for creation of the required operational infrastructure.

B.4.2.4.1 Audio Recording System of Court Sessions

Saving on reduced number of appeals against the actions of judges

The lack of direct economic effects or ability to identify them does not imply their absence, as such activities can be efficiently undertaken and be effective. However, such effects occur at a somewhat different level from the impact on the state of the entire judicial system. Typical examples include training and skills upgrading of judges and judicial staff, knowledge exchange and access to international good practice for judges. It is difficult to assess the direct economic


Page 68 of 148

effects of such activities, yet they do impact Russia’s court system. Overall effect of JRSP. The JRSP contributed to the overall effect of the system-wide judicial reform over the years. The significance of this contribution can be estimated based on certain conservative assumptions, as a percentage of the overall effect of the reform. To measure the effect of the entire judicial reform, the analysis used the global rankings of countries based on a methodology that measures the quality of the judicial system and the extent of protection of the rights of businesses and citizens. Russia’s improved rankings on such ratings during judicial reform implementation could indicate that the reform had a positive impact on Russia’s country's economic and social climate. The World Bank’s annual Doing Business (DB) ratings was found to be appropriate for this analysis. Within the DB ratings, those associated with enforcing contracts are almost entirely based on an analysis of judicial system efficiency and protection of legal rights of parties to contracts being enforced. Hence the changes in the scores and rankings for this indicator may directly reflect changes in the judicial system, including as a result of reforms. Three indicators – obtaining credit, resolving insolvency and protecting minority investors - reflect the level of protection of rights of businesses, specifically creditors and shareholders, and the efficiency of credit and bankruptcy institutions. Each of these three topics includes issues not directly connected with the protection of rights or the judicial system. However, other issues allow us to associate the Distance To Frontier (DTF) dynamics with changes in a country’s judicial system. The other six topics are more distantly connected to the judicial system and cannot be used. Thus, 4 out of 10 DB indicators can be considered to be connected to judicial reforms, in whole or in substantial part. Since all indicators for calculating the total DTF have equal weight, the share of the judicial reform contribution to the overall effect from improved business climate in the DB index can be assumed to be 0.4* [i.e. it is a reduction factor because of partially indirect connection]. For measuring the economic effect of the judicial reform, the approach described in “Does Doing Business Matter For Foreign Direct Investment?” (available on the World Bank website) was used to identify correlation with indicators that could be used to assess the net effect for the economy. Specifically, the coefficient of correlation between cumulative DTF of various countries and internal lending interest rates (LIR) in the non-financial sector was used. The analysis reasonably suggests a correlation between cumulative DTF and LIR. A linear regression coefficient has been estimated based on the analysis results. The conclusion is that correlation between these indices is negative: the higher the country's rating (i.e. the country's DTF), the lower is the interest rates on loans to non-financial corporations. As to the effect of the reforms, this correlation implies that the higher the DB ranking since the reform began, the lower the interest rates on loans to natural and legal persons should be. In this case, the direct effect will be received by borrowers which, in turn, generates follow-on effects in the economy. Therefore, the total annual effect from reducing lending interest rates as a result of DTF growth can be assessed as the difference (reduction) of lending interest rates (LIR) calculated using the linear regression coefficient multiplied by the cumulative volume of non-financial sector lending. The equation to calculate the annual effect of the JRSP is summarized as follows:


Page 69 of 148

Э ∆ ∗ ∗ ∗ 0.4 ∗ ∗

where: ∆ – DTF variation from the basic primary level (prior to the start of reforms in question) for i year; k - a negative linear regression coefficient for DTF and LIR

– volume of non-financial sector lending in i year 0.4 - the share of indices connected with the judicial reform in the total DTF

– reduction factor (determined by the professional assessment method) which takes into account that some DTF indices selected for analysis have somewhat indirect connections with project activities and FTPs on the development of Russia’s judicial system;

– amount of financing of the JRSP; – amount of financing under FTPs on development of Russia’s judicial system for

2007-2016. There is no reason to believe that the DTF direction for Russia will turn negative in the near future and/or the scores/ratings will decrease. At the same time, using a conservative approach, 2017 DTF values were used to estimate the overall effect of JRSP for five years after project closing. The total overall effect of the JRSP is the sum of annual effects. Calculation of JRSP total comprehensive economic effect (CEE). The JRSP’s CEE is the sum of direct economic effects and overall economic effect of the Project, less cumulative actual JRSP expenditures. The CEE is calculated as follows:

∑ ∑ Р , where DEEi - direct economic effects of the JRSP OEE - overall economic effect of the JRSP ∑ Р – amount of expenditures on the JRSP

Inflation estimates used data from the Federal State Statistics Service, while inflation forecasts up to 2022 of the Ministry of Economic Development of the Russian Federation were used16.

16 http://economy.gov.ru/minec/about/structure/depmacro/2017271001


Page 70 of 148

Table 1. Economic effect of videoconferencing implementation

Type of sets

Quantity of sets (2017)

Effect in prices for

2014, thousand

rubles/year for 1 set

Effect, thousand rubles

2013 2014 2015 2016 2017 2018 2019 2020 2021 2022

RF SC 13 1 707 57 602 57 546 48 162 38 778 29 394 30 570 31 793 33 065 34 387 35 763 Court of level I 777 1 020 350 290 331 516 537 557 743 598 949 640 987 625 1 027 130 1 068 215 1 110 944 1 155 382

Court of the level II 2479 -155 -39 102 -203 830 -305 698 -407 566 -509 435 -529 812 -551 004 -573 045 -595 966 -619 805

TOTAL: 368 790 185 232 280 021 374 810 469 599 488 383 507 919 528 235 549 365 571 339 Table 2. Economic effect of digital telephone connection implementation

MGTS line rental fee, Rubles/ 1 telephone number monthly

Numbering capacity (general)

Numbering capacity (MGTS)

DEE, thousand rubles

2017 2018 2019 2020 2021 2022

340 1200 900 1 224,00 1 272,96 1 323,88 1 376,83 1 431,91 1 489,18


Page 71 of 148

Table 3. The effects of activity B.3.1.2 and formulae for the calculation

Effect Main driver Calculation formula Description of terms

Saving of labor cost of judicial staff associated with filing of cases in the court of the higher instance and passing of judgement of the court of the higher instance in the court of the first instance 17

The quantity of falsified judgments of the first-instance courts.

∗ ∗ С / - The quantity of falsified judgments of the first-instance courts. TS - Time saving on the data input for 1 case (man/hours) С / –The court personnel man hour cost


Quantity of complaints ∗ ∗ / – Number of complaints during the basic period – Number of complaints during the present period

–Average labour cost on consideration of 1 complaint (man/hour) С / –Court personnel man hour cost

Saving of labor cost of citizens and organizations’ representatives associated with submission of claims through the Internet

Quantity of the claims submitted though the Internet

∗ ∗ / –number of the claims submitted though the Internet –Average time expenditures on a claim submission, hour

С / – Applicant’s (or his representative’s) man hour cost18

Saving of labor cost of citizens and organizations’ representatives associated with submission of applications for examination of case materials through the Internet

Number of applications submitted though the Internet

∗ ∗ /

–Number of applications for examination of materials of cases submitted though the Internet

–average time expenditures on a claim submission, hour С / – Applicant’s (or his representative’s) man hour cost18

Saving of labor cost of administrative personnel associated with the maintenance of databases of new employees

Number of the new personnel in a year

∗ ∗ С / –Average number of new employees a year –Time saving on the input of the data on 1 new employee (man/hour)

С / – Administrative personnel man/hour cost

17 Time saving due to elimination of manual data input from paper documents by automating document flow 18 Due to difficulty to direct assessments, data on average salary in the Russian Federation has been used: http://www.gks.ru/bgd/free/B09_03/IssWWW.exe/Stg/d02/193.htm


Page 72 of 148

Effect Main driver Calculation formula Description of terms


Number of writs obligatory passed to FBS

∗ С С ∗ С ∗ –Number of writs obligatory passed to FBS С –Cost of a writ obligatory form С –Cost of a certified mail

–Number of a certified mail for 1 writ obligatory –Number of uninsured mail for 1 writ obligatory

С –Cost of an uninsured mail


Page 73 of 148

Table 4. Effects and the main assumptions/parameters for their estimation

Effects Parameters Unit of measure 2017 2018 2019 2020 2021 2022 Saving of labor cost of judicial staff associated with filing of cases in the court of the higher instance and passing of judgement of the court of the higher instance in the court of the first instance

Saving of time on filing of cases hours 0,5 0,5 0,5 0,5 0,5 0,5

Administrative personnel man hour cost RUR

220 229 238 247 257 267


Judges man hour cost RUR 1 539 1 601 1 665 1 731 1 800 1 872

Average time on handling of complaints Person-hours

60 60 60 60 60 60 Administrative personnel man hour cost RUR

220 229 238 247 257 267


Number of claims submitted by citizens and organizations’ representatives in person at courts’ reception

pc. 1 488 1 488 1 488 1 488 1 488 1 488

Average time required on claims submission in person hour

4 4 4 4 4 4

Applicant´s average man hour cost RUR 248 258 268 279 290 301

Saving of labor cost of citizens and organizations’ representatives associated with submission of court claims through the Internet

Number of claims submitted by citizens and organizations’ representatives in person at courts’ reception

pc. n/a n/a n/a n/a n/a n/a

Average time required on claims submission in person hour

4 4 4 4 4 4

Applicant´s average man hour cost RUR 248 258 268 279 290 301


Cost of the writ obligatory form RUR 0,80 0,83 0,87 0,90 0,94 0,97 Number of a certified mail for 1 writ obligatory pc.

5


Page 74 of 148

Effects Parameters Unit of measure 2017 2018 2019 2020 2021 2022

Number of uninsured mail for 1 writ pc. 0

Average cost of certified mail RUR 48.38 50.32 52.33 54.42 56.60 58.86

Average cost of uninsured mail RUR 25.96 27.00 28.08 29.20 30.37 31.58


Page 75 of 148

Table 5. Estimation of the economic effect of activity B.3.1.2., thousand rubles Effect 2018 2019 2020 2021 2022 Total

Saving of labor cost of judicial staff associated with filing of cases in the court of the higher instance and passing of judgement of the court of the higher instance in the court of the first instance

22 947 23 069 23 164 23 231 23 489 115 899

Saving of labor cost of judicial staff associated with handling of complaints 39 038 50 864 63 575 77 220 91 856 322 554


1 534 1 595 1 659 1 725 1 794 8 306

Saving of labor cost of citizens and organizations’ representatives associated with submission of court claims through the Internet

16 819 17 492 18 192 18 920 19 676 91 099

Saving of material and postage costs using electronic exchange with FBS 109 171 131 703 153 408 172 308 188 160 754 751

Total 189 508 224 724 259 998 293 404 324 976 1 292 609


Page 76 of 148

Table 6. JRSP overall effect (by years)

2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017** 2018** 2019** 2020** 2021** 2022** DTF of the Russian Federation

53.06 53.79 56.83 54.93 54.32 56.66 58.93 66.00 71.25 73.20 73.20 73.20 73.20 73.20 73.20 73.20

Funding of the FTP “The development of the Russian Federation Judiciary system”, million RUR

6 380 8 937 10 153 10 930 13 532 10 793 10 000 9 454 9 354 11 725

Average annual rate of exchange RUR/US$

25.577 24.8553 31.7231 30.3692 29.3874 31.093 31.848 38.4217 60.9579 67.0349 58.3344

Funding of the FTP “The development of the Russian Federation Judiciary system”, million US$

249.44 359.56 320.05 359.90 460.47 347.12 313.99 246.06 153.45 174.91

Bank Lending to non-financial sector at the end of reporting year, billion RUR

12 287.10

16 526.90

16 115.50

18 147.70

23 266.20

27 708.50

32 456.30

40 865.00

43 985.00

40 938.60

42 000.00

42 000.00

42 000.00

42 000.00

42 000.00

42 000.00

Bank Lending to non-financial sector at the end of reporting year, million US$

480 396 664 924 508 005 597 569 791 706 891 149 1 019 100

1 063 591 721 563 610 705 720 000 720 000 720 000 720 000 720 000 720 000


Page 77 of 148

2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017** 2018** 2019** 2020** 2021** 2022** Expected effect, mil US$ (average data for 2005 and 2014)

0.00 0.00 25.01 11.04 6.82 41.37 84.69 209.89 203.60 191.56 225.85 225.85 225.85 225.85 225.85 225.85

Minimal effect, mil.US$ (average data for 2005 and 2014)

0.00 0.00 6.58 2.91 1.80 10.89 22.30 55.26 53.61 50.44 59.46 59.46 59.46 59.46 59.46 59.46

Expected effect, mil RUR (average data for 2005 and 2014)

0.00 0.00 793.32 335.25 200.50 1 286.41 2 697.27 8 064.28 12 411.14

12 841.42

13 160.69

14 228.31

14 797.44

15 389.34

16 004.91

16 645.11

Minimal effect, mil RUR (average data for 2005 and 2014)

0.00 0.00 208.87 88.27 52.79 338.70 710.17 2 123.25 3 267.74 3 381.03 3 465.09 3 746.19 3 896.03 4 051.88 4 213.95 4 382.51

Table 7. CEE Calculation parameters of the JRSP

Parameter 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 Costs under the JRSP 175 202 170 259

217 303 208 029 201 304 913 165

364 659 496 288

1 269 871

3 427 234

1 665 169 0 0 0 0 0

Direct effects 0 0 0 0 0 156 850 613 380 484 742 622 218 679 916

1 088 179

1 450 599

1 843 585

2 263 147

2 706 968

3 180 690

Overall (indirect) effect 0 0

208 874 88 270 52 789 338 700

710 166

2 123 254

3 267 744

3 381 033

3 465 092

3 746 187

3 896 035

4 051 876

4 213 951

4 382 509

ECF -175 202

-170 259 -8 429

-119 759

-148 515

-417 616

958 887

2 111 708

2 620 091 633 716

2 888 102

5 196 786

5 739 620

6 315 024

6 920 919

7 563 200


Page 78 of 148

The total direct economic effect is 15.2 billion rubles (around $253.0 million dollars at current exchange rate). The total overall effect is 33.9 billion rubles (around $565.4 million dollars at the current exchange rate).


Page 79 of 148

ANNEX 5. BORROWER COMMENTS

Ministry of Finance of the Russian Federation

Comments to

Implementation Completion and Results Report (Report)

on the Judicial Reform Support Project (Project)

prepared by IBRD

IBRD Loan No. 4849-RU

Ministry of Finance of the Russian Federation (MOF) makes special mention of a large

volume of work has been achieved by the IBRD team to analyze and summarize project results

(the Bank team has analyzed, among other things, the results of project research and analytical

activities) and progress. Considering project-specific conditions (MOED acting as implementing

agency and chief budget funds administrator for the project solely implemented for the judiciary,

three highest-level courts acting as the key project beneficiaries, project focus on providing support

for modernization rather than development of the existing national judicial system, project role to

augment federal targeted programs on judicial system modernization), findings and conclusions

expressed by the Bank team in the Report represent an interest for the IBRD potential borrowers

and contribute to the cumulative implementation experience of projects financed by the IBRD and

other international financial institutions.

The IBRD rating of project efficacy and efficiency as substantial (page 6 of the original

version, references are made to the English version of the Report provided by the IBRD) seems to

be reasonable, reflecting project contribution to strengthening transparency and efficiency of all

judicial authorities involved in the Project. However, we have the following comments on a

number of aspects outlined in the report.

We have to note that the rating of the relevance and importance (significance) of the Project

Development Objective (PDO) to assist the Borrower to strengthen judicial transparency and

efficiency of selected courts through the implementation of information systems and judicial

training (page 6 of the Report) as modest does not seem to be sufficiently justified.

The PDOs were not focused on the issues of judicial independence, as was rightly noted

by the IBRD team. This seems to be reasonable as the tasks to ensure independence of judges and

entire judiciary throughout the period of implementation of the Project were among the key tasks

of the federal targeted programs for judicial system development, e.g. the federal targeted program


Page 80 of 148

on development of Russia’s judicial system for 2007-2012 and the federal targeted program on

development of Russia’s judicial system for 2013-2020/. The Project was originally designed to

supplement the federal targeted programs and other government programs on judicial system

development. Hence, it seems to us that the fact that PDOs have addressed some, but not all,

national judicial development priorities should not be seen as a weakness of the above PDO.

The relevance of informatization issues is emphasized in RF Presidential Decree No. 204

dated 07.05.2018 “On the national agenda and strategic development objectives of the Russian

Federation for the period up to 2024”. This decree orders the RF government to ensure that a

series of national development objectives of the Russian Federation for the period up to 2024,

including enhanced deployment of digital technology in the economy and social sector, are

achieved. This is expected to be achieved, among other things, by introducing digital technology

and platform solutions in the area of public administration and delivery of public services,

including for citizens and small and medium-sized enterprises.

The JRSP-financed activities undertaken to ensure automation of workflow and document

flow based on modern ICT in Moscow courts of general jurisdiction with the aim to increase degree

of electronic interaction in judiciary, interagency interaction and interaction with litigants, directly

address the national tasks related to the development of information society in Russia and future

national development objectives described in the above-mentioned RF Presidential Decree.

The Bank team noted on page 18 (paragraph 20) of the Report that the ICR guidelines

require the relevance of PDOs to be assessed with respect to the CPS goals and priorities at the

time of project closing. However, PDOs can be assessed only with respect to the CPS for the period

up to 2016. It seems that a lack of CPS for the Russian Federation at the time of project closing

as of December 2017 does not constitute sufficient grounds for reducing the ranking of PDO

relevance by project closing.

On the basis of the above, we suggest that the relevance of PDO should be rated as High.

The Bank performance under the Project (page 6 of the Report) is rated by the Bank team

as Moderately Satisfactory. This ranking seems to be undervalued, not reflecting the actual role

of the IBRD leadership and the Bank team in the implementation of the Project. The Borrower's

JRSP implementation completion report submitted by MOF to IBRD provides detailed

justification for high ranking of IBRD performance under the Project. In particular, positive

factors include stable Bank team throughout the project life contributing to high professional level

of support provided by IBRD with respect to all substantive project-related aspects. The distinctive


Page 81 of 148

feature of the Bank team throughout the project life was ongoing monitoring conducted by the

Bank team members with respect to the RF judicial system modernization, including

modernization of the legal and regulatory base relevant to the judiciary. With the meaningful

involvement of the Bank team in dealing with problematic areas relating to improvement of

government organization and management, the Bank timely identified implementation challenges

and supported all reasonable proposals on project restructurings and extensions, which were key

to achievement of project objectives and almost full disbursement of loan funds provided to the

Russian Federation for implementing priority measures aimed to improve judicial transparency

and effectiveness as identified by judicial stakeholders.

On the basis of the above, we suggest that Bank performance should be rated as Highly

Satisfactory.

We assume that overall rating of project performance should be improved to Satisfactory,

taking into account comments above.

The wording of the original project development objective as specified on page 12 of the

Report (paragraph 8) is to strengthen judicial transparency and efficiency in courts financed by the

JRSP, while the PDO wording used in the Loan Agreement is to assist the Borrower to strengthen

judicial transparency and efficiency of selected courts through the implementation of information

systems and judicial training.

We suggest that PDO wording as specified in the Loan Agreement should be used since it

includes the proposed methods for achieving development objective which seem to be substantial

in terms of assessing the project results.

It is stated on page 2 of the Report that the amount of US$ 49 902 853 was disbursed out

of the proceeds of Loan 4849-RU to finance the Project. It is stated in the Borrower's

Implementation Completion Report that after all eligible expenditures had been paid by project

closing, the Foundation for Enterprise Restructuring repaid to the IBRD on April 25, 2018 the

undisbursed amount of loan proceeds equal to US$ 101 106.11 from the Designated Account.

According to the Borrower's Report, the disbursed amount is equal to US$ 49 898 893.89. There

might be a technical error as a cause of data discrepancies because the IBRD team has used for

preparation of the report the data from the IBRD financial system which has not been updated with

the final disbursement data.

On the basis of the above, we suggest that disbursement data should be reconfirmed.

The data on estimated and actual costs by components is cited on pp. 12-13 of the Report


Page 82 of 148

in the description of project components and in Annex 3 to the Report. The data provided is

inconsistent with the actual data obtained from the completed project procurement plan at the time

of preparing the Borrower's Implementation Completion Report. A comparative table below shows

data from the Borrower's Report and the IBRD Report.

Component/Subcomponent Name

Scope of financing, US$ mln.

Planned (Appraisal Document)

Actual disbursed

based on data from

procurement plan included

in the Borrower's

Report

Actual

disbursed based on data

from IBRD Report

A. Institutionalizing judicial transparency and accountability

7.15 4.78 6.15

B. Use of Information and Communications Technology for Judicial Transparency and Effectiveness

146.10 168.76 161.10

B.1. Constitutional Court 5.85 5.62 -

B.2. Supreme Court 26.10 69.25 -

B.3. Judicial Department and Courts of General Jurisdiction

65.65 57.59 -

В.4. Supreme Commercial Court and commercial courts

48.50 36.30 -

C. Strengthening Human Capital 10.10 7.09 10.10

D. Project Management, Monitoring and Evaluation

9.06 8.13 9.75

TOTAL 172.41 188.76 187.10

There might be a technical error as a cause of data discrepancies because the IBRD team

has used for preparation of the report the project procurement plan which has not been updated in

the IBRD data system with contract payments made during grace period provided by the IBRD


Page 83 of 148

(January-April 2018).

On the basis of the above, we suggest that data on project financing by components should

be reconfirmed.

It is stated in paragraph 13(a) on page 16 and in paragraph 18 on page 17 of the Report that

the Supreme Arbitrazh Court (SAC) was abolished in 2012. There seems to be a misprint since

SAC had terminated its operations on August 5, 2014 as per the federal constitutional laws No. 2-

FKZ and No. 3-FKZ dated 05.02.2014.

We suggest that the date for termination of SAC operations should be 2014.


Page 84 of 148

ANNEX 6. OVERVIEW OF KEY JUDICIAL AND LEGAL REFORMS

This Annex summarizes key judicial and legal reform and modernization measures in the Russian Federation which formed the backdrop to JRSP design and implementation. The 1991 Concept of Judicial Reform. The Supreme Soviet Resolution of 24 October 1991 of the Russian Soviet Federated Socialist Republic (RSFSR) enacted the Concept of Judicial Reform. This is generally understood to have signaled the beginning of the modernization of Russia’s judicial system. This Concept articulated the priorities of judicial reform in the nineties in the Russian Federation. Public trust in the judicial system at that time was low. Analytical research of the time, such as VCIOM19 studies of 1989-1991 of the omnibus court system, did not separate courts from other law enforcement and justice entities. VCIOM data indicated that society was divided on trust in courts and prosecution: about 20 percent fully trusted these bodies, 20-33 percent did not trust them at all, and 33-40 percent did not fully trust these bodies. Experts at the time were of the view that factors affecting trust included the level of judicial independence, especially from the executive, powerful private actors and public opinion; fairness; corruption; red tape; and the speed of enforcement of judicial decisions. 1991 establishment of Constitutional Court. The 1991 Concept – approved when the USSR was still in existence - proposed to make the judicial branch of power strong and efficient, independent from executive and legislative branches, to reform the bar (advokatura) and the prosecution service, and to amend criminal procedure to raise the standard of proof and introduce judicial control over preliminary investigations. Judicial reform was declared one of the main State priorities. The Law “On RSFSR Constitutional Court” was adopted in 1991, establishing the CC. The creation of the CC was a significant breakthrough: its main responsibilities were to review federal laws, other normative acts and international agreements of the RF for compliance with the RF Constitution. The CC was authorized to review the constitutionality of established practice in applying the law, and to issue legal opinions at the request of high RF state bodies (e.g. RSFSR People’s Deputies’ Congress and the RSFSR Supreme Soviet) and under its own initiative. The Law on the CC was developed in accordance with high European standards and for the first time introduced such principles as security of judges’ tenure and their accountability only to the law. The 1991 Law prohibited CC justices from involvement in politics or to perform any other paid activities except for teaching and creative work. 1991 establishment of commercial court system. The 1991 Law “On Arbitrazh Courts” established a Supreme Arbitrazh Court and a system of State commercial or economic (arbitrazh) courts on the lines of the system of State arbitration existing in Soviet times for state-owned enterprises. The SAC performed, for the commercial court system, a role analogous to that of the SC for the CGJ. 1992 independence and status of judges protected by law. The 1992 Law “On the Status of Judges” of 23 May 1992 expanded the standards applied to CC justices to all judges across Russia:

19 Russia Public Opinion Research Center (Vserossiiski Centr Izucheniya Obschestvennogo Mnenia). At the time referred to – All-Union Public Opinion Research Center.


Page 85 of 148

it enshrined the autonomy of judicial power and its independence from legislative and executive branches, introduced security of tenure for all judges (a judge could be removed from office only on his/her personal request), personal inviolability of judges (interrogation, search or arrest of a judge was possible only with the consent of the High Qualification Collegium of Judges or lower bodies of judicial self-government), subordination only to the law and increased salaries of judges and other material benefits. It institutionalized the judicial community, including creating bodies of judicial self-governance and judicial qualification collegiums within them having full control over judges’ discipline and their removal for cause. Judges’ procedural powers were widened. The Criminal Procedure Code was amended to introduce judicial control over arrest and detention: the main powers to authorize arrests and detentions still belonged to the prosecution but the defendant acquired the right to appeal against such decisions to the courts, which were to review not only the legality of the arrest (i.e. formal compliance with legal provisions) but also its justification (i.e. whether there were appropriate grounds for the arrest). 1993 reintroduction of jury trials. In 1993 jury trials were reintroduced, a milestone in Russia’s judicial reforms. Jury trials existed in Tsarist Russia, since the 1864 judicial reform, but were later repealed by the Soviet state. Jury trials – by empowering juries alone, not judges, to pronounce guilt - were intended to make judicial proceedings more objective and independent and reduce corruption, bribery and undue/inappropriate influence on judges. Due to budget constraints, jury trials were first introduced as an experiment in some regions but it became the first stage of the full-fledged reintroduction of jury trials in Russia. When citizens’ right to trial by jury was enshrined in the 1993 RF Constitution, the introduction of jury trials in all regions became inevitable, although due to different obstacles (e.g. budget constraints and turbulence in some regions) jury trials could not be fully introduced across Russia until 2003 (indeed it was introduced in the Chechen Republic only in 2010). 1991-1993 – other reforms. During 1990-93, numerous other reforms were introduced by highly respected and reform-minded legal scholars and practitioners who began to shape the emerging judiciary in Russia. For example, key principles and guarantees of judicial independence and of due process (especially with regard to criminal procedure) were reflected in relevant laws and then in the new 1993 Constitution. 1996 - administrative and operational independence of the judiciary from the executive. The next step for Russia’s courts was to gain administrative and operational independence from the executive. In 1996 the Federal Constitutional Law “On Judicial System of the Russian Federation” repealed the existing system of administrative control over the judiciary by the Ministry of Justice (MOJ) and established a Judicial Department (JD) under the RF SC. The JD had already been created in 1998, but the 1996 law gave it true authority and control over CGJs: as a result, the judiciary – through the JD - gained actual control over financing of federal CGJs20. The structure and functions of the JD are in the Federal Law “On Judicial Department under the Supreme Court of the Russian Federation”. The Law tasked the JD with personnel, organizational and material

20 Because the arbitrazh court system was established from scratch in 1991, the SAC already had control over financing of the commercial court system – this was already reflected in relevant legislation. This was also the case with the CC. Both the CC and the SAC established administrative and financial departments within their respective structures.)


Page 86 of 148

support to the courts of general jurisdiction21. In addition, the JD also provides material and technical support to the bodies of judicial self-government. 1998 – reintroduction of Justices of the Peace (JPs). A 1998 landmark Federal Law “On Justices of the Peace in the Russian Federation” restored the institution of JPs which first appeared in Russia as part of the reforms of Alexander II in the late nineteenth century. According to the Law JPs became regional judges of general jurisdiction, appointed by regional legislatures. Significantly, all JP-related expenditures (salaries, allowances, other operating and capital costs) were to be borne by regional governments. JPs’ jurisdiction on criminal and civil cases was set in federal procedural law, although the jurisdiction with regard to administrative offences was to be set by regional legislation. The main tasks of the JPs were to “guard and protect rights, freedoms and legitimate interests of individuals and legal entities” by adjudicating petty criminal cases (i.e. offences with a maximum sentence of up to three years imprisonment), the bulk of administrative offenses and small civil claims up to RUR50,000. The restoration of the institution of JPs was driven by growing case backlogs in courts and public pressure for “fast, right and fair” courts22 - district-level CGJs had become overloaded with cases and could not cope with the rising workload and increasing delays in processing cases, especially civil disputes. In addition, most district courts were located in district centers and were not easily accessible for rural dwellers. The JPs fulfilled the need for arrangements to swiftly adjudicate uncomplicated and frequent disputes, and located closer to the population they served. Budgetary constraints since 1991 and how they were addressed. The above reforms could not be fully implemented due to insufficient budgetary funding and frequent budget sequestrations. According to the 1991 RSFSR Law “On Fundamental Principles of the Budget Organization and Budgetary Process” the state budget was developed by the Council of Ministers and had to be balanced. If the budget was exceeded or if revenues fell, expenditures had to be sequestrated. Such sequestration resulted in proportional lowering of monthly state allocations and expenditures on all budget items, except protected ones, during the fiscal year. For example, in the draft budget for 1996 the MOJ requested a minimum of RUR5.5 billion. This request was reduced and the 1996 budget provided only RUR1.9 billion instead of the amount requested by the MOJ23: the approved budget of the CGJs was less than half the budget request. Such situations left their mark: judges took the reforms into their own hands and mainly tried to secure their minimally required budgets. This also had an impact on legislation: on 10 January 1996, in an effort to maintain decent living conditions for judges, the Federal Law “On Additional Guarantees of Social Protection of Judges and Court Personnel” was passed. Under this law, in addition to monetary salaries, judges received substantial additional benefits. This marked the first phase of the struggle for adequate financing for courts. Later, the 1999 Federal Law “On Court Financing in the Russian Federation” provided for the implementation of the constitutional provision on direct federal financing of courts. Several additional guarantees stipulated in this law included (a) financing of courts according to the

21 In this respect, the JD’s role and functions somewhat resemble those of the Administrative Office of federal courts in the United States, in terms of providing financing and technical/ material support for the courts it supports. 22 Alexeev S.S. Obschaya teoriya prava [General Theory of Law]. vol. 1. - М: Prospekt, 2008, p. 294. 23 Report of the Head of the Council of Judges Yuri Sidorenko on the work of the Council of Judges, accessible at: http://www.ssrf.ru/page/844/detail/


Page 87 of 148

standards authorized by the federal law and with the indication of each of the branches of judicial authority in the budget by a separate line, (b) interaction of the Russian Federation Government, in the course of development of the draft budget for the courts, with the presidents of the CC, SC, SAC and the Council of Judges, (c) the opportunity for court representatives to participate in the discussion on the federal budget in the Federal Assembly, and (4) a requirement to secure the consent of the All-Russia Congress of Judges or the Council of Judges to any reduction in the size of the approved annual budget for courts. The creation of the JD could be also attributed partly to judicial efforts to strive for adequate financing for courts. Overview of first decade of judicial reforms. Reviewing the first decade of judicial reforms, it was said that “the courts had gained key elements of independence (tenure and self-rule) and power (new jurisdiction) but lacked others (financial security; authority to secure implementation). At the same time, severe underfunding and the delays in procedural reform held back progress in making the courts fair, efficient, and accessible”24. 2000 – additional steps to strengthen the judiciary. In 2000 several policy declarations referred to the role of a strong judiciary for the development of the country. For example, a Presidential declaration in 2001 stated that an “Independent and impartial tribunal means that citizens are legally protected” and an independent judiciary was seen as “a fundamental prerequisite for the development of healthy and competitive economy. Finally, this is respect to the State itself, giving credit to the authority of law and justice”25. The 2002-2006 FTP “Development of Judicial System” further increased the judiciary budget, including higher salaries and bonuses for judges. 2001-2004 judicial reform measures. During 2001-2004 several significant steps were taken to improve the functioning of the judicial system and promote further judicial reforms. These included: (a) setting the hierarchy of normative legal acts and elimination of contradictions between normative acts of different levels (including those of federal and regional levels) by adopting the Decree “On Legal Acts’ Classifier”; (b) enactment of new procedural codes (Criminal Procedure, Civil Procedure and Arbitrazh Procedure), which recognized modern procedural rules ensuring equality among parties and corresponding to the principles of international law enshrined in the UN International Covenant on Civil and Political Rights and the European Convention for the Protection of Human Rights and Fundamental Freedoms; (c) elimination of the role of regional legislatures in judges’ appointment and promotion; and (d) a substantial increase in court financing including raising judges’ salaries (while decreasing the number of existing benefits), (e) increasing the number of JPs, thereby reducing the workload of district court judges, (f) expanding jury trials across Russia, (g) increasing the number of judicial staff, (h) renovating courthouses and courtrooms, and (i) first steps towards modern automation of the judicial system26. In 2001 a 24 Peter H. Solomon Jr., “Assessing the Courts in Russia: Parameters of Progress under Putin” in Demokratizatsiya, 2008 Heldref Publications. P. 66, accessible at: https://www2.gwu.edu/~ieresgwu/assets/docs/demokratizatsiya%20archive/GWASHU_DEMO_16_1/38876Q5X89546626/38876Q5X89546626.pdf 25 Borba s korruptsiiei i politicheskim vliyaniiem v sudebnoi sisteme [Fighting Corruption and Political Influence in Judicial System]. URL: http://ru-bespredel.com/index.php?newsid=1354 26 Some initiatives (e.g. drafting procedural codes) were developed during the end of the nineties; they could not be implemented mainly due to budget constraints.


Page 88 of 148

package of measures signed into law introduced disciplinary responsibility for judges, limited the broad immunity from prosecution earlier enjoyed by judges and limited the tenure for presidents of all courts (except the CC) to two consecutive six-year terms. The 2002 Federal Law “On the Bodies of Judicial Community” established new formations of qualification collegiums, adding public representatives and one representative of the President into their structures. 2008-2011 reforms. During 2008-2011 several major steps were taken to modernize the judicial system and procedures. For example: (a) the rules on public and mass media access to judicial information finally received their recognition in legislation; (b) to fight corruption in the judicial system the Law “On the Status of Judges” was amended to compel judges to recuse themselves in case of a conflict of interest, established a prohibition for judges to accept any remuneration with regard to exercising their authority which is not provided by Russian legislation (loans, monetary and other kinds of payments, services, financial coverage of entertainment, resort, transportation expenses) from individuals and legal entities, and mandated declaration of judges’ income and assets together with the income and assets of their spouses and underage children; (c) abolished the “probation period” for newly appointed federal judges; (d) to further improve disciplinary responsibility of judges, a Disciplinary Tribunal was established to hear appeals against decisions of the High Qualification Collegium and regional qualification collegia of judges on early termination of judicial powers for commission of disciplinary offences; (e) in accordance with the European Court of Human Rights jurisprudence the right to receive compensation for not completing a trial or proceeding within a reasonable time was enshrined in the law; (f) the system of continuous professional training of judges was improved by the introduction of special six-month training for newly appointed judges; (g) the selection system for judicial candidates was enhanced by separation of examination commissions from qualification collegia of judges and making them independent bodies; (h) full-fledged appeal procedures were introduced in CGJs which substantially decreased the timeframe for consideration of appeals; (i) a specialized Court for Intellectual Property Rights was established within the arbitrazh court system, (j) the Criminal Procedure and Civil Procedure Codes were amended to permit random case assignment to judges and the possibility of automated random allocation of cases to judges. In addition, two packages of amendments were made to the Criminal Procedure Code: in 2008 jury trials were eliminated for cases involving terrorism charges and anti-government activities and in 2009 the institution of pre-trial cooperation agreement was introduced into Russian criminal procedure, further simplifying it. Lastly, the election of the CC President (by his/her peer judges) was abolished and replaced by Presidential nomination and Federation Council approval. 2013-2014 - fundamental changes to the structure of the higher judiciary. The next stage of judicial reform was announced in 2013. It was decided to merge the SC and the SAC to “provide for the uniform approach in considering disputes with the participation of both individuals and legal entities as well as bodies of state power and local self-government”27. In December 2013 the State Duma (Lower House of the Russian Parliament) approved amendments to nine articles of the Constitution: the SAC was to be abolished and its powers transferred to a ‘renewed’ SC whose

27 President Putin’s presentation at the Plenary Session of St. Petersburg International Economic Forum. URL: http://kremlin.ru/events/president/news/18383 (Last accessed 08.01.2018)


Page 89 of 148

members were to be freshly appointed and a Judicial Collegium for Economic Disputes also established. This Law on Constitutional Amendments became effective on February 6, 2014 after approval by more than two-thirds of Russian regions. The Law provided for a six-month transition period during which the new qualification collegium of judges (established specially for this purpose) selected judicial candidates for the new court and set new, more rigid standards of judicial selection. The new consolidated SC became fully operational on August 6, 2014. In addition, the Constitutional amendment also abolished the Disciplinary Tribunal and established a new Disciplinary Collegium within the ‘renewed’ SC. 2013 and after – other measures. Several initiatives to further modernize the judicial system were implemented. For example: (1) the 2013-2020 FTP “Development of the Judicial System” provided for further increase in financing of the judiciary; (2) the system of judges’ remuneration was modernized; (3) to improve resolution of commercial disputes and increase the authority of civil and commercial arbitration, a new Law on Arbitration (Arbitration Procedure) was adopted in 2014; (4) a highly anticipated Individual Bankruptcy Law finally became effective introducing not only new procedures for such disputes but dramatically changing arbitrazh courts’ jurisdiction – allowing them to consider disputes involving natural persons (i.e. individuals) in addition to individual entrepreneurs and enterprises; (5) in order to distribute caseloads more evenly among JPs, new powers to redistribute cases between judicial sub-district were given to presidents of district courts, and appointment of retired judges as acting JPs was introduced; (6) as a new guarantee of judicial independence, judges were obliged to publish any communications from legislative or executive officials with regard to cases in their dockets; (7) a new Administrative Procedure Code was adopted to regulate judicial control over the legality and reasonableness of exercising state and other public authority; (8) reform of civil legislation including changes to the law of obligations (performance, security and termination of obligations, and liability for breach of obligations) and general contractual provisions; (9) jury trials also became available in district and garrison military courts so many more litigants could now exercise their right to be tried by their peers; (10) new procedures were introduced, such as summary (writ) proceedings and mandatory use of extrajudicial means of settlement before applying to an arbitrazh court – these aimed to exempt arbitrazh courts from lengthy consideration of undisputed debts where the claim was not more than RUR500,000. The SC took initiatives to improve the humanization of criminal law and procedure. For example, in July 2016 several offences previously included in the Criminal Code were decriminalized, for example simple battery and non-payment of alimony. The SC proposes to introduce a new concept of a “misdemeanor” for those crimes now characterized by the Criminal Code as “crimes of minor gravity”. Such crimes are intended to be punished by suspended sentences and a person who committed a misdemeanor for the first time would not have it entered into his/her criminal record28. The SC also announced an initiative to create interregional appellate and cassation courts in the CGJ system, analogous to the arbitrazh court system, proposing to establish five interregional

28 Yuri Belov. Vyacheslav Lebedev gotovit “ugolovny prostupok”. Predsedatel’ Verkhovnogo Suda rasskazal o gumanizatsii ugolovnogo zakonodatel’stva [VyacheslavLebedev is preparing a “misdemeanor”. President of the Supreme Court Tells About the Humanization of Criminal Legislation]. Kommersant, 21.09.2016. URL: https://www.kommersant.ru/doc/3094716


Page 90 of 148

appellate courts to review cases considered by regional level courts as trial courts and nine cassation courts to perform cassation review of those cases instead of the SC. According to the SC this reform, requiring 170 additional judges’ positions for appellate courts and 790 for cassation courts, would cost RUR831 million.29 Automation of court procedures and functions. Major actions were taken to increase court automation. From January 1, 2017 the justice function became more accessible to the public due to the coming into effect of the 2016 Federal Law "On Introduction of Amendments into Certain Legislative Acts of the Russian Federation in Terms of Using Electronic Documents in the Activities of the Judiciary": civil claims, applications, complaints and petitions within civil, arbitrazh, administrative and criminal procedure could be submitted electronically through personal accounts at court websites or through the Public Services Portal. It is also possible to receive judicial decisions electronically. All documents shall be signed by the electronic digital signature but it is not mandatory to receive it as it would be possible to sign electronic documents through the Unified Identification and Verification System which gives access to the Public Services Portal. The CC also provides an advanced search option on its website, allowing users to search court resolutions and other acts by case number, case participant surname, or text of the resolution. Related documents including relevant ECHR decisions are attached to the search results. The video-conference system implemented in CGJs and arbitrazh courts allows the conduct of trials involving participants from remote locations or defendants in detention. In 2017 the new Law unified standards for live broadcasting of court hearings on TV and radio, live streaming on the Internet and Internet publication of judicial decisions. According to this Law, allowing live streaming of court hearings can be decided by the court itself. Upon such a decision, the live stream becomes accessible to all Internet users. The rapid development of artificial intelligence has led to discussions about making judicial proceedings more efficient, for example further automation of legal drafting and enforcement of law, generation of routine decisions using artificial intelligence (AI) and creation of an automated monitoring system for courts’ jurisprudence. Creation of an automated system for legal decision support is envisaged to include services for automatic generation of routine legal documents, and an electronic risk management system to analyze judicial decisions for errors and corruption.30

29 Anastassiya Kornya. Verkhovny Sud odobril sozdaniye sistemy apellyatsionnykh i kassatsionnykh sudov [Supreme Court Approves the Establishment of Appeal and Cassation Courts’ System], Vedomosti, 14.07.2017. URL: https://www.vedomosti.ru/politics/articles/2017/07/14/723566-apellyatsionnih-kassatsionnih-sudov 30 Nadezhda Krasnushkina, Andrey Raisky, Yevgeniya Kruchkova, Denis Skorobogat’ko. Sudebny protsessor. Pravitel’stvo obsuzhdaet ispol’zovaniye iskusstvennogo intellekta v pravovoi sfere [Judicial Processor. The Government is Discussing the Possibility of Using Artificial Intelligence in the Legal Sphere]. Kommersant, 13.11.2017. URL: https://www.kommersant.ru/doc/3466185?ref=tjournal.ru


Page 91 of 148

ANNEX 7. JRSP SURVEYS – SUMMARY OF KEY RESULTS

Five rounds of user surveys were conducted under Component A of the JRSP (2010, 2011, 2012, 2013 and 2017). The surveys covered the general public (i.e. individuals), businesses, professional users (e.g. lawyers, bailiffs and notaries) and court personnel. Key extracts from these surveys are presented in this Annex. The results of the first round of surveys were published by the BEA (the PIU at the time) and by the Levada Center (which was awarded the contract for the first four rounds of the surveys). In addition, two surveys were conducted for the MCC in 2017: they are summarized in a separate Annex.

Part A. Key Extracts From Survey Responses (2010-2017)

The key messages from the surveys are: Public trust in courts is gradually rising (Figure 7.1); Public trust in courts is lower than other institutions such as the church and armed forces; Enterprises feel that private property rights protection is improving (Figure 7.2); TV news and TV court programs, newspapers and magazines remain key information sources

for citizens about the judiciary - online sources are still insignificant; Information channels such as online sources and publications remain less significant for

citizens as a source of information; and For enterprises, the main sources of information about the judiciary are websites of courts and

online legal databases. Below are year-wise responses to key survey questions. Question: Can an ordinary Russian citizen have his/her case considered fairly and without prejudice by the court? (General public) 2006 2007 2008 2009 2010 2012 2013 2017 Trust coefficient

0.46 0.55 0.56 0.46 0.64 0.73 0.51 0.76

30

40

50

60

70

80

2009 2010 2012 2013 2017

Figure 7.1 Can An Ordinary Russian Citizen Have His/Her Case Considered Fairly & Without Prejudice By The Court? (General

Public: % Responding "Yes")


Page 92 of 148

Question: What is your main source of information about the judicial system and courts’ activities? (General public)

2010 2011 2012 2013 2017 TV news, information channels 63 62 61 60 57 TV mock trials like “The Court is in Session”, “The Court’s Hour”, “Judicial Passion”

57 56 50 44 31

Newspapers and magazines 32 32 29 31 22 Films and TV series 22 24 24 25 22 Relatives or acquaintances having experience in litigation

25 23 27 20 23

Radio channels 7 10 7 9 7 Internet sources 4 5 10 8 15 Specialized web sites 7 5 10 7 8 Belles-lettres, fiction 5 3 4 4 3 Personal experience in litigation 10 7 8 5 9 Specialized scientific literature, magazines 5 3 4 4 3 Courts’ web sites - 3 - 4 4 Educational TV channels, text books 5 4 5 3 4 Popular lectures, lawyers’ interviews 2 2 3 3 2 Specialized legal databases 2 1 3 2 3 Professional activity 2 3 2 2 3 Not interested in such information 7 9 11 11 13

Question: Do you trust the following institutions? (General public)

Do trust Do not trust Trust resource (difference

between “do trust” and “do

not trust”) as of 2017

Second wave 2011

Fourth wave 2013

Fifth wave 2017

Second wave 2011

Fourth wave 2013

Fifth wave 2017

President 74 63 81 23 33 16 65 Church 69 63 59 24 29 31 28 Army (armed forces) 55 62 75 38 34 19 56 Security Service (FSB)

58 61 66 29 32 23 43

Government 63 51 57 33 44 39 17 Prosecution Service 47 51 55 44 44 38 17 Presidential Administration

54 47 59 36 44 31 28

Human Rights 48 46 50 39 44 35 16


Page 93 of 148

Organizations Judicial system 41 44 49 50 50 43 6 Federation Council 45 41 49 40 47 38 11 State Duma 42 39 44 52 56 50 -6 Police 31 38 48 63 57 45 2 Trade unions 36 46 40 48 52 42 -2 Political parties 28 26 31 63 68 60 -29

Question: How high is the public standing of the following institutions in the community? (General public)

2011 2013 2017 2011 2013 2017 2011 2013 2017 Very high + Rather

high Very low + Rather low Hesitate to answer

Church 72 63 61 19 30 29 9 7 10 Government 65 49 62 29 46 32 7 5 6 Special services (FSB etc.)

63 65 69 21 26 19 16 9 13

Army 46 55 73 47 41 20 7 5 6 State Duma 45 38 50 48 57 42 7 5 8 Businesses 43 44 52 47 46 36 11 9 13 Courts 41 44 54 49 50 37 9 6 9 Human Rights Organizations

41 40 43 43 48 38 16 12 19

Public Chamber

37 35 40 36 49 37 27 16 22

Law enforcement

34 38 53 58 56 37 8 6 10

Question: Public Standing Index Second wave (2011) Fourth wave (2013) Fifth wave (2017) Church 53 33 32 Government 36 3 30 Special forces (FSB etc.)

42 39 50

Army -1 14 53 State Duma -3 -19 8 Businesses -4 -2 16 Courts -8 -6 17 Human Rights Organizations

-2 -8 5


Page 94 of 148

Public Chamber 1 -14 3 Law enforcement -24 -18 16

Question: Do you trust the following courts? (General public) Fully trust +

Rather trust Fully distrust + Rather distrust

Trust index Do not know, hesitate to answer

Waves 1-4,

average

Wave 5

(2017)

Waves 1– 4,

average

Wave 5 (2017)

Waves 1-4,

average

Wave 5

(2017)

Waves 1-4,

average

Wave 5

(2017)

Supreme Court

59 64 22 21 2.7 3.0 19 15

Constitutional Court

55 61 23 22 2.4 2.8

Justices of the Peace

52 55 30 30 1.7 1.9 18 15

Supreme Arbitrazh Court

49 57 23 22 2.1 2.6 28 21

Jury trials 48 52 33 30 1.4 1.7 19 18 Federal courts of general jurisdiction

44 48 39 35 1.1 1.4 17 17

Constitutional (charter) courts

43 52 26 25 1.6 2.1 31 24

Arbitrazh courts

42 47 28 30 1.7 1.6 30 24

Question: Do you think today’s judiciary is really independent? (General public)

2010 2011 2012 2013 2017 Yes, to a great extent 7 7 7 6 6 Yes, to a certain extent 27 23 27 26 29 No, to a small extent 39 31 42 49 31 No, completely dependent 17 27 15 24 20 Hesitate to answer 10 11 8 10 15

Question: Which of the following do you trust most? (General public) Average 2010 2011 2012 2013 2017


Page 95 of 148

Judges Fully trust 6 7 6 6 6 8 Rather trust 45 47 44 46 44 48 Rather distrust 34 31 34 35 35 39 Fully distrust 8 8 8 7 9 8 Hesitate to answer 7 7 8 7 7 7

Advocates Fully trust 8 9 10 6 9 9 Rather trust 52 53 52 55 49 53 Rather distrust 25 24 24 25 29 23 Fully distrust 7 7 6 7 7 7 Hesitate to answer 7 7 8 7 6 8

Prosecutors Fully trust 6 5 7 6 7 8 Rather trust 42 43 41 42 43 45 Rather distrust 34 31 34 35 34 30 Fully distrust 9 9 8 8 9 8 Hesitate to answer 9 11 10 10 6 9

Investigators Fully trust 5 4 4 5 5 6 Rather trust 36 39 35 36 35 42 Rather distrust 39 37 40 40 40 34 Fully distrust 11 11 11 11 12 9 Hesitate to answer 9 10 9 9 8 9

Notaries public Fully trust 13 15 12 12 12 15 Rather trust 55 56 54 58 53 55 Rather distrust 20 18 21 19 23 17 Fully distrust 5 5 5 4 7 5 Hesitate to answer 7 6 7 7 5 8

In-house lawyers Fully trust 6 6 6 7 7 8 Rather trust 45 44 43 47 45 47 Rather distrust 28 27 29 25 30 24 Fully distrust 9 10 9 7 9 7 Hesitate to answer 13 13 14 14 10 14

Human rights defenders Fully trust 8 7 7 9 9 9 Rather trust 47 49 45 50 44 47 Rather distrust 25 23 26 22 28 24 Fully distrust 8 8 8 7 8 7 Hesitate to answer 13 14 14 13 11 14

Policemen


Page 96 of 148

Fully trust 3 2 3 3 5 7 Rather trust 27 27 27 29 29 39 Rather distrust 42 41 43 40 44 33 Fully distrust 21 24 23 20 16 12 Hesitate to answer 7 6 7 8 6 9

Trust index (justice sector) 2010 2011 2012 2013 2017 Notaries public 3.1 2.5 3.0 2.2 3.2 Advocates 2.0 2.1 1.9 1.6 2.1 Human rights defenders 1.8 1.9 2.0 1.5 1.8 In-house lawyers 1.3 1.3 1.7 1.3 1.8 Judges 1.4 1.2 1.2 1.1 1.5 Prosecutors 1.2 1.1 1.1 1.1 1.4 Investigators 0.9 0.8 0.8 0.8 1.1 Policemen 0.5 0.4 0.5 0.6 1.0

Question: How would you assess the following? (General public) Average 2010 2011 2012 2013 2017

Public standing of Russian judges Very high 6 6 6 6 5 9 Rather high 38 42 34 36 39 42 Rather low 39 37 40 40 41 29 Very low 7 7 8 8 6 6 Hesitate to answer 10 8 13 10 9 13 Ratio of opposite opinions 0.9 1.0 0.9 0.9 0.9 1.4

Qualification and expertise of Russian judges Very high 5 6 4 6 5 8 Rather high 44 48 43 43 43 46 Rather low 30 26 29 31 32 24 Very low 5 4 6 4 5 5 Hesitate to answer 16 17 18 16 15 18 Ratio of opposite opinions 1.4 1.9 1.3 1.4 1.3 1.9

Moral qualities of Russian judges Very high 4 5 3 4 4 5 Rather high 29 31 28 27 30 33 Rather low 38 35 37 40 41 31 Very low 9 9 9 9 8 9 Hesitate to answer 20 21 22 21 18 21 Ratio of opposite opinions 0.7 0.8 0.7 0.6 0.7 0.9

Independence of Russian judges


Page 97 of 148

Very high 4 4 4 4 4 5 Rather high 23 24 22 22 26 28 Rather low 45 44 45 45 45 37 Very low 13 13 13 16 12 12 Hesitate to answer 14 14 16 13 13 18 Ratio of opposite opinions 0.5 0.5 0.4 0.4 0.5 0.7

Human qualities of Russian judges Very high 3 3 3 4 3 5 Rather high 27 31 25 26 27 30 Rather low 40 37 39 39 43 33 Very low 10 9 11 10 10 12 Hesitate to answer 20 19 21 21 17 21 Ratio of opposite opinions 0.6 0.7 0.6 0.6 0.6 0.8

Attentiveness of Russian judges Very high 4 4 3 4 3 5 Rather high 25 29 23 23 24 27 Rather low 43 39 43 43 46 37 Very low 12 12 13 13 12 13 Hesitate to answer 17 16 18 17 15 18 Ratio of opposite opinions 0.5 0.6 0.5 0.5 0.5 0.6

Fairness of Russian judges Very high 4 4 3 4 3 5 Rather high 25 28 23 23 27 30 Rather low 42 39 42 45 42 33 Very low 12 13 13 12 12 12 Hesitate to answer 17 17 19 16 16 19 Ratio of opposite opinions 0.5 0.6 0.5 0.5 0.6 0.8

Russian judges’ devotion to duty Very high 5 - 4 6 5 7 Rather high 30 - 28 29 33 34 Rather low 36 - 35 36 37 29 Very low 10 - 10 9 10 10 Hesitate to answer 19 - 22 20 15 19 Ratio of opposite opinions 0.8 - 0.7 0.8 0.8 1.0

Courage of Russian judges Very high 5 - 4 - 5 7 Rather high 28 - 26 - 30 32 Rather low 34 - 33 - 36 27 Very low 12 - 13 - 12 10 Hesitate to answer 21 - 24 - 18 24 Ratio of opposite opinions 0.7 - 0.7 - 0.7 1.1

Judges’ qualities index


Page 98 of 148

Average 2010 2011 2012 2013 2017 Qualification and expertise 1.4 1.9 1.3 1.4 1.3 1.9 Public standing 0.9 1.0 0.9 0.9 0.9 1.4 Devotion to duty 0.8 - 0.7 0.8 0.8 1.0 Moral qualities 0.7 0.8 0.7 0.6 0.7 0.9 Courage 0.7 - 0.7 - 0.7 1.1 Human qualities 0.6 0.7 0.6 0.6 0.6 0.8 Fairness 0.5 0.6 0.5 0.5 0.6 0.8 Attentiveness 0.5 0.6 0.5 0.5 0.5 0.6 Independence 0.5 0.5 0.4 0.4 0.5 0.7

Question: What issues in judges’ work are the most acute from your point of view? (General public) 2010 2011 2012 2013 2017 Bribery, corruption of judges and court personnel 56 59 55 48 45 Judges’ dependence on the authorities and influential groups

46 45 50 40 39

Absence of equality of parties in court proceedings: for example public officials have priorities over ordinary citizens

25 27 30 25 27

Red tape, delays, unproductivity 29 26 26 25 20 Low public standing of judges 17 24 17 25 14

Question: Are you ready to sue a …? (Index) 2010 2011 2012 2013 2017 Judge 0.42 0.55 0.42 0.46 0.8 Investigator 0.54 0.76 0.54 0.59 1.0 Public official 0.59 0.78 0.58 0.65 1.0 Policeman 0.72 0.94 0.62 0.9 1.1 Public employer 1.2 1.6 1.2 1.4 1.8 Private employer 1.3 1.7 1.4 1.4 2.2

Question: When your rights were violated did you apply to…? (General public) Second wave (2011) Fourth wave (2013) Fifth wave (2017) State institutions 14 16 8 Court 15 10 21 Prosecution service, law enforcement agencies

8 6 13

Acquaintances that can 5 5 5


Page 99 of 148

settle the case Executive bodies of any level

10 4 7

Police 7 3 15 Influential public officials 2 2 2 Your member of the parliament

4 1 1

Mass media 2 1 2 Public and human rights organizations

5 0 6

Arbitration tribunals 0 0 1 Criminal groups 0 0 0 Other 1 2 5 Did not apply anywhere 42 57 37

Question: Are your private property rights protected? (Entrepreneurs) Second wave

(2011) 812 respondents

Third wave (2012) 823 respondents

Fourth wave (2013) 802 respondents

Fifth wave (2017) 800 respondents

Yes/rather yes 43 48 52 51 No/rather no 54 50 45 44 Hesitate to answer

3 2 4 6

Figure 7.2 Enterprises’ Response – Improved Protection of Property Rights


Page 100 of 148

Question: What are your sources of information on the arbitrazh courts’ activity? (Entrepreneurs) 2010 2011 2012 2013 2017 Legal databases (“Garant”, Consultant+”) 67 70 74 58 54 Supreme Court, Constitutional Court web sites 38 47 50 58 21 First instance arbitrazh courts’ web sites 32 34 36 43 49 Information exchange with the colleagues 29 29 25 22 25 Professional magazines 28 24 22 17 4 Newspaper articles (“Vedomosti”, Kommersant”) 20 17 13 14 25 Specialized literature 16 11 11 10 15 Other web sites, blogs 10 9 12 9 1 TV programs 13 10 7 6 14

Question. How important are the following systems for you?

General

Population Court

Personnel Professional

Users External Internet portal of MCC and Moscow district courts

62 64 59

Data exchange system between the courts of different levels

57 64 50

Accessibility of all subsystems of the Integrated Information System of Moscow CGJ

57 64 48

Possibility of conducting videoconferences with detention centers and prisons.

55 56 55

Unified archive of judicial documents and case files with bar codes application.

54 59 48

Unified storage of audio and video records of court sessions; installation of audio and video recording equipment in courtrooms.

54 60 47

Information and analytical system – unified database of MCC and Moscow district court cases

52 57 45

Internal Internet-portal of MCC and Moscow district courts

50 57 42

Automation of case files’ archiving function before their destruction after the expiration of their storage time or transfer to the Central Archive for life storage

48 51 45

Access to video records of court sessions to external users including live broadcasting and streaming.

46 43 49


Page 101 of 148

Part B. 2017 Survey of General Public

Constitutional state. The main criteria of the rule of law for the population are equality before the law (57 percent), law compliance (55 percent), respect for human rights and fundamental freedoms (47 percent). However, 39 percent of Russians believe that they are not under the protection of the law. They often think that laws are freely interpreted by authorities and there are too many people feel that they are above the law. Subjective assessments of knowledge of own constitutional rights are at a low level: only 3 percent of Russians claim that they are very well aware of their constitutional rights, another 20 percent say they know their rights well. 41 percent of respondents believe that judicial proceedings will be considered by the court fairly and impartially. 60 percent of Russians know about the European Court of Human Rights in Strasbourg. However, only a third of them is ready to apply to the court to protect their interests. Overall, 64 percent of Russians have a positive attitude towards a jury trial, but only 36 percent of respondents are ready to be on the jury.

Awareness about the Russian judicial system. Russians assess the level of accessibility of information on the activities of courts at a low level. Only 4 percent of respondents note that information is fully available, another 20 percent say it is more likely to be available. Only 2 percent of Russians know very well how the judicial system in Russia operates. Another 13 percent say they know it well. However, 50 percent of the population have only a general idea of the judicial system and every third (33 percent) knows nothing about it. A low level of awareness about Russian judicial system is associated with a low level of interest and willingness of the population to understand how it is arranged. Only 22 percent of respondents claim that they are interested in information about the judicial system operation in Russia. The main source of information about the judicial system in Russia is news on TV (57 percent). On the second place is themed programs and talk shows (31 percent).

Independence of the judiciary. Half of Russians believe that the judiciary is currently dependent on other branches of government (51 percent). 35 percent of respondents say that the judicial system in Russia is independent. It should be noted that a third of Russians consider that courts must be dependent on other branches of government. According to citizens, the strongest pressure on judges is provided by federal and higher judicial authorities. The most urgent problems in the judicial system operation are: 45 percent of Russians pay attention to bribery, corruption of judges and court personnel, 39 percent on dependence of judges on authorities and influential groups and 27 percent on the violation of the principle of equality of parties in judicial proceedings. Key measures that could raise the authority of courts in Russian society are: increasing the competence of judges and the quality of their education, raising the public awareness about the work of judges, a possibility for people to choose or recall judges.

Situations related to violation of rights. Over the last 12 months, 10 percent of Russians were faced with a conflict situation related to the violation of rights or the need to protect them. Most often, these conflict situations arose in the housing, labor and family relations. 61 percent of those who faced a conflict situation sought a legal advice. 44 percent of them applied to the law office, 24 percent - to familiar lawyers, 21 percent - to law firms, 12 percent - to people who has already faced similar problems, 9 percent - in human rights organizations. 15 percent of Russians have faced conflict situations that violate or prejudice their rights (except for criminal offenses) and


Page 102 of 148

these conflicts could be resolved in a court. Most often, these conflict situations arose in the housing law, violations of property rights, family and consumer law. 58 percent of Russians who faced such conflict situations applied to a court at least once. 42 percent did not apply to a court in such situation, most often due to lack of time and energy, the bureaucracy and high financial costs for judicial procedure (18 percent).

Actual experience with judicial proceedings. 24 percent of Russians have attended court proceedings. Most often, respondents were present at court in civil proceedings (63 percent), in criminal proceedings (36 percent) and in cases of administrative offenses (21 percent). 47 percent of Russians independently represented their interests at a trial. 25 percent engaged a lawyer. In 17 percent of cases there was a free lawyer appointed by a court. Key reasons for refusal of Russians to engage lawyers: lack of money to hire a private lawyer and a low level of confidence in free legal aid or assistance from the state. 55 percent of Russians believe that they have been explained their rights in court. However, every third participant of the survey indicated that the rights were explained formally and fast (31 percent). Half the participants in the judicial proceedings said that they were well or generally familiar with the code of practice. 68 percent of respondents were satisfied with the work of court officers and noted that court officers did everything that depended on them and all they were obliged to do. 70 percent of Russians found the judge's decision fair and 26 percent found the decision unfair. 43 percent of respondents incurred significant financial costs for family budget due to judicial procedure. 33 percent of Russians said they would appeal to a court to protect their rights in the future. However, 52 percent of respondents said they would not appeal to a court at all or would attend only in extreme cases.

Corruption in the Russia judicial system. Only 7 percent of respondents believe that court officers are highly corrupt. 50 percent consider that in federal courts of general jurisdiction bribes are often taken. The main cause of the prevalence of corruption in the federal courts of general jurisdiction is self- interest of court officers (41 percent). Russians tend to clarify the representation of corruption in courts by the prevailing view in the society that all authorities in Russia take bribes (38 percent) and by the information from the media about cases of corruption in the courts (35 percent). Even though almost half of Russians support an opinion on corruption, 56 percent of respondents will not try to solve their problems in court giving a bribe. 7 percent of respondents are aware of cases about corruption in courts personally or through relatives and friends. Moreover, many representatives of this group (84 percent) answered that the case was related to a bribe. A bribe was accepted in a half of the cases described, in a quarter - a bribe was not accepted. Two-thirds of respondents who had relevant experience tried to bribe to achieve a desired decision and a quarter to speed up a trial process. A third of those who bribed reported about the full achievement of a goal, 53 percent - about a partial achievement. On measures that could reduce corruption in Russian courts, most respondents were inclined to tougher penalties for court officers (46 percent). One third of respondents considers that strengthening of control by the executive, legislative authorities will be effective (32 percent), and from higher courts (27 percent).

Part C. 2017 Survey of Businesses

Constitutional state. 47 percent of entrepreneurs consider that their property rights are protected. However, 46 percent of respondents believe their rights are unprotected. 52 percent of


Page 103 of 148

entrepreneurs are afraid of the deprivation of property (raiding). There is an atmosphere of partial trust and partnership. 78 percent of respondents trust business colleagues, although only 17 percent are absolutely sure of their reliability. Every fourth businessman is ready to work legally even with restrictions on business, other 45 percent entrepreneurs are inclined not to break the law. 23 percent of respondent has an opposite opinion, even tenth evades a direct answer. Participants explain that citizens and organizations apply to foreign courts due to their independence (41 percent), the absence of corruption (25 percent) and the use of better legal frameworks abroad (25 percent).

Awareness about the Russian judicial system. 37 percent of business representatives are interested in the practice of arbitration courts. The main sources of information for business are such systems as «Garant» and « Consultant Plus» - 54 percent, official websites of the arbitration court - 49 percent. Entrepreneurs are looking for a database of court decisions and appeals (63 percent), also comments and explanations of law enforcement practice (57 percent). 23 percent of entrepreneurs encounter difficulties in obtaining information about a court practice of arbitration courts. It is caused by the lack of systematic information (48 percent), closed information (35 percent), poor quality and fragmentariness of information (29 percent). 65 percent of entrepreneurs support open access to texts of various documents of arbitration courts. 45 percent of respondents say about possible negative consequences of the publication of documents of arbitration courts. About the same number of participants in the survey do not expect negative consequences for enterprises in the case of the publication of such materials (41 percent). 16 percent of respondents know cases when the openness of information about the decision on their enterprise case led to undesirable consequences. As a compromise, many entrepreneurs (60 percent) believe that it is necessary to give an open access to the texts of various documents of arbitration courts without specifying participants. 26 percent indicate the need to specify the all information. 49 percent of entrepreneurs consider that the media provide materials about a litigation in arbitration courts objectively, but 34 percent - not objectively. 16 percent find it difficult to assess the work of the media. Entrepreneurs pay attention to the fact that the media partiality is caused by the desire to submit the «hot» material (45 percent), «invited» articles (36 percent) and the desire to emphasize the correctness of state authorities that participated in the trial (32 percent).

Independence of the judiciary. 75 percent of entrepreneurs believe that courts in Russia must be independent of the country's leadership. 12 percent of participants have the opposite opinion. 15 percent of entrepreneurs believe that in Russia there are no independent judges. 23 percent consider that many judges under any conditions are independent. 16 percent say that a half of the judges are independent. 17 percent believe that such judges are a minority. The most frequent subject of pressure on judges of an arbitrazh court are stated to be influential persons (38 percent), a chairman or deputy chairman of an arbitrazh court (35 percent) and federal authorities (31 percent).

Actual experience with judicial proceedings. 46 percent of enterprises do not have their own legal service or department, or even one staff unit for the relevant specialist. 29 percent of participants had to act in 2000-2016 as a plaintiff or defendant in litigation in an arbitration court. Large businesses participated in court proceedings more often (70 percent), 40 percent in medium enterprises, 27 percent in small businesses, and 20 percent in microenterprises. Many enterprises had legal proceedings with another enterprise, organization, bank (63 percent). Among those who took part in court proceedings their interests were most often represented by an internal lawyer (63 percent), every fifth respondents noted that the interests were represented by an invited lawyer (21


Page 104 of 148

percent). 64 percent of respondents named contractual delinquency as the main reason for the arbitrazh court case. 26 percent of entrepreneurs know about the possibility of attracting arbitration assessors who have the same powers as a federal judge. The participation of an arbitration assessor had a positive effect on the case in court for 46 percent of entrepreneurs. Almost a quarter of respondents noted the absence of any influence (23 percent). 66 percent of respondents are satisfied with the way the court proceedings were conducted. 80 percent were satisfied with how the court dealt with the enterprise case. Entrepreneurs complain that the trial was often postponed for various reasons (18 percent), and that the trial did not start for a long time (13 percent). 18 percent say that the judicial act that entered into legal force was not executed at all. 11 percent of enterprises incurred significant financial costs for budget, 53 percent - insignificant, 34 percent - did not affect at all. 20 percent of respondents will appeal to a court in all situations in the future. 66 percent of the entrepreneurs go to a court if there is no other choice. Only 12 percent of the respondents will try to resolve a conflict situation in a different way.

Corruption in the Russia judicial system. 60 percent of respondents are not satisfied with ways which are used now to fight corruption. Many entrepreneurs note that bribery hinders fair and impartial consideration of cases in a court, and therefore it is necessary to fight corruption (70 percent). 21 percent of respondents believe that corruption is present in arbitration courts to a small extent, 13 percent - to a large extent, 7 percent - note a very high level of bribery. The main reason for the spread of corruption in arbitration courts is the fact that courts are the part of a government system whose agencies are characterized by a high prevalence of bribery (40 percent). Approximately the same number of respondents pay attention to the impunity of court officers (39 percent). 35 percent respondents consider the hidden nature of bribe taking and the hard-to-prove nature of this fact as the main reason for bribery in arbitration courts. 15 percent of entrepreneurs are aware of the facts of bribery in the arbitration court of their region. Among those who said that there are many such facts, a large proportion is in Moscow and Saint Petersburg (16 percent). 30 percent of respondents consider that organizations rarely have to bribe in order to speed up the trial or take the necessary decision in an arbitration court. 16 percent - often give such payments. 23 percent of respondents did not encounter such situations at all.


Page 105 of 148

ANNEX 8. 2017 MOSCOW CITY COURTS SURVEYS – SUMMARY

1. Two rounds of surveys were conducted in 2017 for the MCC: a baseline survey in February 2017 and a final survey in December 2017, before project closing. The surveys covered litigants (i.e. court users), professional court users (e.g. lawyers, prosecutors, bailiffs and notaries) and court personnel (i.e. judges and court staff). The MCC published the survey report on its website (http://mos-gorsud.ru/mgs/info/orders). This Annex summarizes key findings from these surveys. The MCC plans to conduct the survey annually to track progress.

2. The final survey had higher positive responses to questions on changes to the court system (question 1) and there were higher positive responses about the changes in the MCC (question 2). The increased positive responses about the MCC may be attributed to the JRSP-financed next-generation MCC ICT system which went live from January 1, 2017: its functioning rapidly improved during 2017 as interagency coordination issues and technical issues were addressed. Question 1. From your perspective, how has Russia’s court system changed in the last 1-2 years?

Question 2. From your point of view, how has the MCC system changed in the past year?

29

25

44

32

32

26

13

18

34

37

37

46

38

42

28

23

8

9

5

4

8

10

13

13

15

12

6

8

18

13

21

15

13

17

9

11

4

9

25

32

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Baseline survey

Changed positively

Some changes are positive, some are negative

Changed negatively

Did not change at all

Hesitate to answer

All respondents

Court personnel

Professional users

General public


Page 106 of 148

3. On questions about the effectiveness of the Russian and Moscow court systems (questions 3 and 4), responses were more positive about the MCC, largely attributable to the implementation of the JRSP-financed MCC ICT system. Within the MCC responses, court personnel – who worked most closely with the new MCC ICT system – had significantly higher responses at the final survey compared to the baseline survey.

Question 3. How do you assess the effectiveness of Russia’s court system?

28

26

46

38

28

25

12

15

33

36

32

37

36

46

32

25

6

5

1

1

9

5

6

9

17

13

8

8

19

13

22

16

17

20

14

15

8

11

29

34

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Baseline survey

Changed positively

Some changes are positive, some are negative

Changed negatively

Did not change at all

Hesitate to answer

All respondents

Court personnel

Professional users

General public


Page 107 of 148

9

7

15

11

10

8

2

4

31

29

45

37

32

31

19

19

41

41

34

45

43

45

44

34

10

8

1

3

10

7

19

14

5

6

1

3

4

11

11

4

9

5

3

2

5

4

19

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Baseline survey

Very effective

Rather effective

Sometimes effective, sometimes not

Rather ineffective

Absolutely ineffective

Hesitate to answer

All respondents

Court personnel

Professional users

General public


Page 108 of 148

Question 4. How do you assess the effectiveness of the MCC system?

4. When asked to identify the top three problematic areas in court functioning, different respondent groups had different responses (question 5):

a) Court personnel cited salary levels in the judiciary, material and technical support (i.e. operations and maintenance) and interagency interaction;

b) Professional users cited organization of workflow and document flow, salary levels in the judiciary and judicial independence;

c) The general public’s top three concerns were organization of workflow and document flow, judicial independence, and organization of courts’ activities in general and at different stages of proceedings; and

d) For all respondents, the top three problematic areas were salary levels in the judiciary, organization of workflow and document flow and material and technical support.

Question 5. Name the three most problematic areas in the functioning of Russia’s court system

12

9

23

14

10

10

3

3

32

30

49

41

29

31

20

19

37

38

22

33

47

47

42

34

9

7

1

3

8

4

17

14

3

4

0

1

2

8

9

7

12

5

9

4

6

11

20

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Baeline survey

Very effective

Rather effective

Sometimes effective, sometimes not

Rather ineffective

Absolutely ineffective

Hesitate to answer

All respondents

Court personnel

Professional users

General public


Page 109 of 148

All Respondents Court Personnel Professional Users General Public

Baseline survey

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Baseline survey

Final survey

Salary level in the judiciary 30 33 64 72 20 22 7 7 Organization of workflow and document flow 36 31 26 24 32 32 48 38

Material and technical support (e.g. printers, paper) 15 22 32 46 5 13 7 8

Interagency interaction (i.e. with other government agencies)

23 20 32 29 24 18 13 14

Independence of the judiciary 11 20 4 7 14 22 16 28

Informing participants of court cases 20 18 11 15 16 21 33 16

Organization of courts’ activities in general and at different stages of proceedings

21 15 10 5 18 15 33 24

Building maintenance, office space availability (e.g. courtrooms, judges’ chambers), electricity supply, modern equipment, furniture etc.

9 14 14 21 8 15 6 8

Infringement of parties’ rights during court proceedings 7 14 0 4 10 14 10 22

Direct access to justice for the public (e.g. difficulties with filing an application/claim and its further consideration)

19 11 14 6 17 9 25 17

Interaction between courts of different levels. 16 10 19 5 22 11 7 14

Legislative framework 7 10 4 4 6 16 11 11 Acquainting parties to the case with the case files 10 8 5 1 12 11 12 12

Transparency and openness of the court system 7 6 1 4 9 7 11 9

Security issues 2 4 1 6 5 4 2 2 Hesitate to answer 9 9 9 6 18 12 1 10

5. Respondents believe the most urgent problems of the organization of the work of


Page 110 of 148

courts are the length of time taken for case examination (58 points), workflow in legal proceedings (56 points), familiarization with electronic case documents (56 points), numerous requests from litigants (51 points), and interdepartmental interactions and coordination (51 points).

User Feedback on ICT System Components

6. The following automated systems received the highest performance scores: the external internet portal of MCC and district courts in Moscow (63 points), the internal internet portal of MCC and district courts in Moscow (61 points), the information and analytical system (61 points), the videoconferencing system with correctional institutions (60 points). In addition, satisfaction with the work of all automated systems rose by 10-20 points and by 13 points on average for all systems among respondents, compared to the baseline survey. Scores for ICT system components increased as follows:

The e-Case system - on average by 25 points among court personnel; The data exchange system between courts at different levels in Moscow – on average

by 24 points among court personnel; The audio-video recording system - on average by 18 points among court personnel; The external internet portal of courts of general jurisdiction - on average by 26 points

among court personnel and by 12 points among professional users; The automated information and analytical subsystem - on average by 26 points among

court personnel; The electronic archive - on average by 18 points among court personnel and by 10

points among professional users; The internal information portal of courts of general jurisdiction in Moscow - on

average by 29 points among court personnel. 7. The elements of the ICT system most in demand among court personnel are: the automated archive functions (65 points, a rise by 14 points), the information and analytical system (65 points, an increase by 8 points), the internal portal of MCC and district courts in Moscow (64 points, a rise by 7 points), e-filing of applications and complaints from the public and other users and e-enforcement system (63 points, a growth by 7 points). 8. Court personnel and professional users’ assessment of key components of the MCC electronic case subsystem (a critical user-facing element of the ICT system) significantly improved in the final survey, reflecting increasing levels of user satisfaction with the JRSP-financed MCC E-case system (question 6). Question 6. Please assess the functioning of the following components of the MCC E-case subsystem


Page 111 of 148

9. Assessments of key elements of another JRSP-financed MCC ICT system – the audio and video recording system for court proceedings – improved significantly at the final survey, reflecting increasing satisfaction among court personnel and professional users with this system. This improvement can be attributable to the JRSP intervention, including training provided to these user categories as the system was piloted and rolled out. Question 7. Please assess the functioning of the following elements of the audio-video recording system

66

66

63

58

40

37

38

38

Document search subsystem

Administration subsystem

Storage and tracking of documents

Scanning and recognition of documents

Final survey Baseline survey


Page 112 of 148

10. Court personnel and professional users’ assessment of the functioning of key elements of the automated information and analytical subsystem – which has automated many key tasks including case tracking, decision publication and depersonalization of personal and other information as required by law – significantly improved by the final survey (question 8). Question 8. Please assess the functioning of the automated information and analytical subsystem according to the following criteria…

72

69

69

69

68

67

64

50

54

49

53

52

50

47

Quality of video recording

Accessibility of records

Level of synchronization between video andaudio

Friendliness of Chief Judge's workstation

Friendliness of Secretary's workstation

Acсessibility of case information for litigants

Quality of audio recording



Page 113 of 148

11. Court personnel and professional users expressed satisfaction with introducing the automated system: 69 percent noted the ease of use of the system for daily tasks (an increase by 16 percentage points in comparison with the baseline survey). Such users also gave high scores to the use of the system for daily tasks, with 67 percent of court personnel (a rise of 5 percentage points) and 70 percent of professional users (a rise of 27 percentage points) expressing satisfaction. 12. 73 percent of general public respondents stated that the internet portal of MCC was convenient, while 62 percent said the same for information kiosks/terminals in court buildings and 67 percent for the E-case system. 51 percent of such users believe that the quality of the work of district courts will improve after introducing the automated system. 13. The following subsystems of the new automated system saw the highest demand from users: the external internet portal of MCC and district courts in Moscow (70 points), the information and analytical system (66 points), the unified e-repository of judicial documents, cases with barcoding technology (63 points), automated archive for Moscow courts (63 points), the internal portal of MCC and district courts in Moscow (62 points), the videoconferencing system with correctional institutions (61 points).

73

70

69

68

67

66

66

65

63

58

46

40

38

43

42

42

41

34

39

35

Ability to track the stage of case processing

Loading information from workflow managementdatabases to unified database

Monitoring the results of loading information fromdifferent sources

Payment of court fees through the portal

Automated publication of case decisions andinformation under Law 262-FZ

Accessibility of step-by-step guide with documentsamples

Ability to process information and generateautomated reports

Ability for automated electronic filing of documents

Ability to adapt the system to changingrequirements

Ability to apply automatic depersonalization ofinformation in decisions and documents



Page 114 of 148

14. The highest rise in scores in comparison to the baseline survey were for: (a) the automation of the archive functions for courts in Moscow (63 points, a rise by 15 points), (b) the information and analytical system (66 points, an increase by 14 points), (c) the internal portal of the MCC and district courts in Moscow (62 points, a rise by 12 points) and (d) the unified e-repository of judicial documents (63 points, a rise by 9 points).


Page 115 of 148

ANNEX 9. MOSCOW CITY COURTS AUTOMATED SYSTEMS ‐ KEY FEATURES AND RESULTS

The information below has been adapted from materials from the Moscow City Courts provided courtesy MCC President Ms. O. Egorova and MCC Deputy President Mr. D. Fomin on May 29-30, 2018. Results relate to achievements as of December 31, 2017


Page 116 of 148


Page 117 of 148


Page 118 of 148


Page 119 of 148


Page 120 of 148


Page 121 of 148


Page 122 of 148


Page 123 of 148

ANNEX 10. SUPREME ARBITRAZH COURT & COMMERCIAL COURTS ICT SYSTEM

The information below has been adapted from materials from the Supreme Arbitrazh Cpurt provided courtesy Ms. Helena Avakyan, former Head, Department of Control and Analytics, SAC in 2012. Results relate to 2012 when the JRSP-financed ICT systems for the commercial courts and SAC were installed.


Page 124 of 148


Page 125 of 148


Page 126 of 148


Page 127 of 148


Page 128 of 148


Page 129 of 148


Page 130 of 148


Page 131 of 148


Page 132 of 148


Page 133 of 148

ANNEX 11. SUPPORTING DOCUMENTS

1. World Bank. Project Appraisal Document, JRSP, 2007 2. World Bank. Country Partnership Strategies (Russian Federation), 2007, 2012, 2017 3. World Bank. Systematic Country Diagnostic, 2017 4. World Bank. Implementation Status Reports, 2007-2017 5. JRSP Implementation Status Reports 6. MCC Survey Reports


Page 134 of 148

ANNEX 12. SUMMARY OF RUSSIAN FEDERATION’S ICR

The Russian Federation prepared and shared a detailed ICR which has been filed. The Executive Summary of that ICR, prepared by the Borrower, is attached below.

Loan Agreement No. 4849-RU

between the Russian Federation and the International Bank for Reconstruction and Development

BORROWER'S IMPLEMENTATION COMPLETION REPORT

Judicial Reform Support Project

(Executive Summary) General Project Information

Loan Agreement No. 4849-RU between the Russian Federation and the International Bank for Reconstruction and Development (IBRD, the World Bank) for financing the Judicial Reform Support Project (Agreement, Loan Agreement) in the total amount of US$ 50 million was signed on July 17, 2007 and became effective on November 15, 2007.

During the course of the project implementation the Loan Agreement was amended in the context of changes in the objective project implementation conditions to reflect (also in the tables with PDO level and intermediate results indicators) closing date extension, project restructuring issues agreed between the Russian Federation and the IBRD as a result of selection of a new Project Implementation Unit, and structural judicial reforms. The substantial change was made to the project financing percentage of expenditures to be financed from the Loan under category (1) from 29% to 11.57% as of May 01, 2017.

The implementation of project activities was completed on December 31, 2017. The final payments under the project contracts for goods and services supplied before December 31, 2017 were made during the grace period provided by the IBRD through April 30, 2018. Overall, the amount of US$ 49 898 893.89 was disbursed out of the proceeds of the Loan during project implementation, or 99.8% of the Loan amount. Given the change in the portion of the loan proceeds allocated to finance project costs under category “(1) Goods, technical services and consultant's services for the Project”, the amount of counterpart funds provided by the Government of the Russian Federation was equivalent to more than US$ 138.86 million. The total project cost was US$ 188.76 million.

The Ministry of Economic Development of the Russian Federation31 was acting as the federal executive authority, the project entity and the chief budget funds administrator in accordance with the legal and regulatory acts applicable to the Project32. The Constitutional Court of the Russian

31. Until May 12, 2008 ‐ the Ministry of Economic Development and Trade of the Russian Federation 32 In particular, RF Government Resolution 43 “Operational procedures for projects being implemented by the Russian


Page 135 of 148

Federation, the Supreme Court of the Russian Federation, the Supreme Commercial Court of the Russian Federation and commercial courts (including the Court of Intellectual Rights), courts of general jurisdiction (including the Moscow City Court and Moscow district courts of general jurisdiction), and administrations of the Judicial Department of the Supreme Court of the Russian Federation were acting as the recipients of goods and services under the Project - Project Beneficiaries. The following project management bodies were responsible for interagency coordination under the Project: Inter Agency Management Council created by order of the MOED and comprised of representatives from MOF and judicial authorities - Project Beneficiaries and Project Implementation Unit; working groups responsible for project oversight and management created by the key Project Beneficiaries (Constitutional Court, Supreme Court, Supreme Commercial Court, Court of Intellectual Rights, Moscow City Court, Judicial Department of the Supreme Court) responsible for implementing project activities. The working groups included the representatives from MOED, MOF and PIU.

The Project Implementation Unit assisted the MOED and the Project Beneficiaries with project implementation. The PIU functions were performed by the Bureau of Economic Analysis 33(BEA) during the period from 2007 to 2011. In December 2011, PIU functions were delegated to the Noncommercial Foundation for Enterprise Restructuring and Financial Institutions Development (FER)34. The PIU was acting on the basis of the Agency Agreement between PIU, MOED and MOF.

Project Goals and Objectives

The project development objective (PDO) as specified in Schedule A “Project Description” to the Loan Agreement is to assist the Borrower to strengthen judicial transparency and efficiency of selected courts through the implementation of information systems and judicial training. The project objective was not revised during project restructuring.

The project structure included the following components. The composition of project components was not revised during project implementation and is reflected in Schedule 1 to the Loan Agreement:

(1) Component A. Institutionalizing judicial transparency and accountability

(2) Component B. Use of Information and Communications Technology for Judicial Transparency and Effectiveness

(3) Component C. Strengthening Human Capital

(4) Component D. Project Management, Monitoring and Evaluation

Key Project Results

Federation with the assistance of international financial organizations” dated January 28, 2005; RF Government Resolution 361 “On signing the Agreement between the Russian Federation and the International Bank for Reconstruction and Development for financing the Judicial Reform Support Project” dated June 9, 2017 33 Legal entity registered under the Russian laws in the form of noncommercial organization. 34 Legal entity registered under the Russian laws in the form of noncommercial organization.


Page 136 of 148

All project activities were performed in compliance with the project development objective described in the Loan Agreement. Most project activities were aimed to improve transparency and effectiveness of judicial authorities involved in the Project through upgrading existing and developing new ICT and providing judicial training. At the same time, some activities envisaged under project components in accordance with the Loan Agreement were supplementary and had indirect impact on achievement of the key project objective. These activities include completed social surveys, analysis of needs for training of specific categories of judicial staff, assistance to Project Beneficiaries with analysis of current information and communication infrastructure and preparation of concepts and technical requirements/terms of reference for extensive use of modern ICT in case management processes, services of interpreters and auditors, operations of the Project Implementation Unit.

Overall, completed project activities fully cover all thematic areas mentioned in the Loan Agreement in the description of project components despite significant changes made in the list of project activities as compared with the original project implementation plans developed at the project preparation and appraisal stage. The most substantial project results broken down by components are briefly described below.

Component A. Institutionalizing Judicial Transparency and Accountability

The activities implemented under Component A contributed to the achievement of the project development objective “Availability of data from periodic surveys of users of judicial services” which is measured with the use of logical yes/no scale. The target value of this indicator was achieved: during project implementation 5 waves of national social surveys of the attitude of citizens and organizations toward the judicial system and 2 waves of micro surveys of judicial process participants in Moscow courts of general jurisdiction (Moscow City Court and district courts) were carried out.

The results of the final survey of citizens carried out under the Project showed that the percentage of population that trusts the judicial system (49%) in 2017 was higher than the percentage of population that does not trust the judicial system (43%). First time during the implementation of social surveys under the Project the level of trust towards judiciary system showed positive value at 6 points. However, despite the positive trends, the judicial system continues to lag behind the most credible institutions in terms of the level of public trust. The survey results reported that judicial system was perceived in 2017 as a more credible institution. First time during project implementation the judicial system credibility index measured as the difference between positive and negative measures of judicial performance showed positive value and gained 17 percentage points. Furthermore, judicial system was less criticized by respondents for unnecessary bureaucracy and poor management (22% in 2012 versus 17% in 2017).

The attempt to carry out social surveys that more accurately reflect public assessment of extensive use of ICT by judiciary was initiated by the Moscow City Court.

Unlike national surveys, this survey covered not only potential users of judicial services acting as respondents, but also the staff of Moscow courts, professional trial participants (prosecutors, legal counsels, lawyers - representatives in civil cases, representatives of the government authorities), non-professional trial participants (plaintiffs, respondents, applicants in special proceedings, offenders in administrative cases). Questionnaires included additional blocks with


Page 137 of 148

the aim to seek respondents perception of the last steps made towards informatization of Moscow courts of general jurisdiction and respondents assessment of these steps and financial aspects of providing and receiving judicial services.

The survey showed that the implementation of the Integrated Information System of Moscow city general jurisdiction courts (GJC IIS) did not go unnoticed. The level of satisfaction with almost all automated court systems has increased by 10-20 points and by 13 points in average with all systems compared to baseline survey. Overall, the highest (on a 100-point scale) performance assessments obtained under these surveys were given to the following GJC IIS subsystems: external Internet portal of the Moscow City Court and Moscow district courts (63 points), internal portal of MCC and Moscow district courts (61 points), information and analytical system (61 points), videoconferencing sessions with correctional facilities of Federal Penitentiary Service (60 points).

The research and analytical activities, including those related to development of approaches to measuring the performance of justice systems, were undertaken under this component through implementing the following activities:

(1) Conducting survey and assessment of efficiency of interaction between courts and mass media, including social survey on organization of communication between Russian judicial authorities and mass media. Upon completion of survey and based on the analysis of related best practices, recommendations were developed on a number of aspects to improve communication between judicial authorities and mass media and to change the status and improve qualifications of court press relation officers. The recommendations to improve qualifications of press relation officers were used during preparation of advance training program for judicial staff involved in communication with mass media under Component C of the Project.

(2) Scientific analysis and systematization of documents of the Constitutional Court, including development of principles for classification of decisions of the Constitutional Court, approbation of approaches to the identification of constitutional and legal provisions in the decisions and development of approaches to processing and systematization hereof, development of classifiers of decisions of the Constitutional Court and identified constitutional and legal provisions that are consolidated into unified classifier of decisions and constitutional and legal provisions; development of subsystem “CC Analytics” in the information system of the Constitutional Court for making available results of the decision systematization; systematization of 15 000 acts of the Constitutional Court. The search function of the developed subsystem improves judicial performance by reducing time needed to find relevant and complete information, which, in turn, facilitates the adoption of consistent and coherent decisions.

(3) Analysis of the Russian legal and regulatory framework and development of proposals to improve it with the aim to create a legal framework for ICT use in case management processes and in communication with other government authorities of the Russian Federation and their territorial bodies. Under this activity, the legal gaps and inconsistencies were identified in the federal legislation, subordinate legislation, departmental legal and regulatory acts. To confirm conclusions of analytical survey, field visits were undertaken to Moscow courts of general jurisdiction, commercial courts and some regional government authorities with the aim to identify the needs of judicial system and civil society in legal regulation, including issues related to


Page 138 of 148

protection of state, commercial and official secret, confidential information, compliance with personal data use, processing and protection rules. Further, the status of IT use in these organizations was checked. Survey results contributed to the development of proposals to improve Russian legal and regulatory framework regulating ICT use in the case management processes and in communication with other entities. The proposals were reflected in 72 legal and regulatory acts at the federal, regional, interagency and intra-agency levels.

(4) Under the activity “Development of criteria and indicators of efficiency of functioning of judicial system under the Judicial Reform Support Project”, the analysis covered the court performance criteria and indicators (at the level of judicial statistics, Rosstat indicators and indicators of federal target program) used in the Russian practice, the system of court performance criteria and indicators developed in the Netherlands, Finland and the USA and at the international level (system of IFCE criteria and indicators developed by the International Consortium for Court Excellence with the assistance of IBRD). On the basis of analytical results, the approach was suggested to developing the system of judicial performance assessment indicators and criteria. It was determined based on proposed criteria and indicators to use the survey method as the main source of information on dynamics of indicators.

The pilot test was performed by the Consultant to assess the proposed system of criteria and indicators. However, it appeared that this system could not be used directly by judicial authorities on a regular basis since such activities were not included in the job descriptions of judicial staff (for conducting surveys) and/or it was necessary to increase workload on already overloaded judicial staff. Further, in order to introduce additional reporting forms it was necessary to prepare regulatory documents of the judicial system, justification and cost analysis for additional functions to be performed by judicial staff. At the same time, the survey results were used under the Project during the preparation of social survey instruments.

(5) Additional input in assessment of judicial performance was attributed to such activities as analysis of efficiency and development of recommendations for improving the videoconferencing system in the courts of general jurisdiction, a comprehensive assessment of the economic, financial and social impact of project activities and project sustainability. Both surveys showed that though public services projects are social projects that do not bring direct profits, investment in judicial informatization can be considered economically effective taking into account economic benefits from automated court case management systems and economic benefits which can be received by various counterparties of judicial system, as well as economic assessments of social effects.

Component B. Use of Information and Communications Technology for Judicial Transparency and Effectiveness

The component included four subcomponents focused on the key Project Beneficiaries.

Subcomponent B.1 included such activities performed for the Constitutional Court in full compliance with project development objectives and tasks of Component B as supply and installation of integrated case management system and automated system for publication of information on the Internet portal of the Constitutional Court, upgrade of information and telecommunication infrastructure of the Constitutional Court.


Page 139 of 148

Subcomponent B.2 supported the implementation of extensive range of activities initiated by the Supreme Court. The most substantial project results under this subcomponent include:

- Development of federal videoconferencing (VC) system supervised by the Supreme Court of the Russian Federation. Overall, the VC system was installed in all regional and same-level courts of general jurisdiction. 278 FPS offices were used as automation sites under two consistently implemented related activities. High economic efficiency (first of all, through savings of budget funds used for transportation of trial participants from detention facilities to the place where trial proceedings take place) and substantial social effects from using the VC system for remote participation in court proceedings of persons kept in detention facilities have been confirmed by analytical surveys conducted under the Project. Further, judicial authorities use VC systems, apart from court sessions, for conducting meetings, training and knowledge sharing events. The use of VC systems also improves judicial efficiency through saving of funds and time needed for participation in field visits, provides the opportunity to hold meetings with all judicial agencies equipped with VC systems.

- Upgrade of information and communication infrastructure of the Supreme Court, including strengthening of capacities for storage and processing of information electronically through upgrading court’s server system (among other things, introduction of advanced technologies for virtualization of IT resources), conversion of equipment of judges and judicial staff workplaces to a new hardware and software level, development of information protection system, etc. Through implementing these activities it became possible not only to improve judicial performance but also to create in a timely manner the technical conditions to ensure compliance with the requirements of the Russian legislation for publication of information on the activity of the Supreme Court in the Internet.

- Development of the Mid-Term Concept on Informatization of the Supreme Court, including delegation of powers of the supreme court on economic disputes and planned reallocation of the Supreme Court to St. Petersburg). This Concept was developed to ensure substantiated balanced complex plan of actions on extensive use of the latest information technologies in courts coordinated with other programs on development of information society. The first steps toward the implementation of the Concept had already been made under the Project, specifically the implementation of specific activities on development of the court information protection system and upgrade of information systems, creation of the information system “Judicial Document Flow and Case Management in the Supreme Court of the Russian Federation” supporting modern information technologies. The main advantages of the developed system include substantial extension of the inter-agency and inter-level interaction functions and communication with trial participants electronically. The developed system allowed the Supreme Court to use electronic documents in case management processes in accordance with federal law No. 220-FZ of June 23, 2016 starting from January 1, 2017.

- The task to strength judicial transparency was addressed by the Supreme Court through installing the shared information display equipment (information panels and kiosks) in the courthouses and through creating a new version of the court's website linked to the developed information system of judicial document flow and case management.

The development and implementation of the Integrated Information System of Moscow city


Page 140 of 148

general jurisdiction courts (GJC IIS) initiated by the Moscow City Court under Subcomponent B.3 is considered to be one of the most significant results of the Project. The GJC IIS created under the Project includes such functional blocks as electronic archive, electronic case, internal portal of the Moscow City Court and district courts, video conferencing system, inter-level interaction system, information and analytical system, audio and video recording system, interagency interaction system. The interagency interaction system supports communication between the administration of the Federal Bailiffs Service, administration of the Federal Penitentiary Service and subordinate pre-trial detention facilities, Russian Post, Moscow Department of Housing and Utilities, Moscow Administration of Federal Treasury, Judicial Department and Moscow Administration of the Judicial Department, 438 JPs, Moscow Prosecutor's Office, Moscow Main Investigation Department of the RF Investigation Committee, lawyers, interpreters, and citizens. Overall, the system represents a pilot project designed to organize a unified information space for judicial authorities of the Russian regions to exchange electronic information with the relevant regional segment of the system for interagency electronic interaction of executive authorities and other bodies. The performance of Moscow court of general jurisdiction has been improved significantly even during relatively small period of the GJC IIS system operation. According to data for the 1st quarter of 2018 provided by the Moscow City Court, the enforcement rate of judicial decisions of Moscow courts of general jurisdiction is 39% out of the total number of decisions in civil cases, which is by 15% more than baseline of this indicator as of June 2014 (24%). The average time for preparation of the response to citizens’ appeals has been reduced from 30 to 14 calendar days. After the implementation of audio and video recording system, the number of cases challenging the minutes of court sessions has been reduced by 35.2%, the number of complaints from litigation participants on rights’ abuse during court proceedings has been reduced by 31.4%. Despite very impressive results of the implementation of the GJC IIS system, target values have not been completely achieved under all results indicators of this subcomponent during 3 months of system operation. This situation is partially attributable to rigorous commitments undertaken by the Moscow City Court to improve its performance indicators, not all of which are free from external impacts (e.g, low level of legal knowledge of trial participants) which are beyond efforts of Moscow courts. At the same time, we may assume that with overall high level of achievement of target indicators (not less than 78.5%), exceeded values under some indicators and positive dynamics of results, target values will be fully achieved under all indicators considering that GJC IIS system’s operational life is expected to be increased over time by 1-1.5 years.

Subcomponent B.4 supported the activities aimed to strengthen the openness and efficiency of commercial courts of the Russian Federation.

The most substantial activities undertaken in 2007-2014 in all commercial courts of the Russian Federation under this component include:

- Installation in all commercial courts of modern systems for storage of electronic documents with sufficient capacity to meet the needs of commercial courts for storage and access to electronic judicial documents, thus addressing the task of the Supreme Court to be able to publish all decisions of commercial courts in the Internet within 24 hours after the adoption of decisions.

- Development of departmental telephone network and unified corporate e-mail network for


Page 141 of 148

commercial courts contributing to enhanced efficiency in communication between judges and judicial staff by providing additional functionality and increasing system fault tolerance and to reduced recurrent costs of these courts by reducing the cost of telephone communication and maintenance of technical resources, including centralized management of networks;

- Installation of information mini kiosks in the premises of commercial courts in order to increase judicial openness and implementation of audio recording systems providing access to audio records of trial participants.

The implementation of subcomponent in 2014-2017 by Court of Intellectual Rights (CIR) contributed to achieving project development objectives. Considering that CIR had started its activity as part of the system of RF commercial courts in the first half of 2013, the initial court operations were supported by the automated case management system used earlier in the system of commercial courts, and CIR's information and communication needs were satisfied mainly based on the information and communication infrastructure of the Supreme Commercial Court. Given a relatively small period of CIR's operation and some information and communication problems caused by cessation of activities of the Supreme Commercial Court, one of the prioritized tasks of the Project for the period up to 2016 was to adjust the automated case management system to CIR's specific needs and changes in the Russian legislation within a short period of time and to assist CIR with the development of its own information and communication infrastructure. To address the above tasks, CIR's server system was created under the Project supporting CIR's needs for data processing and storage (including data of stream broadcasts of court sessions), active network equipment was upgraded, modern system of electronic document management was procured, new modules with arbitration case management functionality specific for CIR were developed, VC system was installed in the Presidium Hall. The Project also included the development of the court informatization concept. The CIR informatization concept includes analysis of CIR informatization needs; key informatization areas; conceptual CIR informatization architecture; concept implementation plan; feasibility study on selected architecture and technology stack. All activities performed for CIR contributed to improving its performance.

Component C. Strengthening Human Capital

This component contributed to improving qualifications through off-the-job training in the use of modern ICT of 272 ICT specialists in all commercial courts covering the topic on “Support of enterprise network infrastructure and mail system”, training of 196 employees from 111 commercial courts covering the topic on “Development of network infrastructure of commercial courts with the use of modern Microsoft network technologies”, training of 2750 employees of the courts of general jurisdiction and administrations of the Judicial Department of the Supreme Court of the Russian Federation in the use of modern software supporting GAS Pravosudie. 80 judges and judicial staff of the Court of Intellectual Rights were trained in the program on “Effective use of office software”, 3 IT specialists were trained in the standard training courses (certified by software manufacturers) on the use of modern software.

At the same time, training provided under the contracts on implementation of information systems was rated by the Project Beneficiaries as the most effective form of training on the use of such systems. Overall, 4109 people were trained in the use of new systems under the contracts on development of information systems in the judicial authorities involved in the Project.


Page 142 of 148

The component also supported upgrading of qualifications of 287 civil servants involved in communication with mass media in the courts of general jurisdiction, commercial courts, administrations of Judicial Department of the Supreme Court of the Russian Federation. The component included experience sharing between the representatives of the judicial authorities of the Russian Federation and foreign countries during visits of the representatives from the Constitutional Court, Supreme Court and Supreme Commercial Court to the supreme courts of foreign countries. The RF representatives shared the results of their work and studied the experience of the European Court of Human Rights, supreme courts of Germany, France, Austria, Switzerland, Australia, Singapore, and Republic of Korea in the area of application of information technologies in case management systems.

On the basis of the above, target value of the indicator “Number of manpower trained under the Project (number of people)” during project implementation was exceeded by more than 1.5 times.

The tasks under Component D “Project Management, Monitoring and Evaluation” described in the Loan Agreement were fully completed. The PIU costs were covered throughout the life of the project. The PIU was responsible for providing management and technical support for Inter Agency Management Council and other project management bodies (working groups, evaluation committees created by Project Beneficiaries), collecting information and preparing regular reports on achievement of PDO level and intermediate results indicators.

As to provision of technical assistance to courts involved in the Project, the component financed the contracts for provision of consulting services to the Supreme Court on development of design and estimate documentation for optimization of scheme for installation of local area network and upgrading the engineering infrastructure supporting LAN with the aim to increase system reliability (the developed documents were used by the Supreme Court to upgrade engineering infrastructure using budget funds) and contracts for provision of consulting services to the Judicial Department of the Supreme Court in the area of implementation of activities under the Judicial Reform Support Project for automation of courts of general jurisdiction and the system of judicial department of the Supreme Court in order to provide material and technical conditions for improving openness and access to justice (including the development of specifications for equipment, detailed design and terms of reference for provision of technical services). The foreign consultant was engaged under the component to provide assistance in conducting two-stage bidding process for award of contract on GJC IIS system development. Key Factors Affecting Implementation and Outcomes

1. Constructive interaction supported by IBRD, MOF and MOED on issues of project oversight and supervision and in dealing with project implementation issues.

2. The RF Government’s attention to project implementation issues and to ensuring project effectiveness, including sustainability of project outcomes. With the RF Government's assistance, it became possible to ensure uninterruptable project financing in the required amounts, to prepare the second project in support of judicial system development and enforcement of judicial decisions, which is aimed to ensure, among other things, sustainability of the JRSP results and will


Page 143 of 148

be financed out of the loan provided by the New Development Bank.

3. The composition of the IBRD project team has not been changed. The IBRD project team members supported the Project at the project preparation stage and at all times during implementation of the Project. The project team included highly skilled specialists with extensive experience in judicial system development projects implemented in foreign countries (including informatization of case management systems) and contributed significantly to successful implementation of the Project and achievement of Project objectives. In particular, extensive practical experience of the IBRD experts contributed to timely identification of project implementation problems and to the development of effective mechanisms for addressing such problems, including through effective project restructuring within a short period of time.

4. Bank's flexibility in project management and consideration of specific aspects of country and national legislation, directly or indirectly affecting the Project. At all times during implementation of the Project, the Bank was flexible in its response to ongoing changes, taking into account the Borrower’s suggestions, where possible. The flexible approach in review of IT-related procurement and contract amendments contributed to timely procurement of necessary equipment to address the Beneficiaries’ tasks, on the one hand, and to effective project disbursement, on the other. The Bank's project team and management showed understanding of project implementation delays at the initial project stage and structural changes in the judicial system at the subsequent project stages and created the most favorable conditions for successful completion of the Project given the social importance of the judicial system improvement.

5. Project design aimed to supplement federal targeted programs on development of the judiciary system in Russia and other programs. Overall, commitments declared in all programs to achieving key objectives of judicial system modernization - strengthening judicial transparency and efficiency - allowed all Project Beneficiaries to carry out regular analysis of the list of prioritized informatization activities and reallocate (if necessary) activities which are of primary importance between the sources of funding to achieve better results. Notwithstanding that changes in the list and/or scope of project activities proposed by Project Beneficiaries slowed down the pace of project implementation, extension of the project implementation period was compensated by increased efficiency in the use of financial resources by Project Beneficiaries (e.g., implementation of identified prioritized needs not covered by budget funding). Thus, despite a substantial extension of the actual project implementation period as compared with originally planned dates, project performance was measured at 100%, provided that values of some project performance indicators were exceeded, in accordance with performance assessment methodology developed by MOF and MOED that reduces the project rate in case of project extension.

6. High level of project management

The clear project management system that included Inter Agency Management Council and working groups created by the key Project Beneficiaries was used throughout the life of the project.

The Operational Manual was developed and agreed with the IBRD to ensure proper level of project management. The Operational Manual included the key documents regulating daily project activities: project management rules and procedures, procurement procedures, financial management system, staffing policy, etc.


Page 144 of 148

7. The special organization (project implementation unit, PIU) was engaged to perform procedural and technical functions during project preparation and implementation, including procurement, opening and maintaining the Designated Account, preparing financial reports in accordance with the IBRD procedures, providing assistance to Project Beneficiaries with preparation of documents required for conducting bidding procedures and contract signing, and ensuring day-to-day contacts with the IBRD.

8. Sustainability of project results supported by project planning in accordance with the long-term strategic plans for building information society; incorporation of achieved results and needs for further improvement of information and communication infrastructure of the judiciary. Performance of the World Bank and the Borrower

Performance of the World Bank

Performance of the World Bank during project preparation and supervision can be rated as highly satisfactory.

The close monitoring over the project implementation by IBRD helped ensure compliance with the Loan Agreement and applicable procurement guidelines. The cooperation with the World Bank’s Moscow office was very productive and helpful in dealing with emerging issues, thus contributing to successful project implementation.

The IBRD reviewed and approved implementation strategies, project implementation plans and procurement plans, bidding documents/RFPs at various stages of procurement procedures, advised MOF, MOED, judiciary - Project Beneficiaries and Project Implementation Unit on various project implementation issues in accordance with the Loan Agreement, the General Conditions Applicable to Loans of the International Bank for Reconstruction and Development (dated July 1, 2005, revised and amended on October 17, 2007), the Guidelines for Procurement under IBRD Loans and IDA Credits (May 2004), the Guidelines for Selection and Employment of Consultants under IBRD Loans (May 2004) and other documents. The IBRD comments were generally constructive and justified and contributed to enhancing effectiveness of procurement procedures and the quality of bidding documents/RFPs.

The IBRD reviewed the submitted reports (both financial and substantive) on the progress

with the implementation of project components as part of the supervision of the project progress. The IBRD also regularly organized field supervision missions for evaluating the project’s implementation and outcomes and provided its recommendations to MOF, MOED, judiciary - Project Beneficiaries, and Project Implementation Unit upon completion of such missions. Of particular note is constructive interaction between IBRD, MOF and MOED in view of unsatisfactory project rating in 2009-2011. Owing to advisory and management support provided by the World Bank team, it was possible to agree and make a decision to replace the Project Implementation Unit, extend project implementation period in order to achieve objectives of beneficiaries, prepare proposals and make amendments to the Loan Agreement. The Project was rated as satisfactory by the IBRD in 2012 based on the results of the implemented measures.

Performance of the Borrower and Project Implementation Unit


Page 145 of 148

MOF, MOED, judiciary - Project Beneficiaries, the Noncommercial Foundation for Enterprise Restructuring and Financial Institutions Development performed their functions in accordance with the rules and procedures applicable to the Project. The performance of MOF, MOED, judiciary - Project Beneficiaries, the Noncommercial Foundation for Enterprise Restructuring and Financial Institutions Development performing PIU functions can be rated as satisfactory.

Advantages of implementation of the IBRD-assisted projects

A wide range of judicial system modernization activities was expected to be implemented under the federal target programs on development of the judiciary system in Russia for 2007-2011 and 2013-2020 in the context of limited funds allocated in the federal budget. However, the planned activities were not exhaustive taking into account challenges for the judiciary. The IBRD participation in project financing helped significantly speed up the informatization processes in the supreme courts of Russia (Constitutional Court, Supreme Court, Supreme Commercial Court), address a number of urgent information system upgrade issues and implement a number of IT development activities in all commercial courts to support the development of a single information space for this branch of the judiciary. The most significant factors of the IBRD involvement in the project include favorable project financial conditions; provision of significant amount of loan funds by IBRD; grace period for repayment of principal amount, low interest for debt service at all times throughout the duration of the Project. Additional advantage of attracting funds from the international financial organizations for implementation of development projects is that Russian legislation guarantees protection of federal budget articles on debt service from reductions in case of adverse financial situation in the country. Unlike other sources of funding of public projects, this fact guarantees sustainable implementation of agreed modernization plans.

Despite factors related to favorable financial conditions, the following factors played a positive role in enhancing efficiency in the use of funds under the Project and other IBRD-assisted projects:

- Regular and close monitoring and evaluation of Project by the IBRD at all stages;

- Strict and unified procedures for procurement of goods, works and services in accordance with the IBRD standards;

- IBRD participation in the analysis and approval of objectives, tasks and scope of works under each important area of activity;

- A wide coverage of markets of suppliers of goods, consulting services, training services, including international markets, due to the use of international bidding procedures.

- Enhanced quality and objectivity of assessment of project achievements through regular independent assessment of progress and achievements conducted by IBRD highly skilled experts and invited experts with extensive international experience;

- Advice of IBRD highly skilled experts during project implementation;

- Enhanced project reputation and importance due to participation of major international financial organizations.


Page 146 of 148

Conclusion

Upon completion of the Project we may confidently say that almost all project activities contributed to achieving Project development objectives which cannot be always measured quantitatively for objective reasons. Based on the wording used to describe the project development objective we may assume that this objective has been certainly achieved by the Project and by almost all judicial authorities - Project Beneficiary.

Sustainability of the achieved judicial system upgrade results, including the results of the Judicial Reform Support Project is confirmed by actual data on the use of the created capacity.

The international conference on disseminating project results and sharing knowledge in the area of judicial informatization “21st Century Court: Technology in Service of Justice” which was held in Moscow in May 28-29, 2018 demonstrated a very impressive data on the functioning of information systems which were created mainly as a result of project activities.

The report presented at the conference by Ms. Avakyan, who was acting as the Head of Control and Analytical Department of the Supreme Commercial (Arbitrazh) Court of the Russian Federation (SCC) before the SCC was abolished, showed that unified database on arbitration disputes of commercial courts is currently used as a day-to-day instrument by legal practitioners. As of the end of 2017, 22 556 242 cases had been registered in the database, and up to 35 000 new judicial documents are added daily into the database. Nearly 400 000 unique users use database every day, the number of queries amounted to 93 million in 2017. The intensity of database updates and usage shows that it has been growing steadily during the entire period of system operation. Thus, in 2013, the number of new documents in the system amounted to 22 million, the number of new registered judicial cases amounted to 1 million. In 2014, 2 million new cases were registered in the system, and 27 million of documents were added. In 2016, 2 million of new cases were registered, the number of added documents amounted to 32 million. The number of unique system users was 3 million in 2010, 10 million in 2014, 12 million in 2016.

The distinctive feature of the information resource is that it is possible to add court orders as image of original document and to obtain complete information on the case, including results of case review in all instances. The information resource provides electronic real time access to information on 117 commercial courts, as well as the possibility to track new claims and changes in the case using case number and the name of the company acting as a party to proceedings. Further, the system provides the possibility to receive notifications by e-mail. The speaker specifically noted that information resource on judicial practice in resolution of economic disputes created in the system of commercial courts is currently used by legal practitioners as a good alternative to legal search systems because of better search function.

In 2017, 17% of claims were submitted to commercial courts electronically in the first instance, and around 35% of requests were submitted electronically to the appellate instance.

Since the entry into force of Federal Law #220-FZ dated June 23, 2016, 126 081 documents have been signed with a digital signature in the system of commercial courts and 172 375 court orders have been issued electronically.

The modern system of departmental e-mail of commercial courts created under the Project has been also intensively used. Nearly 7,000,000 e-mail messages are processed in the system


Page 147 of 148

monthly.

Ms. Egorova O.A., Chief Justice of the Moscow City Court, stated at the conference that before implementation of GJC IIS system created under the Project the level of public awareness about judicial system was low, courts, trial participants and related agencies communicated in paper, the level of judicial transparency was low. This significantly reduced the effectiveness of Moscow courts, particularly given the increase in the number of requests for judicial protection of violated rights (nearly 0.5 million cases and materials were submitted to Moscow courts in 2010, while in 2017 1.5 million cases and materials were submitted).

100% of information is received electronically in Moscow City Court from district courts of general jurisdiction through GJC IIS. Even though the system became operational in December 2017, 22 000 citizens have already created personal accounts on mos-gorsud.ru portal. More than 450 thousand documents had been scanned in 2017, generation of statutory reports has been accelerated by 30 times, more than 1500 writs of execution have been submitted to FBS electronically. Enforcement rate for collection of state fees has been increased in half in Moscow and Moscow region due to electronic interagency interaction. This indicator was at 90% in 2017.

The Data Center (DC) created under the Project in 2016 based on MCC consolidates all

computing resources and data storage media (legal cases in electronic format, audio and video records of court sessions and other data) that support stable and smooth operation of IIS GJC. As of June 15, 2018, DC contained information on 3 145 758 cases (including cases for previous years transferred to DC) and on 1 376 629 court orders. The archive of audio and video records of court sessions contains more than 659 Tb of data.

The data provided shows that material and technical capacity of project beneficiaries

created under the Project is of high demand and is actively used , and this strengthens judicial transparency and efficiency.

Sustainability of project results obtained by Supreme Court (SC) and Court of Intellectual

Rights (CIR) will depend, first of all, on implementation of the mid-term informatization concepts developed under the Project. The key areas and tasks of further implementation of modern technologies in SC and CIR as specified in the said concepts correspond to the tasks of further development of judicial system and information society in Russia identified at the highest level of government authorities and administration.

The use of project results by beneficiaries will contribute to achieving goals and objectives of high-level programs such as the Concept of the Long-term Social and Economic Development of the RF (SED Concept)35 and the Information Society Development Strategy36 and to achieving sustainability of project outcomes.

35 The Concept of the Long-term Social and Economic Development of the RF through 2020 (SED Concept) was approved by the RF Government Resolution #1662-p dated 17.11.2008 “On the Concept of the Long-Term Social and Economic Development of the RF through 2020”. 36 The Concept of the Long-term Social and Economic Development of the RF through 2020 (SED Concept) was approved by the President of the Russian Federation No. Pr-212 of 07.02.2008.


Page 148 of 148

Given the advantages of implementation of plans on modernization of specific sectors of the government authorities with the assistance of the international financial organizations as demonstrated under the Project, the project results are expected to be further developed under the Judicial System Development Project (JSDP) to be financed from the loan of the New Development Bank provided to the Russian Federation. The objective of the new project is to enhance judicial efficiency and improve judicial protection of rights and legitimate interests of individuals and legal entities by infrastructure development and implementation of modern information technologies. This underlines continuity of strategic judicial system development completed under the Judicial Reform Support Project.