Data Analytics National Academy of Indian Railways Workshop on 'New Financial Initiatives' in Indian Railways For Principal/ Coordinating FA&CAOs 23 December

Data Analytics

National Academy of Indian RailwaysWorkshop on 'New Financial Initiatives' in Indian Railways

ForPrincipal/ Coordinating FA&CAOs

23 December 2015

Data Analytics - NAIR 2

There areLies

Damned lies; and?Statistics

23-Dec-15


Data AnalyticsWhat do you understand by it?

23-Dec-15

Data Analytics - NAIR 423-Dec-15

An example

Data Analytics

Digital Dashboard

Use of Numbers

Data on Indian Railways

Reorganising Statistical Units

Analytics Ecosystem on IR

A word of caution

Plan for the session


An example from RailwaysNot Financial Data

23-Dec-15


Accidents on Indian RailwaysAn illustration

23-Dec-15


Cause 2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15Failure of Railway

staff84 87 83 63 56 52 46 51 62

Failure of other than Railway staff

87 81 68 75 57 63 59 56 58

Failure of equipment

7 9 0 6 5 5 6 4 2

Sabotage 8 7 13 14 16 6 3 3 3Combination of

factors1 0 4 1 3 1 0 0 0

Incidental 7 8 5 4 4 3 7 3 8Could not establish

1 2 4 2 0 1 0 0 0

None Held 0 0 0 0 0 0 1 0 2Awaited 0 0 0 0 0 0 0 1 0

Total 195 194 177 165 141 131 122 118 135

Cause wise Analysis of Consequential Train Accidents over IR(2006-07 to 2014-15)

23-Dec-15


2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15

195 194

177

165

141

131

122118

135

Accidents on Indian Railways

23-Dec-15


2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15

195 194

177

165

141

131

122118

135


23-Dec-15


2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15

195 194

177

165

141

131

122118

135


23-Dec-15


2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15

8 8

139

59

64 5

96100

85

80 80

55

4953

63

4 53 2 2

4

9 7 67

12

75 5

75 4

68

47

41 2

03

5

72

6562

65

48

54 53

4750

Collisions Derailments Fire Manned Level Crossing Accidents Miscellaneous Accidents Unmanned Level Crossing Accidents

Consequential Accidents over the years

23-Dec-15


Collisions5%

Derailment48%

Fire3%

Manned Level Crossing Accidents

4%

Unmanned Level Crossing Accidents

37%

Miscellaneous Accidents2%

Accidents by type in 2006-2015

23-Dec-15


2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13

2013-14

2014-15

96100

8580 80

55

4953

63

Derailments

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13

2013-14

2014-15

72

6562

65

48

54 53

4750



23-Dec-15


2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13

2013-14

2014-15

96100

8580 80

55

4953

63

Derailments

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13

2013-14

2014-15

72

6562

65

48

54 53

4750



23-Dec-15


Seasonal variations?

A widely prevalent belief

Specific types of accidents have higher frequency in different times of the year

23-Dec-15


April May June July August September October November December January February March

120

131

115109

136

102

125

103

119 117

10398

Month-wise distribution of Total Accidents in the period 2006-2015

23-Dec-15



120

131

115109

136

102

125

103

119 117

10398


23-Dec-15



120

131

115109

136

102

125

103

119 117

10398


23-Dec-15


January February March April May June July August September October November December

117

10398

120

131

115

109

136

102

125

103

119


23-Dec-15


July August September October November December January February March April May June

109

136

102

125

103

119 117

10398

120

131

115


23-Dec-15


May-June

23-Dec-15


July-September

23-Dec-15


December - January

23-Dec-15


March – AprilOctober - November

23-Dec-15


Month-wise distribution of Different types of Accidents in the period 2006-2015

AprilMay

JuneJuly

August

Septem

ber

October

November

December

January

Febru

aryMarc

h

46

65

52

39

46

27

33

45

3937

45

42


AprilMay

JuneJuly

August

Septem

ber

October

November

December

January

Febru

aryMarc

h

52

46

5457

76

60

79

41

5452

4446

Derailment

23-Dec-15


Month-wise distribution of Different types of Accidents in the period 2006-2015

AprilMay

JuneJuly

August

Septem

ber

October

November

December

January

Febru

aryMarc

h

46

65

52

39

46

27

33

45

3937

45

42


AprilMay

JuneJuly

August

Septem

ber

October

November

December

January

Febru

aryMarc

h

52

46

5457

76

60

79

41

5452

4446

Derailment

23-Dec-15


Collision 3%

Derailment 52%

Fire 3%

MLC Accident 8%

Miscellaneous Accident 2%

UMLC Accident 32%

Monsoon-Gangetic Plain

Collisions5%

Derailment48%

Fire3%

Manned Level Crossing Ac-cidents

4%


37%

Miscellaneous Accidents2%

Accidents by type in 2006-2015

Derailment 72%

Fire 6% UMLC Accident

22%

Monsoon-East

Collision 3%

Derailment 41%

Fire 6%

MLC Accident 3%

UMLC Accident 47%

Monsoon-West

Collision 8%

Derailment 39%

Fire 3%

MLC Accident 11%

Miscellaneous Accident 3%

UMLC Accident 37%

Fog-Gangetic Plain

Collision4%

Derailment49%

Fire4%

MLC Accident4%

Miscellaneous Ac-cidents

2%

UMLC Accidents37%

Harvest Seasons

23-Dec-15


Failure of Railway staff50%

Failures other than of Railway staff35%

Failure of equipment 5%

Sabotage4%

Combination of factors1%

Incidental4%

Could not be estb.1%

None Held0%

Causes of Accidents 2006-15

23-Dec-15


2000-2001 2001-2002 2002-2003 2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008- 2009 2009- 2010

2010 - 2011

2011 - 2012

2012-2013 2013-2014 2014-2015

293

249

186161

119 12085 88 75 63 56 52 46 51 60

109

103

118

107

78 86

84 8176

7557 63

59 5758

33

24

18

18

14 8

9 9

06

5 56 3

3

19

14

10

18

4 6

8 7

13 14

16 63 3

3

4

0

2

2

1 0

1 0

4 1

31

0 00

11

20

15

17

16 11

7 85

4

43

7 48

4

5

2

2

2 3

1 14

2

01

1 01

0

0

0

0

2

Causes of Accidents 2006-2015: Number wise break-up

Failure of Railway staff


Failure of equipment

Sabotage

Combination of factors

Incidental

Could not be estb.

None Held

23-Dec-15


2000-2001 2001-2002 2002-2003 2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008- 2009 2009- 2010

2010 - 2011

2011 - 2012

2012-2013 2013-2014 2014-2015

293 249186

161 119 12085 88 75

63 56 52 4651 60

109 103118

107 7886

84 8176

75 5763

59

57 58

33 24 1818

148 9 9

0 65

5 63

3

19 14 1018 4 6 8 7

1314

16

6 33

3

40 2 2

10 1 0 4

1 3 10

0

0

11 20 15 17 16 11 7 8 5 4 4 37

48

4 5 2 2 2 3 1 1 4 2 0 1 1 01

0 0 0 2

Causes of Accidents 2006-2015: Percentage break-upFailure of Railway staff Failure of other than Railway staff Failure of equipment Sabotage Combination of factors Incidental Could not be estb. None Held

23-Dec-15


Causes of Accidents 2006-2015: Trends

2000-2001

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008- 2009

2009-

2010

2010 -

2011

2011 -

2012

2012-2013

2013-2014

2014-2015

23%25%

34% 33% 33%

37%

43%42%

43%

45%

40%

48% 48% 48%

43%


2000-2001

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008- 2009

2009-

2010

2010 -

2011

2011 -

2012

2012-2013

2013-2014

2014-2015

62%60%

53%

50%51% 51%

44%45%

42%

38%40% 40%

38%

43%44%


23-Dec-15


Causes of Accidents 2006-2015: Trends

2000-2001

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008- 2009

2009-

2010

2010 -

2011

2011 -

2012

2012-2013

2013-2014

2014-2015

23%25%

34% 33% 33%

37%

43%42%

43%

45%

40%

48% 48% 48%

43%


2000-2001

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008- 2009

2009-

2010

2010 -

2011

2011 -

2012

2012-2013

2013-2014

2014-2015

62%60%

53%

50%51% 51%

44%45%

42%

38%40% 40%

38%

43%44%


23-Dec-15


Failure of Railway staff67%


7%

Failure of equipment 7%

Sabotage11%


1%

Incidental7%

Could not estb.1%

None Held0%

Awaited0%

DerailmentsCause-wise Analysis(2006-07 to 2014-15)

23-Dec-15



79%

Failures other than of Railway staff

19%


2%

Manned LC AccidentsCause wise Analysis(2006-07 to 2014-15)

23-Dec-15


Unmanned LC AccidentsCause wise Analysis(2006-07 to 2014-15)

23-Dec-15

Failure of Railway Staff1%

Failure of other than Railway Staff99%


Preventive Measures To Curb UMLCs

UMLC By Closure/ Merger/Subway By Manning Total

2010-11 800 434 12342011-12 481 777 12582012-13 700 463 11632013-14 777 325 11022014-15 721 427 1148

Total 2010-15 3479 2426 5905

2015-16 (Up to July’15) 206 65 271

As on 01.04.2015, there were approximately 29487 LC on IR out of which 19047 (64.6%) are Manned and 10440 (35.4%) are Unmanned.

Progress made in elimination of LC in last 5 years & up to July, 2015 by Closure, Merger, Subway and Manning are as under

MLC 2010-11 2011-12 2012-13 2013-14 2014-15 Total 2010-15 2015-16 (Up to July’15)

By Closure 133 225 257 301 310 1226 89

23-Dec-15


Data Analytics

23-Dec-15


What is data analytics?

Derive meaning from data by incorporating

Statistics

Mining of data

Visualisation

23-Dec-15

Extracting actionable data in a manner that supports

decision-making



Massive expansion in the ability of computers to handle data leading to certain

other crucial items now becoming possible:

Machine Learning

Database engineering

All this to solve complex problems

23-Dec-15



Caters to Big Data with a view to capturing major and seemingly minor relationships of performance indices

Caters to day to day reporting needs

Caters to ad hoc querying

Provides analytical dashboards and alerts

Provides comprehensive information and actionable insights for taking informed decisions

23-Dec-15


What has changed?

More powerful computers

Methods to obtain data directly from source

Data feeds available from diverse sources

Algorithms to extract information from unstructured data

23-Dec-15


4 Vs of Big Data

23-Dec-15

• Scale of data • Different forms of data

• Uncertainty of data• Analysis of data

Volume Variety

VeracityVelocity


Insights into the data

Traditional business intelligence

What happened?

Diagnostic analytics

Why is it happening?

Predictive

What will happen in future?

Prescriptive

What should we do?

23-Dec-15

Communication Flow in Analytics

Service request by User

Request assigned a priority number

Analyst, with assistance from

User, creates functional

requirements

Design team selects dashboard

format

Implementation team implements

the selected model

Feedback is sent to Analyst for

recalibration/modification

Continuous monitoring and

upgradation

Analytics Team

Functional User

Functional Analyst

providing liaison between User and Technology Team

Technology team

Software Programmers

to extract data from databases and prepare it for analytical models

Data Scientists

to decide choice of model and provide interpretation of analytical output in functional terms

Statistical Analysts/Econometricians

for developing appropriate logical and physical data models

Quantico Analysis Team

testing the quality of product from non-functional point of view


Present

23-Dec-15

Extract data Decide action

Delays in access to data

Limited electronic filing and diary

Limited integration with policy and financials

Multiple manual processes to enter, correct, extract and analyze data

Delayed decision making based on limited information and insights

Action?


Possibilities in future

23-Dec-15

Extract data Decide action

Shorter duration for analysis

Shorter time-lapse for decision

Quality of data analysis provides decision support

Continuous feedback on decisions taken

Rapid course correction if needed

Easier launching of new initiatives

Action

Features determining analytics maturity

Use of real time data

Electronic filing and diary in a centralised model for each business

Information integrated across the organization

Advanced modeling techniques used to evaluate across functional areas

Fully simulated business operations to evaluate decisions


Digital DashboardA crucial tool

23-Dec-15





The Digital Dashboard

Visual depictionTimely alerts

Multi deviceDrilldown capability

23-Dec-15


The Digital Dashboard

Relevance

Right insight at the right time

Convenience

Readily available when needed

Validation

Checked for errors and model validity

23-Dec-15



Use of numbersStatistician’s delight

23-Dec-15


la loi des grands nombres(Law of Large Numbers)

If the expected result of an experiment is random

And the experiment is repeated a large number of times

Then the results tend to stabilise over a period of time

23-Dec-15




Statistician’s Delight

23-Dec-15


Dependent and Independent variables

y = a0x0+ a1x1+ a2x2+ a3x3+ ε

23-Dec-15


Dependent and Independent variables

Throughput = f (Availability ofpowerreliable signalling,Crew,TXR cleared RS,efficient controllers,motivated staff and officers,reliable equipment of all types,

etc.)23-Dec-15


Optimisation

Max. [y – (ax+ by+ cz)]

23-Dec-15


Optimisation

Maximise Throughput

Subject to: loco failures, signal failures, crew non-availability, failing rolling stock, absence of crucial staff, demotivated staff and officers, other asset failures, etc.

23-Dec-15


Data on Indian RailwaysOur position

23-Dec-15


IDigitally CapturedDigitally Reported

IIManually CapturedDigitally Reported

IVDigitally CapturedManually Reported

IIIManually CapturedManually Reported

Quadrant I is ideal.However, on IR data can be available to varying degrees in all Quadrants.

Recording of data on IR

23-Dec-15



Several IT projects on IR have potential for data analytics:

Expenditure side application of PRIME/AFRES/IPAS/ e-Recon/ARPAN etc.

Data Warehouse for PRS

Data Warehouse for UTS

Data Warehouse for FOIS

Control Office application

Integrated Coach Monitoring System (ICMS)

Loco Sheds Management System (LSMS)

Software for Locomotive Asset Management

(SLAM) for Electric Loco Sheds

Track Management System (TMS)

MIS for Land & Amenities on IR

Traction Distribution Management System (TDMS)

Signaling Maintenance Management System (SMMS)

23-Dec-15



Tickets

reserved / unreserved

Freight

RRs

Train movement

control office

Crew

movement and lobbies

Maintenance

P Way and fixed assets

Maintenance

Rolling stock

Materials

purchase and depots

HR

manpower deployment and staff welfare

23-Dec-15

Costing and accounting – all of the above


Reorganising Statistical UnitsChange

23-Dec-15


Reorganizing Statistical Unit to Analytics Unit

Dynamic officers with adequate field experience and IT knowledge should lead the team

Identify comparatively younger staff and train them in data handling and analysis

Involve young JE level staff of EDP centres as part of the Unit

Unit should provide analyzed inputs to all departments

23-Dec-15


Analytics Ecosystem on IRSomething new

23-Dec-15


An Analytics Ecosystem for IR

23-Dec-15


Possible Areas for Analytics

Predictive maintenance

Dynamic pricing

Evaluating different marketing strategies

Improving capacity utilization/route congestion

Real time management of linear and rolling stock assets

23-Dec-15


Possible Areas for Analytics

Expenditure on Fuel consumption

Correlation with changes in fuel prices, Specific Fuel Consumption of locomotives, route electrification and transport output

RCD wise energy rate and consumption data

Automatic alerts in case the inventory of fuel increases to more than 10 days

Expenditure on Traction Bills

Correlation with prices of traction, Loco-wise and EMU coach wise energy consumption and transport output

TSS-wise energy rate and consumption data

Automatic alerts in cases of diesel locos running under wires23-Dec-15


Don’t rely too much on predictive capacity of current data

Some theoretical stuff

23-Dec-15


An economist can affect the economyas much as

The weatherman the weather

23-Dec-15


Logical fallacy

Post hoc ergo propter hoc?

The rooster crows before sunrise.

Ergo the rooster causes sunrise.

Cum hoc ergo propter hoc?

Rate of deaths in India due to TB increased even as civilian deaths during war in Iraq was increasing

Ergo War in Iraq was the cause of increase in TB death rate in India

23-Dec-15


Chaos Theory

Chaos

When the present determines the future,

but the approximate present does not approximately determine the future

23-Dec-15


Chaos Theory

Does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?

The butterfly does not power or directly create the tornado

The flapping of wings by the butterfly is a set of initial conditions which are followed by the tornado – the final result

Had the butterfly not flapped its wings, the trajectory of the system might have been vastly different

23-Dec-15

For want of a nail the shoe was lost.For want of a shoe the horse was lost.For want of a horse the rider was lost.

For want of a rider the message was lost.For want of a message the battle was lost.For want of a battle the kingdom was lost.And all for the want of a horseshoe nail.


Thank You

23-Dec-15

Documents

Data Analytics National Academy of Indian Railways Workshop on 'New Financial Initiatives' in Indian Railways For Principal/ Coordinating FA&CAOs 23 December