Insight Analysis on Big Data1

Peng Lai

Ling Zheng

Analysis on News Stories about “Big Data” in Popular Press

（ 2010-2015）

1. For years, Big Data has been applied in fields including health care, technology, advertising, banks and even politics. Topics around big data has been changing over time since 2010

Topics overview from 2010 to 2015

Big data swept through academia, business and government.

China and India grow fast in big data market.

Talents for big data are rare, especially in Java field.

Legal issues including privacy and security are concerned by government.

TopicsCloud, Storage, Solution, Enterprise, ApplicationBank, Marketing, Buy, Campaign, SalePatient, Health, Care, Medical, DiseaseJob, Student, Skill, JavaChina, City, India, Growth, GovernmentPrivacy, Law, Government, Protection, Legal

2. ‘Cloud’ occurs in all the years from 2010 to 2015. This shows that cloud computing is a core technology for the development of big data.

2010

2011

2012 2013

2014 2015

Cloud, Storage, Solution, Enterprise, Hadoop

Bank, Credit, Loan, Financial, Buy

Patient, Health, Care, Medical, Clinical

China, City, Government, Chinese, Province

Greenberg, Trump, Republicans, Presidential

Student, Science, University, Skill, Job

Campaign, Voter, Election, Political, Media

Job, Status, Experience, Java, Applicant

Solution, Could, Analytics, Storage, IBM

Patient, Health, Medical, Care, Treatment

Privacy, Law, Government, Protection, White House

China, City, Billion, Investor, Growth

Cloud, Storage, Application, Hadoop, Server

Surveillance, Government, Security, Law, Privacy

Patient, Health, Disease, Care, Medical

India, Cent, Growth, Revenue, Investment

Student, Science, University, Skill, Job

Ad, Consumer, Advertising, Marketing, Google

Cloud, Server, Network, Switch, Ethernet

Investor, IPO, Valley, Proofpoint

Risk, Insurer, Organisation, Analytics, Hadoop

Scientist, Science, Data Scientist, Researcher, University

Voter, Campaign, Political, Obama, Privacy

India, Cloud, Mobility, Skill, Wipro

Management, Oracle, Application, Organisation, Database

Amazon, Cloud, Netflix, Computing, Backup

Age, Manyika, Privacy, Technological, Thought

Email, Card, Breach, Law, Credit

Enterprise Software, Venture, Investor, Capital, Produce

Teradata, Egypt, Active, Warehouse

Challenge, Analytics Amounts, Technology, Value Real

Cloud ‘Could Computing’, Open Cloudera, Computing Built

3. From 2011 to 2012, terms like venture and investor appeared frequently indicating that big data dragged investors’ attention in the early stage

2010

2011

2012 2013

2014 2015

Cloud, Storage, Solution, Enterprise, Hadoop

Bank, Credit, Loan, Financial, Buy

Patient, Health, Care, Medical, Clinical

China, City, Government, Chinese, Province

Greenberg, Trump, Republicans, Presidential

Student, Science, University, Skill, Job

Campaign, Voter, Election, Political, Media

Job, Status, Experience, Java, Applicant

Solution, Could, Analytics, Storage, IBM

Patient, Health, Medical, Care, Treatment

Privacy, Law, Government, Protection, White House

China, City, Billion, Investor, Growth

Cloud, Storage, Application, Hadoop, Server

Surveillance, Government, Security, Law, Privacy

Patient, Health, Disease, Care, Medical

India, Cent, Growth, Revenue, Investment

Student, Science, University, Skill, Job

Ad, Consumer, Advertising, Marketing, Google

Cloud, Server, Network, Switch, Ethernet

Investor, IPO, Valley, Proofpoint

Risk, Insurer, Organisation, Analytics, Hadoop

Scientist, Science, Data Scientist, Researcher, University

Voter, Campaign, Political, Obama, Privacy

India, Cloud, Mobility, Skill, Wipro

Management, Oracle, Application, Organisation, Database

Amazon, Cloud, Netflix, Computing, Backup

Age, Manyika, Privacy, Technological, Thought

Email, Card, Breach, Law, Credit

Enterprise Software, Venture, Investor, Capital, Produce

Teradata, Egypt, Active, Warehouse

Challenge, Analytics Amounts, Technology, Value Real

Cloud ‘Could Computing’, Open Cloudera, Computing Built

4. Privacy issues in big data field arose in early 2011 and related laws were discussed afterwards

2010

2011

2012 2013

2014 2015

Cloud, Storage, Solution, Enterprise, Hadoop

Bank, Credit, Loan, Financial, Buy

Patient, Health, Care, Medical, Clinical

China, City, Government, Chinese, Province

Greenberg, Trump, Republicans, Presidential

Student, Science, University, Skill, Job

Campaign, Voter, Election, Political, Media

Job, Status, Experience, Java, Applicant

Solution, Could, Analytics, Storage, IBM

Patient, Health, Medical, Care, Treatment

Privacy, Law, Government, Protection, White House

China, City, Billion, Investor, Growth

Cloud, Storage, Application, Hadoop, Server

Surveillance, Government, Security, Law, Privacy

Patient, Health, Disease, Care, Medical

India, Cent, Growth, Revenue, Investment

Student, Science, University, Skill, Job

Ad, Consumer, Advertising, Marketing, Google

Cloud, Server, Network, Switch, Ethernet

Investor, IPO, Valley, Proofpoint

Risk, Insurer, Organisation, Analytics, Hadoop

Scientist, Science, Data Scientist, Researcher, University

Voter, Campaign, Political, Obama, Privacy

India, Cloud, Mobility, Skill, Wipro

Management, Oracle, Application, Organisation, Database

Amazon, Cloud, Netflix, Computing, Backup

Age, Manyika, Privacy, Technological, Thought

Email, Card, Breach, Law, Credit

Enterprise Software, Venture, Investor, Capital, Produce

Teradata, Egypt, Active, Warehouse

Challenge, Analytics Amounts, Technology, Value Real

Cloud ‘Could Computing’, Open Cloudera, Computing Built

5. Talents in big data became a focus since 2012. Academic institutions, such as universities, pay efforts to offer students big data skills to satisfy the increasing big data job market.

2010

2011

2012 2013

2014 2015

Cloud, Storage, Solution, Enterprise, Hadoop

Bank, Credit, Loan, Financial, Buy

Patient, Health, Care, Medical, Clinical

China, City, Government, Chinese, Province

Greenberg, Trump, Republicans, Presidential

Student, Science, University, Skill, Job

Campaign, Voter, Election, Political, Media

Job, Status, Experience, Java, Applicant

Solution, Could, Analytics, Storage, IBM

Patient, Health, Medical, Care, Treatment

Privacy, Law, Government, Protection, White House

China, City, Billion, Investor, Growth

Cloud, Storage, Application, Hadoop, Server

Surveillance, Government, Security, Law, Privacy

Patient, Health, Disease, Care, Medical

India, Cent, Growth, Revenue, Investment

Student, Science, University, Skill, Job

Ad, Consumer, Advertising, Marketing, Google

Cloud, Server, Network, Switch, Ethernet

Investor, IPO, Valley, Proofpoint

Risk, Insurer, Organisation, Analytics, Hadoop

Scientist, Science, Data Scientist, Researcher, University

Voter, Campaign, Political, Obama, Privacy

India, Cloud, Mobility, Skill, Wipro

Management, Oracle, Application, Organisation, Database

Amazon, Cloud, Netflix, Computing, Backup

Age, Manyika, Privacy, Technological, Thought

Email, Card, Breach, Law, Credit

Enterprise Software, Venture, Investor, Capital, Produce

Teradata, Egypt, Active, Warehouse

Challenge, Analytics Amounts, Technology, Value Real

Cloud ‘Could Computing’, Open Cloudera, Computing Built

6. Health care industry is one of first industries to apply big data

2010

2011

2012 2013

2014 2015

Cloud, Storage, Solution, Enterprise, Hadoop

Bank, Credit, Loan, Financial, Buy

Patient, Health, Care, Medical, Clinical

China, City, Government, Chinese, Province

Greenberg, Trump, Republicans, Presidential

Student, Science, University, Skill, Job

Campaign, Voter, Election, Political, Media

Job, Status, Experience, Java, Applicant

Solution, Could, Analytics, Storage, IBM

Patient, Health, Medical, Care, Treatment

Privacy, Law, Government, Protection, White House

China, City, Billion, Investor, Growth

Cloud, Storage, Application, Hadoop, Server

Surveillance, Government, Security, Law, Privacy

Patient, Health, Disease, Care, Medical

India, Cent, Growth, Revenue, Investment

Student, Science, University, Skill, Job

Ad, Consumer, Advertising, Marketing, Google

Cloud, Server, Network, Switch, Ethernet

Investor, IPO, Valley, Proofpoint

Risk, Insurer, Organisation, Analytics, Hadoop

Scientist, Science, Data Scientist, Researcher, University

Voter, Campaign, Political, Obama, Privacy

India, Cloud, Mobility, Skill, Wipro

Management, Oracle, Application, Organisation, Database

Amazon, Cloud, Netflix, Computing, Backup

Age, Manyika, Privacy, Technological, Thought

Email, Card, Breach, Law, Credit

Enterprise Software, Venture, Investor, Capital, Produce

Teradata, Egypt, Active, Warehouse

Challenge, Analytics Amounts, Technology, Value Real

Cloud ‘Could Computing’, Open Cloudera, Computing Built

7. Both India and China have made rapid growth in IT field recent years. It can be seen in big data as well.

2010

2011

2012 2013

2014 2015

Cloud, Storage, Solution, Enterprise, Hadoop

Bank, Credit, Loan, Financial, Buy

Patient, Health, Care, Medical, Clinical

China, City, Government, Chinese, Province

Greenberg, Trump, Republicans, Presidential

Student, Science, University, Skill, Job

Campaign, Voter, Election, Political, Media

Job, Status, Experience, Java, Applicant

Solution, Could, Analytics, Storage, IBM

Patient, Health, Medical, Care, Treatment

Privacy, Law, Government, Protection, White House

China, City, Billion, Investor, Growth

Cloud, Storage, Application, Hadoop, Server

Surveillance, Government, Security, Law, Privacy

Patient, Health, Disease, Care, Medical

India, Cent, Growth, Revenue, Investment

Student, Science, University, Skill, Job

Ad, Consumer, Advertising, Marketing, Google

Cloud, Server, Network, Switch, Ethernet

Investor, IPO, Valley, Proofpoint

Risk, Insurer, Organisation, Analytics, Hadoop

Scientist, Science, Data Scientist, Researcher, University

Voter, Campaign, Political, Obama, Privacy

India, Cloud, Mobility, Skill, Wipro

Management, Oracle, Application, Organisation, Database

Amazon, Cloud, Netflix, Computing, Backup

Age, Manyika, Privacy, Technological, Thought

Email, Card, Breach, Law, Credit

Enterprise Software, Venture, Investor, Capital, Produce

Teradata, Egypt, Active, Warehouse

Challenge, Analytics Amounts, Technology, Value Real

Cloud ‘Could Computing’, Open Cloudera, Computing Built

8. Big data works in politics since 2012 which is intrigued by Obama’s successful political campaigns by implementing and using big data. Trump became a big word in big data In 2015.

2010

2011

2012 2013

2014 2015

Cloud, Storage, Solution, Enterprise, Hadoop

Bank, Credit, Loan, Financial, Buy

Patient, Health, Care, Medical, Clinical

China, City, Government, Chinese, Province

Greenberg, Trump, Republicans, Presidential

Student, Science, University, Skill, Job

Campaign, Voter, Election, Political, Media

Job, Status, Experience, Java, Applicant

Solution, Could, Analytics, Storage, IBM

Patient, Health, Medical, Care, Treatment

Privacy, Law, Government, Protection, White House

China, City, Billion, Investor, Growth

Cloud, Storage, Application, Hadoop, Server

Surveillance, Government, Security, Law, Privacy

Patient, Health, Disease, Care, Medical

India, Cent, Growth, Revenue, Investment

Student, Science, University, Skill, Job

Ad, Consumer, Advertising, Marketing, Google

Cloud, Server, Network, Switch, Ethernet

Investor, IPO, Valley, Proofpoint

Risk, Insurer, Organisation, Analytics, Hadoop

Scientist, Science, Data Scientist, Researcher, University

Voter, Campaign, Political, Obama, Privacy

India, Cloud, Mobility, Skill, Wipro

Management, Oracle, Application, Organisation, Database

Amazon, Cloud, Netflix, Computing, Backup

Age, Manyika, Privacy, Technological, Thought

Email, Card, Breach, Law, Credit

Enterprise Software, Venture, Investor, Capital, Produce

Teradata, Egypt, Active, Warehouse

Challenge, Analytics Amounts, Technology, Value Real

Cloud ‘Could Computing’, Open Cloudera, Computing Built

9. Hot topics of big data “technology” have been changed from software, cloud to internet of things

Technical Review

Prepare Data: split data sets to year by year

Parsing and Filtering 1:Set Synonyms, Ignore of Entities,

Stop List

Text Mining:Compare Text Cluster and

Text Topics Results

Technical Review

Text Filtering 2:Drop Words that are not

Informative based on previous results

Check Text Mining Results:Check words in Filter Viewer

Discover Concept Links

Technical Review

High frequency words comparison &

percentage of occurrence for certain term comparison

through time

Insights Building