View
232
Download
12
Category
Tags:
Preview:
Citation preview
ASEMINAR ON
BIG DATA
PRESENTED BY:-VIKAS KATAREM.TECH(I.T.)
EMail: vikashsharmamy@gmail.comcell no.+917031120786
WHAT IS DATA
• The data is binary sequence with weighing factor.
• Information of any thing is consider as data.
• Data is distinct pieces of information , usually formatted in a special way.
Big Data Definition
• No single standard definition…
“Big Data” is data whose scale, diversity, and
complexity require new architecture, techniques,
algorithms, and analytics to manage it and extract
value and hidden knowledge from it…
3 V’S OF BIG DATA
Lots of Data
• 2.5 quintillion bytes of data are generated every day!– A quintillion is 1018
• Data come from many quarters.– Social media sites
– Sensors
– Digital photos
– Business transactions
– Location-based data
Who’s Generating Big Data
Social media and networks(all of us are generating data)
Scientific instruments(collecting all sorts of data)
Mobile devices (tracking all objects all the time)
Sensor technology and networks(measuring all kinds of data)
6
Challenges
How to transfer Big Data?
• Storage & Transport issue
• Data management issue
• Processing issue
• Privacy & security
• Data access and sharing information
• Fault tolerence
9
Past Big Data Solutions
• Data Shard’ing
– Is a “shared nothing” partitioning scheme for large databases across a number of servers increasing scalability of performance of traditional relational database systems. Essentially, you are breaking your database down into smaller chunks called “shards” and spreading them across a number of distributed servers. The advantages of Sharding is as follows:
• Easier to manage
• Faster
• Reduce Costs
BIG DATA ANALYTICS
• Examining large amount of data
• Appropriate information
• Identification of hidden patterns unknown correlations
• Competitive advantages
Types of Tools Typically Used in Big Data Scenario
• Where is the processing hosted?
– Distributed server/cloud
• Where data is stored?
– Distributed Storage (eg: Amazon s3)
• Where is the programming model?
– Distributed processing (Map Reduce)
• What operations are performed on the data?
– Analytic/Semantic Processing (Eg. RDF)
12
Big Data Solutions
• SANS– SANS are essentially dedicated, high performance storage networks that transfer
data between servers and storage devices, separate from the Local Area Network (usually through fiber channels).
– ADVANTAGES
• Ability to move large blocks of data
• High level of performance and availability
• Dynamically balances loads across the network.
– DISADVANTAGES
• Complex to manage a wide scope of devices
• Lack of Standardization
• SANs are very expensive11
RDF
• (RESOURCE DESCRIPTOR FRAMEWORK)
• Why is RDF uniquely suited to expressing data and data relationships?
• More flexible – data relationships can be explored from all angles
• More efficient – large scale, data can be read more quickly
– not linear like a traditional database
– not hierarchical like XML
HADOOP
Software platform that lets one easily write and run applications that process vast
amounts of data. It includes:
– Map Reduce – offline computing engine
– HDFS – Hadoop distributed file system
– HBase (pre-alpha) – online data access
– Scalable: It can reliably store and process petabytes.
– Economical: It distributes the data and processing across clusters of commonly
available computers (in thousands).
– Efficient: By distributing the data, it can process it in parallel on the nodes
where the data is located.
– Reliable: It automatically maintains multiple copies of data and automatically
redeploys computing tasks based on failures.
MAP REDUCE
• Parallel programming model meant for large clusters
– User implements Map() and Reduce()
• Parallel computing framework
– Libraries take care of EVERYTHING else• Parallelization
• Fault Tolerance
• Data Distribution
• Load Balancing
• Useful model for many practical tasks (large data)
Map+Reduce
• Map:– Accepts input key/value
pair
– Emits intermediate key/value pair
• Reduce :– Accepts intermediate
key/value* pair
– Emits output key/value pair
Very big
data
ResultMAP
REDUCE
PartitioningFunction
Finally….
‘Big- Data’ is similar to ‘Small-data’ but bigger
.. But having data bigger it requires different approaches:
Techniques, tools, architecture
… with an aim to solve new problems
Or old problems in a better way
12
THANKING YOU
REFRENCES
• www.wikipedia.com
• www.slideshare.com
• www.powershow.com
• www.lv-aitp.org/2012-2013%20Programs/Big%20Data.ppsx
Recommended