11

International Journal of Database Management Systems ( IJDMS ), Vol.3, No.2, May 2011 DOI: 10.5121/ijdms.2011.3206 89 S TATISTICAL D ATA A NALYSIS OF CONTINUOUS S TREAMS USING STREAM DSMS Nadeem Akhtar Department of Computer Engineering, Aligarh Muslim University, Aligarh, India [email protected] A BSTRACT Several applications involve a transient stream of data which has to be modeled and analyzed continuously. Their continuous arrival in multiple, rapid, time-varying and possibly unpredictable and unbounded way make the analysis difficult and opens fundamentally new research problems. Examples of such data intensive applications include stock market, road traffic analysis, whether forecasting systems etc. In this study, we have used a Data Stream Management System tool- Stanford STREAM to model and analyze data from two different application domains- Road Traffic analysis and Habitat Monitoring analysis. Based on the results we discuss advantages and disadvantages of STREAM. K EYWORDS Continuous Stream, DSMS, Stanford STREAM 1. INTRODUCTION The number of tools for analyzing continuous stream data is continuously increasing, because there is an increasing need in many different application domains. Several application domains require analysis of continuous data stream like stock market, security, telecommunications data management, web applications, manufacturing, sensor networks and others. One example of such application is Road Traffic Analysis. Several countries have been using RFIDs to collect real time vehicular information [1]. RFID readers read information from RFID tags attached to vehicles and sent to a centralized server. For a large city, number of vehicles passed through a particular point may go into millions. Moreover, vehicular traffic is intrinsically unpredictable and time-varying. Another example is Network Packet Analysis. Network operators and service providers need to monitor data traffic to analyze the provided service level, to identify bottlenecks, and initiate appropriate counter measures, if possible [2][3]. This is especially important in the Internet, because the amount of data traffic continuously increases and the behavior and requirements of end- users are changing over time, like accepted response time from web servers. Data stream management systems have been developed to monitor the continuously arriving data. They are different from traditional Database Management Systems in that they work on transient tables rather than persistent tables. Database Management Systems (DBMSs) may be used for analyzing continuous stream data. Traditional DBMSs, however suffer from some serious bottlenecks which limit their functionality from such real time complex applications requiring continuous monitoring of ever-changing data-streams. First, the DBMS is a passive repository storing a large collection of data elements and that humans initiate queries and transactions on

Statistical Data Analysis of Continuous Streams Using Stream DSMS

Download PDF Report

Upload
maurice-lee
View
220
Download
0

Embed Size (px)

Citation preview

Page 1: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 1/11

Page 2: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 2/11

Page 3: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 3/11

Page 4: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 4/11

Page 5: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 5/11

Page 6: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 6/11

Page 7: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 7/11

Page 8: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 8/11

Page 9: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 9/11

Page 10: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 10/11

Page 11: Statistical Data Analysis of Continuous Streams Using Stream DSMS

8/6/2019 Statistical Data Analysis of Continuous Streams Using Stream DSMS

http://slidepdf.com/reader/full/statistical-data-analysis-of-continuous-streams-using-stream-dsms 11/11

Control-based Quality Adaptation in Data Stream Management Systems (DSMS)

Control-based Quality Adaptation in Data Stream Management Systems (DSMS)

Documents

Models and Issues in Data Stream Systems (with changes by CZ · • Stream Models and DSMS Architectures • Query Processing • Runtime and Systems Issues • Algorithms • Conclusion

Models and Issues in Data Stream Systems (with changes by CZ · • Stream Models and DSMS Architectures • Query Processing • Runtime and Systems Issues • Algorithms • Conclusion

Documents

This will occur when streams receive more water than their channels can hold, i.e. when stream discharge is greater than stream capacity. Streams

This will occur when streams receive more water than their channels can hold, i.e. when stream discharge is greater than stream capacity. Streams

Documents

Streams 101: Stream Basics & Planning for Flood Resiliency

Streams 101: Stream Basics & Planning for Flood Resiliency

Government & Nonprofit

Data Stream Management Systems (DSMS) - Introduction, Concepts and Issues -

Data Stream Management Systems (DSMS) - Introduction, Concepts and Issues -

Documents

Knowledge Streams: Stream Processing of Semantic Web Content

Knowledge Streams: Stream Processing of Semantic Web Content

Documents

CS 3370. Inserters and Extractors Stream State Files Streams String Streams Formatting Manipulators Internationalization

CS 3370. Inserters and Extractors Stream State Files Streams String Streams Formatting Manipulators Internationalization

Documents

Tutorial Stream Reasoning SPARQLstream and Morph-streams

Tutorial Stream Reasoning SPARQLstream and Morph-streams

Technology

Tclips MPEG Test Streams Technical Reference · Reference Tclips MPEG Test Streams Technical Reference Test Streams Overview All stream files are Transport Stream format except for

Tclips MPEG Test Streams Technical Reference · Reference Tclips MPEG Test Streams Technical Reference Test Streams Overview All stream files are Transport Stream format except for

Documents

Java I/O. Agenda Introducing I/O. What Is a Stream? Network Streams Filter Streams Data Streams Reader and Writers Streams in Memory Object Serialization

Java I/O. Agenda Introducing I/O. What Is a Stream? Network Streams Filter Streams Data Streams Reader and Writers Streams in Memory Object Serialization

Documents

Lab 2 Data Streams. Lab 2: Data Streams Introduction Reading from an Input Stream Writing to an Output Stream Filter Streams

Lab 2 Data Streams. Lab 2: Data Streams Introduction Reading from an Input Stream Writing to an Output Stream Filter Streams

Documents

Handle Data Streams in DBS? Data Management: … · Handle Data Streams in DBS? Traditional DBS DSMS ... – Massive data volumes of transactions and measurements ... • Various

Handle Data Streams in DBS? Data Management: … · Handle Data Streams in DBS? Traditional DBS DSMS ... – Massive data volumes of transactions and measurements ... • Various

Documents

Asynchronous stream processing with Akka Streams

Asynchronous stream processing with Akka Streams

Engineering

Aerospace Stream Selection: Streams A, B, C

Aerospace Stream Selection: Streams A, B, C

Documents

Lecture Contents I/O Streams. –Input/output streams. –Unformatted vs formatted streams. –Stream manipulators. –Stream error state. –Stream tying. –Examples

Lecture Contents I/O Streams. –Input/output streams. –Unformatted vs formatted streams. –Stream manipulators. –Stream error state. –Stream tying. –Examples

Documents

Hydrological Cycle. Streams Stream = water flowing downhill in a defined channel. (River = large stream; Creek & Brook = small streams) Lotic system =

Hydrological Cycle. Streams Stream = water flowing downhill in a defined channel. (River = large stream; Creek & Brook = small streams) Lotic system =

Documents

Stream Processing Beyond Streaming Data with Apache Flink · Flink Runtime Stateful Computations over Data Streams Stateful Stream Processing Streams, State, Time Event-driven Applications

Stream Processing Beyond Streaming Data with Apache Flink · Flink Runtime Stateful Computations over Data Streams Stateful Stream Processing Streams, State, Time Event-driven Applications

Documents

Data Streams & Continuous Queries The Stanford STREAM Project stanfordstreamdatamanager

Data Streams & Continuous Queries The Stanford STREAM Project stanfordstreamdatamanager

Documents

2017 Stream Health Assessment for City of Bothell Streams

2017 Stream Health Assessment for City of Bothell Streams

Documents

The C++ Programming Language Streams. Contents u Output Stream u Input Stream u Formatting u Manipulators u Files & Streams

The C++ Programming Language Streams. Contents u Output Stream u Input Stream u Formatting u Manipulators u Files & Streams

Documents

Extending DSMS for Data Stream Mining

Extending DSMS for Data Stream Mining

Documents

Parallel Processing - Basavaraj TalawarFlynn's Classification Single instruction stream, single data stream (SISD) – Uniprocessor. Single instruction stream, multiple data streams

Parallel Processing - Basavaraj TalawarFlynn's Classification Single instruction stream, single data stream (SISD) – Uniprocessor. Single instruction stream, multiple data streams

Documents

Lecture 12 Running Water and Streams 1.Hydrologic cycle 2.Stream hydraulics 3.Stream erosion 4.Transportation of sediment by streams 5.Deposition of sediment

Lecture 12 Running Water and Streams 1.Hydrologic cycle 2.Stream hydraulics 3.Stream erosion 4.Transportation of sediment by streams 5.Deposition of sediment

Documents

Rivers and Streams - ruby.colorado.eduruby.colorado.edu/~smyth/G1010/18Streams.pdf · Rivers and Streams Chapter 18 Streams: Outline ... – A braided stream has many channels bars

Rivers and Streams - ruby.colorado.eduruby.colorado.edu/~smyth/G1010/18Streams.pdf · Rivers and Streams Chapter 18 Streams: Outline ... – A braided stream has many channels bars

Documents

Tagging Stream Data for Rich Real-Time Servicesresources.mpi-inf.mpg.de/departments/d5/teaching/... · Data Stream Management System (DSMS): The database for managing continuous data

Tagging Stream Data for Rich Real-Time Servicesresources.mpi-inf.mpg.de/departments/d5/teaching/... · Data Stream Management System (DSMS): The database for managing continuous data

Documents

Data Stream Management System for Moving Sensor … · · 2015-03-26Data Stream Management System for Moving Sensor Object Data ... DSMS applications should be Web based. ... The

Data Stream Management System for Moving Sensor … · · 2015-03-26Data Stream Management System for Moving Sensor Object Data ... DSMS applications should be Web based. ... The

Documents

DSMS - Community

DSMS - Community

Documents

EVENT STREAM PROCESSING USING KAFKA STREAMS...github.com/fredriv/kafka-streams-workshop DATA-DRIVEN APPLICATIONS

EVENT STREAM PROCESSING USING KAFKA STREAMS...github.com/fredriv/kafka-streams-workshop DATA-DRIVEN APPLICATIONS

Documents

STREAMS ATTENDANCE - Digital Nations · registered for live stream broadcast 190 average minutes watched live stream broadcast digitalnations.co.nz LIVE STREAMING STREAMS ATTENDANCE

STREAMS ATTENDANCE - Digital Nations · registered for live stream broadcast 190 average minutes watched live stream broadcast digitalnations.co.nz LIVE STREAMING STREAMS ATTENDANCE

Documents

Concepts in Stream Ecology Streams are ecosystems

Concepts in Stream Ecology Streams are ecosystems

Documents