Upload
leonardo-martins
View
95
Download
1
Embed Size (px)
Citation preview
História
Type to enter text
Yahoo!!
Facebook!
Cloudera !
Hortonworks!
Walmart Brasil ;)
Leonardo Martins [email protected]
HDFS - Design
Type to enter text
Very large files Low-latency data access
Streaming data access! Lots of small files!
Commodity hardware Multiple writers, arbitrary file modifications!
Leonardo Martins [email protected]
HDFS - Arquitetura
Type to enter texthttp://www.hadoopwizard.com/which-big-data-company-has-the-worlds-biggest-hadoop-cluster/
HDFS - Sysadmin
Type to enter text
Adicionar e remover Data Node ao cluster!
Check de integridade do filesystem!
Balanceamento do cluster!
Adicionar e remover Task Tracker!
Monitorar os recursos do cluster!
Leonardo Martins [email protected]
MapReduce
Type to enter text
Desenvolvimento mais simples!
Escala em paralelo!
Distribuição automática de tarefas!
Tolerância a falhas!
Leonardo Martins [email protected]
MapReduce - Submit
Type to enter text
HDFS + MapReduce
Type to enter text
Task Tracker !
Job Tracker
Data Node !
Name Node
Possibilidades de uso
Type to enter text
Text mining Index building
Graph creation and analysis Pattern recognition
Prediction models Sentiment analysis
Cálculo de frete Cálculo de rota de caminhões!
Mecanismos de Antifraude
Leonardo Martins [email protected]
Quem usa?
Type to enter text