CrowfordCrowd Investment Data Portal
Group 13 LABBE, Kevin Patrick Joseph
MARTYNAVA, Karina THOMPSON, Julien Edward
Topics
• Crowdfunding basics
• Schema Mapping / ER (Gathering Data)
• Data Fusion (Data Analysis)
• Data Portal
Crowdfunding
! Crowdfunding
! Schema mapping / ER
! Data fusion
! Data Portal
Crowdfunding• Fund a project by a large number of people
• Start-up, Video games, charity…
! Crowdfunding
! Schema mapping / ER
! Data fusion
! Data Portal
Crowdfunding
• over 2 billions $
• 100K projects
• 10 milion contributors
! Crowdfunding
! Schema mapping / ER
! Data fusion
! Data Portal
Crowford• Gather projects from different sources
• Predict if a project will be successful or not
! Crowdfunding
! Schema mapping / ER
! Data fusion
! Data Portal
Schema Mapping / ER
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
Gathering data
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data PortalData Sources
• Same structure
• Same theme (fund projects)
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data PortalData Sources
Crowdfund data
Project
Idea (Pen, Video game, Product…)
Packages / Perks What you get / Money
Author(s)
Crawling! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
Crawling
• Use JavaScript to generate project page
• Private API that generate JSON
• Python script w/ http2 that generate requests
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
https://www.indiegogo.com/private_api/explore?filter_funding=&filter_percent_funded=&filter_q
uick=new&filter_status=&pg_num=2
Crawling
• Have to extract data from websites
• 2 Spider bots (crawlers) using Scrapy
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
Crawling w/ Scrapy
• Python Framework for extracting data
• Write Spider (crawling bots)
• Parse data and extract with xpath
• Export data (Schema mapping)
• Initialize the spider
• Download and extract data
• Export the item
• Export the item
Data scheme
Author
Project Perks / Packages
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
Project
Author
Project Summary Perks / Packages
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data PortalData scheme
Data scheme
• Project / Author
• Project / Perks
• Recommendation : Project / Related_Project
• ER for multiple authors / perks / projects…
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
Results
• Our working set
• 36 000 projects
• 65 000 authors
• over 230 000 perks
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
Data fusion
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
And other data analysis
Recommendation• Use buzzwords in project description
• Use n-grams (word combination)
• Similarity measures using Pairwise metrics
• Linear kernels
• Can be used for data fusion
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
Success Prediction
• How much money has been collected
• How much time
• The average pledge
• …
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
Success PredictionRandom Forests Logistic regression
RESULT 1 RESULT 2
COMPARISON
RESULT
! Crowdfunding
! Schema mapping / ER
! Data Fusion
! Data Portal
Data Portal
! Crowdfunding
! Schema mapping / ER
! Data fusion
! Data Portal
Goal
Browse successful projects
! Crowdfunding
! Schema mapping
! Data fusion
! Data Portal
DataBase
PostgreSQL
DataBase
PostgreSQL
Web Interface
Django
DataBase
PostgreSQL
Web Interface
Django
Project List Project Info
Web Interface
• Project list
• Filter
• Access project page
• Allows you to download the datasets
! Crowdfunding
! Schema mapping
! Data fusion
! Data Portal
Demo