OSCON 2010 - Panning for Gold: A Web Prospector's Guide to Mining and Filtering Community Data

Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC

Welcome to“Panning for Gold”

A “Web” Prospector’s Guide to Mining and Filtering Community Data


Introductions

Ben Bassi, President & CEOSeasoned Internet Pioneer

At the forefront of online innovation

for almost two decades…

Bassi was a part of the founding management team of Lycos.com responsible for creating many of the first communities and portals for companies like Microsoft, CompuServe, Prodigy, MTV and GTE.

EVP of the Firefly Network (acquired by Microsoft asthe basis for the Microsoft Passport for MSN).

CommonPlaces e-Solutions

• Founded in 1998 in Cambridge, MA• Open Source Development since 2003• We create rewarding and intelligent

experiences for online communities…

– CMS Powered Online Communities

– Subscription-Based e-Commerce Applications

– CMS Integration (single sign on, SFA, back-office)

– UG Media / Multimedia Sites

– Advanced Search / Data Collaboration


Let’s Get Started!

“Panning for Gold”

A “Web” Prospector’s Guide to Mining and Filtering Community Data


Panning for Gold

There are 4 Steps to Mining For Gold(and as it turns out, for mining for useful

data from Web and Internal Data)

1. Prospecting 2. Mining 3. Extracting 4. Refining


1. Prospecting

You Know There's Nuggets of Info Out There Businesses today have access to an unimaginable amount of user data andcontent, both structured and unstructured.

Social Media Support DeskCRMEmailsDatabases

UG and Product Reviews

Blogging / Micro-Blogging


Traditional Prospecting Methods

• Relying on What You Know Already

– “Like relying on outdated maps”, predetermined keywords, product names, and preconceptions

• Not the best way to discover new ideas

• Best ROI is found on “Discovered” intelligence

• No Competitive Edge


Plenty of Commercial Search Tools

• Automate the Search Process • Provide Sentiment Data • Expensive • You have to know what you’re searching for first


Plenty of Free, Open Source Applications

• Automate the Search Process • Tell You Who Your Influencers Are• You have to know what you’re searching for first


These Tools Still Aren’t “Discovery” Tools

The frustration is that most can’t quite reach or understand the value of the data with the search tools they’re currently using.

• You’re Still Searching For What You Already Know

• Like searching for treasure on the beach – the ‘beep’ (or information returned in a report) could be Gold, or a nickel. You have no way of aggregating the data across platforms to know what’s really useful – so they’re really “blind” searches.

• A lot of work for very small ROI.


2. Mining

vs.

Traditional Search Data Discovery


3. Extracting

Social Media

Support DeskCRMEmailsDatabases

Blogging / Micro-BloggingUG and Product Reviews


4. Refining

• Here’s where the available Search tools fall down

• What can you do with the data once its collected?

• How to keep Context

• How automated is the process (million of spreadsheets? Interns?)


CP’s Data Discovery Technology Delivers

Timely actionable information discovery for impactfulbusiness results

• Detailed Voice of Customer insights

• Market intelligence to stay ahead of competitors

• Develop new strategies for products and services for revenue growth

• Improve customer satisfaction to increase loyalty and up-sell


What Clients Tell Us They Need

Timely Results• Quickly see what is in the information• Guide “me” to what I should look at - Go beyond traditional search listings• Relevant and interesting information

For their users:

Real Information• Customer/user verbatim: Can I see what they are actually saying? • See citations from documents• Show me how information is connected

Help “Me”• Address competitive realities and

strategies• Identify new product/process

opportunities• Identify operational

improvements• Quickly find emerging trends for

opportunities/risks on the long tails of information

• Maximize the use of my information

For their business:

Discover relevant information in a timely basis – uncover actionable insights.

Empower “Me”• Enable “me” to do my own exploration• Twist, spindle, and manipulate “my” data at will and see how it actively responds• See clusters, long tail info, and trends


Why is it Different?Search & Data Mining Exploration & Discovery

Howit

Works

Uses search tools where users may key in words or data for which they are interested in matching results.

Creates free-form interactive visual displays of information and its characteristics, connections and details.

Attributesof the

Approach

• Requires that the user decide what they want to find before they find it.

• Provides no means of narrowing a large set of potentially irrelevant results.

• People develop understanding of large bodies of information before reading in detail.

• Allows people to narrow in to the most important relevant results.

• Requires no presupposition of the outcome prior to application.

Discovery Limited to database exposure Utilizes various data inputs- internal, internet, social media, consumer generated content, subscription services all combined for data discovery

Result People find only what they decided they wanted to find ahead of time, minimal surprises or novel information.

People discover the unexpected. New and emerging ideas are discovered without knowing how to search for them in advance.

Sample Data Sources

Blogs, Polls, Surveys

Call Center Data

Data Discovery Utilizes Both Structured and Unstructured Data

Why is Discovery Technology Different?

Take Data “As-Is” with no need to

first clean source files

No Pre-Defined Dictionary Required

No Need To Know What is In Your Data

Before Beginning Exploration

No Need To Know Data Relationships –

Technology Identifies Them for You Real

Time

Works on the“Long Tail” of Information

Enabling Detection of Emerging

Opportunities and Risks

Solution Enables “Lenses” Which are Reusable

Knowledge Digital Assets That Can be Utilized Throughout the Organization For Discovery in

other Fields of Interest

The Connection Engine Platform that Enables Real Time Data Exploration and

Discovery

Sample Applications

Regulatory Analysis

Operational Analysis

Strategic & Competitive Intelligence

Market Research

Call Center Operations

Warranty Claims

Product Development

Litigation Support

E -Discovery

Patent Research

Pricing and Packaging

Claims Processing

SurveysWEB/BLOG

Forum

Subscribed Content

Customer/UserProfiling

Social Media Exploration

Voice of Customer

Fraud RelatedDiscovery


Prospecting with CP Data Discovery• Interactive Exploration- no programming for

user…drag and drop • Pre-defined dictionary not required • Cluster & Long Tail Exploration for Advanced

Early Detection• Ability to utilize anywhere internet access

availability exists…..mobile field real time applications

• Can set up threshold and alerting system for automated processing of information


Mining/Extracting with CP Data Discovery

• Information Connection Discovery• Multiple Simultaneous Data Sets • Can be utilized as an overlay enhancement

to existing programs• Structured and Unstructured Data

Correlation• Social Media Exploration • Multi-language capabilities


Refining with CP Data Discovery

• Structured and Unstructured Data Correlation• Development of exception reporting based on

results of exploration• Word Proximity and Phrase Analysis • Information Connection Discovery• Trend Visualization • Lens Processing - user created knowledge

assets to share, repurpose and drive new discoveries


Data Discovery Demo


Thank You! Questions?

CommonPlaces e-Solutions Follow us on LinkedIn

Ben BassiPresident & CEO

[email protected]@BenBassi

Michael ReichDirector of Business Development

[email protected]

@CPOpenSource Follow us on Twitter

www.CommonPlaces.com603.329.6760