Wolphram Alpha

Wolfram AlphaAn introduction to the underlying technology

SIGC 2010/2011

Pedro Gaspar

Outline

IntroductionHistoryTechnology – The “Four Pillars”Technology – Interesting FactsConclusionsReference

Wolfram Alpha - Pedro Gaspar 2

IntroductionReal-time computational answering

systemNot a Search Engine like GoogleNot as static as Wikipedia or as an

Encyclopedia


IntroductionGoal:

Systematic knowledge:◦Objective Data◦Models◦Methods◦Algorithms◦Formulae


“Wolfram|Alpha's long-term goal is to make all systematic knowledge immediately computable and accessible to everyone.”

IntroductionSome of the explored areas:


MathematicsStatistics & Data AnalysisPhysicsChemistryMaterialsEngineeringAstronomyEarth SciencesLife SciencesComputational Sciences

Units & MeasuresDates & TimesWeatherPlaces & GeographyPeople & HistoryCulture & MediaMusicWords & LinguisticsSports & GamesColors

Money & FinanceSocioeconomic DataHealth & MedicineFood & NutritionEducationOrganizationsTransportationTechnological WorldWeb & Computer Systems

HISTORY


How did the project start?

History – Wolfram Alpha

Project lead by Stephen Wolfram

It is the culmination of 5 years of work, and 25 more years of previous development

Stephen started Wolfram Research in 1987, focusing mainly on the Mathematica software


History – Wolfram Alpha

In 2002 Stephen publishes “A New Kind of Science”

In 2004 the company tries to apply the concepts from the book to a real-world product and thus started developing Wolfram Alpha

In May 18th, 2009 Wolfram Alpha is officially launched to the public


History – Computable KnowledgeThe history of Systematic Data and the

Development of Computable Knowledge goes back to the 20,000 BC with the invention of arithmetic

Scientific Books, Encyclopedias, Census, Maps and other sources of information have been collecting data since Ancient Mesopotamia


http://www.wolframdatasummit.org/2010/timeline.html

TECHNOLOGY


How does it work?

Technology – the “Four Pillars”

Curation Formalization NLP Visualizati

on


Pillar1 - Curation

Field Experts help the team find the best content sources and validate the data

Community input is also accepted, but all the data has to go through a rigorous validation process before being used

Almost none of their data comes from the Internet now

It turned out that curation and data gathering was only 5% of the work


Pillar1 - Curation


Pillar 2 - FormalizationOrganizing the curated data so that it can be

computable

Figuring out its conventions, units, definitions and how it connects to other data

All these are encoded algorithmically in Wolfram Alpha so that they’re available when needed

All the algorithms, models and equations are encoded into functions in Mathematica, the programming language behind Wolfram Alpha


Pillar 2 - Formalization

Mathematica’s language is able to represent data of all kinds using arbitrarily structured symbolic expressions

As a result, the code is much more compact than in a lower-level language like Java or Python

Mathematica already includes a very big set of algorithms and functions, making it easier to implement new (usually more complex) algorithms



This creates a recursive process, that makes implementing new algorithms easier and easier through software reutilization






Pillar 3 – Natural Language ProcessingHow could users interact with the system and

use its computing powers? Through human language is the most natural response

The problem is not the one we are used to – instead of trying to make sense of a big set of words, the system has to map small pieces of human input (queries) into its large set of symbolic representations

The implemented solutions generally achieve good results


Pillar 3 – Natural Language Processing








Pillar 4 – Visualization

Wolfram Alpha’s ability to present results in formats other than text is one of its most visually appealing features

Mathematica includes some functionality to deal with this challenge, through what they call “computational aesthetics”

This automates, for a specific symbolic representation, what to present and how to present it


















Technology – Interesting Facts


More than 10 trillion of dataMore than 50,000 types of algorithms

and modelsLinguistic capacity for more than 1000

domainsMore than 8 million lines of symbolic

Mathematica codeRuns in clusters of supercomputers,

including the 44th largest supercomputer in the world - R Smarr

Hundreds of terabytes of storage

ConclusionsIt is all a matter of representing data

and mapping queries to the set of things they can compute about

Uses an internal and pre-structured database to find the answers to the queries

Computation brings a lot of value when comparing it to search engines like Google

Little to no information available about how the system works internally


ReferencesWolfram Alpha's websiteWolfram Alpha's blogThe Story of the Making of Wolfram AlphaOpinion: Wolfram Alpha: How does it work

?How the hell does Wolfram Alpha WorkWolfram Alpha ArchitectureWolfram Data Summit 2010Wolfram Alpha's YouTube channelWhat is Mathematica?


http://wolframalpha.com/

http://blog.wolframalpha.com/

http://www.stephenwolfram.com/publications/recent/50yearspc/

http://www.stephenwolfram.com/publications/recent/50yearspc/

http://www.computerweekly.com/Articles/2009/05/21/236131/Opinion-Wolfram-Alpha-How-does-it-work.htm

http://www.computerweekly.com/Articles/2009/05/21/236131/Opinion-Wolfram-Alpha-How-does-it-work.htm

http://www.techradar.com/news/internet/how-the-hell-does-wolfram-alpha-work--599561

http://highscalability.com/wolfram-alpha-architecture

http://www.wolframdatasummit.org/2010/

http://www.youtube.com/user/WolframAlpha

http://documents.wolfram.com/v2book/contents/whatis.pdf



QUESTIONS?


[email protected]

Pedro Gaspar