Embed Size (px)
DESCRIPTION
A lightning talk on Gearman
Citation preview
Gearman: a technology for distributed
computingGiuseppe Maxia
MySQL Community Team Lead Sun Microsystems
Wednesday, 07 October 2009
This is a quick talk that introduces technologies in the age of cloud computing
about me -Giuseppe Maxia a.k.a. The Data Charmer MySQL Community Team Lead Long time hacking with MySQL features Formerly, database consultant, designer, coder. A passion for QA An even greater passion for open source ... and community Passionate blogger
http://datacharmer.blogspot.com Wednesday, 07 October 2009
Mainframe
Mainframe
operating system
hardware
application
terminal
client
USER FRIENDLINESS
0 100
terminal
terminal
terminal
Wednesday, 07 October 2009
We start with a big of history of computing.In the age of mainframes, users could only choose the whole system, or nothing. Hardware, operating system, and software came from the same company. The only priority was towards getting the job done with the minimal expenses. Users needs were negligible
Mini computers
Mainframe
operating system
hardware
application
client
USER FRIENDLINESS
0 100
Mini terminal
terminal
terminal
terminal
Mini
Wednesday, 07 October 2009
Minicomputers added some level of flexibility to the user's experience, although the information system was still (with little exceptions) a matter of choosing a company. The user experience improved a bit, but only for very power users.
Networks
server
hardware
application
client
USER FRIENDLINESS
0 100
personal computer
personal computer
personal computer
personal computer
hardware
operating system
operating system
hardware
Wednesday, 07 October 2009
Networks made a giant leap towards a better user experience. Users were relatively free of choosing their hardware, possibly different operating systems, although there was still only one application for the main business. Yes, users could run their local applications (solitaire started being popular these days :-) ) but still the system was a bit stiff
Web applications
web server
hardware
application
client
USER FRIENDLINESS
0 100
browser
browser
browser
browser
operating system
hardware
operating system
operating system
hardware
operating system
INT
ERN
ET
Wednesday, 07 October 2009
More flexibility came with the adoption of web infrastructures. The most important being freedom from close location. Web applications run quite well regardless of the location of the servers, thanks also to the availability of better hardware and bandwidth. Applications for the main business, though, remain stiff. Mainly, the business data is only accessible through one set of applications, written in one language, and often tied to a specific operating system.
Cloud applications
web server
hardware
application
client
USER FRIENDLINESS
0 100
browser
browser
browser
browser
operating system
hardware
operating system
operating system
hardware
service provider
applicationapplication
service providerservice
providerservice provider IN
TER
NET
Wednesday, 07 October 2009
With cloud computing, the possibilities grow. Not only the clients are free of choosing hardware and operating systems, but they can also share and choose different applications that are available through providers, independently from the operating system and language used by the clients.
Some actors
• memcached
• gearman
Used in production by Facebook and Digg
Wednesday, 07 October 2009
In this quick introduction, we will only mention a few popular systems that are used in popular web sites. Memcached is a distributed cache environment that helps scaling the performance of demanding web sites. Gearman is described in the next slides
MAG A NE R
Wednesday, 07 October 2009
What is it? Pay attention to the spelling
MG A NER!=
Wednesday, 07 October 2009
Not "german"! It is not a language. It's a system, where a server accepts requests from clients, sends the requests to workers and dispatches the answers back.
MAG A NE R ?
Wednesday, 07 October 2009
And what exactly is it? Think about it. It's an anagram. Do you know of somebody who doesn't do anything except telling the workers what to do and then getting a "thank you" from clients?
M A GA N E R
Wednesday, 07 October 2009
of course! it's a MANAGER.
server worker client
taskjob
request
http://gearman.orgWednesday, 07 October 2009
Anyway, in Gearman, there are several concepts.A worker will register with the server the list of tasks that it can perform.A client will request jobs from the server. The server gives tasks to the workers, collects the results, and passed them back to the clients.
Distributed
Wednesday, 07 October 2009
The beauty of Gearman is that everything can be on a different host.
Multiple operating systems
Wednesday, 07 October 2009
There is no difference if one of the actors is using one operating system rather than the same one used by the server. Actually, clients can get specific advantage of this architecture, by requesting tasks that are not available in their operating system but can be easily performed in the OS of one of the workers.
multiple languages
Wednesday, 07 October 2009
Language, the greatest divider in the recent technology, is not an obstacle anymore. Clients can keep coding in the language they are most familiar with, and the workers will use libraries and classes that are only available to specific languages, without need for the clients to be involved with the nitpicks of such complex systems.
freedom of choice
Wednesday, 07 October 2009
Clients can really use several languages, and request tasks that are performed remotely in any language.
redundancy
Wednesday, 07 October 2009
Gearman can be a robust system. It allows redundant servers for high availability. Each worker can register its tasks to several servers, and every client can request a job from more than one server.
More on similar matters
• Italian Perl Workshop, Pisa, 22-Oct-2009
• FOSS.MY, Kuala Lumpur, Malaysia, 25-Oct-2009
• CodeBits, Lisbon, Portugal, 3-4-5-Dec-2009
http://datacharmer.blogspot.com
http://gearman.orgWednesday, 07 October 2009
