YAPC::EU 2010 - Going Postal

MotivationOur Solution

IssuesSummary

Going Postal

Chisel Wright

NET-A-PORTER

YAPC::EU 2010

Chisel Wright Going Postal


IssuesSummary

Talking To Strangers

Motivation

Why Bother?



IssuesSummary


A Long Time AgoWhen There Weren’t Any Better Solutions

Direct, far away database writesNeed to know their dialectThey can’t easily change their schemaTightly coupled



IssuesSummary






IssuesSummary






IssuesSummary






IssuesSummary


Not quite so long agoWhen There Were Options

TheSchwartzresolved a slightly different issuefirst step in the right directiondidn’t help with far away issueperl only



IssuesSummary






IssuesSummary






IssuesSummary






IssuesSummary






IssuesSummary

ActiveMQNet::StompNet::ActiveMQ

Our Solution



IssuesSummary


ActiveMQ



IssuesSummary


ActiveMQ

Java guys like itPerl guys can interact with itReliable, well-used solution



IssuesSummary


ActiveMQ




IssuesSummary


ActiveMQ




IssuesSummary


Net::Stomp

STOMP is great for talking to ActiveMQNet::Stomp excellent for quickly interacting with ActiveMQ



IssuesSummary


Net::Stomp

STOMP is great for talking to ActiveMQNet::Stomp excellent for quickly interacting with ActiveMQ



IssuesSummary


Sending with Net::StompTalking to a Queue

# send a message to the queue ’foo’use Net::Stomp;my $stomp = Net::Stomp->new({

hostname => ’localhost’,port => ’61613’,

});$stomp->connect({

login => ’hello’,passcode => ’there’,

});$stomp->send({

destination => ’/queue/foo’,body => ’test message’,

});$stomp->disconnect;



IssuesSummary


Receiving with Net::StompSubscribing to a Queue

# subscribe to messages from the queue ’foo’use Net::Stomp;my $stomp = Net::Stomp->new({

hostname => ’localhost’,port => ’61613’,

});$stomp->connect({

login => ’hello’,passcode => ’there’,

});$stomp->subscribe({

destination => ’/queue/foo’,’ack’ => ’client’,’activemq.prefetchSize’ => 1,

});



IssuesSummary


Receiving with Net::StompPlucking from the Queue

while (1) {my $frame = $stomp->receive_frame;warn $frame->body; # do something here$stomp->ack( { frame => $frame } );

}$stomp->disconnect;



IssuesSummary


Net::StompThoughts

Quickly and easily talk to ActiveMQDoes what it promises

Low-levelCopy-and-paste codingError-handling, . . .



IssuesSummary


Net::StompThoughts





IssuesSummary


Net::StompThoughts





IssuesSummary


Net::StompThoughts





IssuesSummary


Net::StompThoughts





IssuesSummary


Net::ActiveMQ



IssuesSummary


Net::ActiveMQ

Standardise our AMQ solution across appsLower barrier to entry



IssuesSummary


Net::ActiveMQ

Standardise our AMQ solution across appsLower barrier to entry



IssuesSummary


What Is It?

Essentially:Net::StompCatalyst::Engine::Stomp

with a lovely ribbon and bow around it.

Message producersMessage consumers

all in one place.



IssuesSummary


Sending with Net::ActiveMQTalking to a Queue

use Net::ActiveMQ::Producer;

my $producer = Net::ActiveMQ::Producer->new({hostname => ’localhost’,port => 61613

});

$producer->send(’Some::Message’,{ message => ’data’, goes => ’here’ }

);

Net::ActiveMQ::Producer - message type does not exist -Some::Messageat /path/to/.../Class/MOP/Method/Wrapped.pm line 159



IssuesSummary


Sending with Net::ActiveMQA Producer

package Net::ActiveMQ::Producer::Some::Message;use Moose;

with ’Net::ActiveMQ::Role::Producer’;

sub transform {my ($self, $header, $data) = @_;# make sure it goes somewhere$header->{destination} ||= ’/queue/some-message’;

# the desired action from the consumer$data->{’@type’} ||= ’action_method’;

# "transform" the data$data->{process_time} = scalar localtime;

return ($header, $data);}

1;



IssuesSummary


Sending with Net::ActiveMQValidate What You Send

# This is completely optional!sub message_spec {

# Data::Rx formatreturn {type => ’//rec’,

required => {message => ’//str’,

},

optional => {’@type’ => ’//str’,goes => ’//str’,process_time => ’//str’,

},};

}



IssuesSummary


Sending with Net::ActiveMQCatalyst Model

# create your model class./script/myapp_create.pl model \

MyMQ \Net::ActiveMQ \localhost \61613 \YES

# in your controller$c->model(’MyMQ’)->send(

’Some::Message’,{ message => { a => ’shiny’, hash => ’reference’ } }

);



IssuesSummary


Sending with Net::ActiveMQWhy did you do that?

sub message_specUsed by proof-of-conceptCan catch your own mistakesNo compelling reason to remove it

Data::RxNicer error messagesSlightly more agnostic



IssuesSummary







IssuesSummary







IssuesSummary







IssuesSummary







IssuesSummary







IssuesSummary







IssuesSummary


Receiving with Net::ActiveMQQuick But Boring

$ CATALYST_DEBUG=1 \> net_activemq_consumer_server.pl

[debug] Loaded engine "Catalyst::Engine::Stomp"...[debug] Loaded components:.--------------------------------+----------.| Class | Type |+--------------------------------+----------+| ...::Controller::Root | instance |’--------------------------------+----------’...[info] Application powered by Catalyst 5.80016

Not very useful in this stateFairly easy to add something useful



IssuesSummary


Receiving with Net::ActiveMQQuick But Boring

$ CATALYST_DEBUG=1 \> net_activemq_consumer_server.pl

[debug] Loaded engine "Catalyst::Engine::Stomp"...[debug] Loaded components:.--------------------------------+----------.| Class | Type |+--------------------------------+----------+| ...::Controller::Root | instance |’--------------------------------+----------’...[info] Application powered by Catalyst 5.80016

Not very useful in this stateFairly easy to add something useful



IssuesSummary


Receiving with Net::ActiveMQConsuming Some::Message

$ net_activemq_consumer_create.pl controller Some::MessageActiveMQ

created "lib/Net/ActiveMQ/Consumer/Controller/Some"created "t"created "lib/Net/ActiveMQ/Consumer/Controller/Some/Message.pm"created "t/controller_Some-Message.t"

Catalyst::Helper to do the hard work



IssuesSummary


Receiving with Net::ActiveMQConsuming Some::Message

$ CATALYST_DEBUG=1 \> PERL5LIB=$PWD/lib \> net_activemq_consumer_server.pl

[debug] Loaded engine "Catalyst::Engine::Stomp"...[debug] Loaded components:.--------------------------------+----------.| Class | Type |+--------------------------------+----------+| ...::Controller::Root | instance || ...::Controller::Some::Message | instance |’--------------------------------+----------’...[info] Application powered by Catalyst 5.80016



IssuesSummary


Receiving with Net::ActiveMQController Actions

package Net::ActiveMQ::Consumer::Controller::Some::Message;use Moose;BEGIN {

extends’Net::ActiveMQ::Consumer::ControllerBase::MessageDriven’

}__PACKAGE__->config(

action_namespace => ’some-message’);# this will handle ’@type’ of ’action_method’ in# the ’some-message’ queuesub action_method :Local {

my ($self, $c, $message) = @_;$c->log->warn(’Some::Message / some-message / action_method’);

}__PACKAGE__->meta->make_immutable;1;



IssuesSummary


Receiving with Net::ActiveMQQueue::Spec

package Net::ActiveMQ::Consumer::Queue::Spec::Some::Message;use Moose;

sub action_method {return { type => ’//any’ };

}

1;



IssuesSummary

Peer ReviewImplementation

Issues



IssuesSummary


Peer Review

Only recently been used by the rest of the team

BugsHandling errors

Lack of clear documentation

PODHow-To / “How do I. . . ?”



IssuesSummary


Peer Review


BugsHandling errors





IssuesSummary


Peer Review


BugsHandling errors





IssuesSummary


Peer Review


BugsHandling errors





IssuesSummary


Peer Review


BugsHandling errors





IssuesSummary


Peer Review


BugsHandling errors





IssuesSummary


ImplementationGeneral

Some early features not fully phased out@type => ’...’JMSType



IssuesSummary






IssuesSummary






IssuesSummary


ImplementationProducers

Shared spec files implementation not completeStill living in the ::Consumer:: namespaceNot easily used by ::Producer:: classesmessage_spec vs build_message_spec



IssuesSummary






IssuesSummary






IssuesSummary






IssuesSummary


ImplementationConsumers

Consumers limited to Net::ActiveMQ namespaceError-Handling still evolving



IssuesSummary


ImplementationConsumers

Consumers limited to Net::ActiveMQ namespaceError-Handling still evolving



IssuesSummary

Summary

Avoid direct database interactionUse message queuesDRY

The FutureEvolution - final stagesWe’re starting to use it in productionWe hope to release to the CPAN soon



IssuesSummary

Summary





IssuesSummary

Summary





IssuesSummary

Summary





IssuesSummary

Summary





IssuesSummary

Summary





IssuesSummary

Summary





IssuesSummary

QUESTIONS?



IssuesSummary

Obligatory LOLCATLOL-Rilla?

[email protected]


Technology

YAPC::EU 2010 - Going Postal