Big guns for the small guys (reloaded)

Jorge Lopez-Lago6 March 2016

Annotated!

http://www.wallpaperup.com/150483/transport_military_cargo_A400M_f.html

Engineer

The last airplane I was part of.Testing the engines!

https://www.amazeelabs.com/en/blog/fun-potatoes-other-wallpapers

Drupaleer Not endorsed.I just like the table :)

My company!

AT-AT

http://starwars.wikia.com/wiki/All_Terrain_Armored_Transport

Advice and tools.Repeat.

Disclaimer(*) There is always a disclaimer.

(*)

I do not endorse any of the products or brands mentioned in this talk.I have used them, I am using them or I am looking into using them in the near future.

This is an opinionated talk.Some swearing may be featured in it.

***!

What have I learnt?

95% of what we learn happen before the age of 7 (I heard somewhere).

Humans build stuff as badly as possible, but not worse.Me Some call it “optimization”.

It is an strategy that works pretty well… until it doesn’t.When it doesn’t it is usually because we crossed the crap line.An expert is someone that has failed many times. That helps to estimate the crap line in projects.

AdviceBased on my experience and preferences. You may disagree violently...

http://nutritiouslife.com/edible-sunscreen-you-need-to-know-about/

Sunscreenhttps://en.wikipedia.org/wiki/Wear_Sunscreen

At least you will have a piece of solid, backed by science, good advice.

We are in the business of selling.

There is no other business.The sooner you get it the better.

Do not sell your time!Sell the product of your work,

not your time.

FYI: this is bad stuff.

Bodyshopping is a form of modern slavery based on “employment” ideas of the 18th Century.

Small is good.It is the right thing to do.

Less people = less bullshit, less noise, more clarity.Cut down the middleman between you and the final client.

Small clients are good for this strategy.Everyone has his personal agenda, with personal objectives.

Personal agendas interfere with project objectives. Do the math..

Luke: Is the dark side stronger?Yoda: No, no, no. Quicker, easier, more seductive.

AKA: big corporation teams.

Broken telephone effect:CEO > Marketing > Designer > B. Analyst > Designer > P. Owner > T. Lead > Developer

Littered with middle managers as well.

Fear is the path to the dark side...fear leads to anger… anger leads to hate… hate leads to suffering.Yoda

Fear of losing their jobs.Large companies policies make people to worry about their jobs, not their work.

This industry is cheap.Code is cheap.

And this is one of the main problems with it. Everyone thinks that we can keep making changes to code indefinitely because it

has a perceived cost of almost nothing; after all it is “digital/virtual” and we can rewrite it at no cost…

Agile?Whatever works for you!

For me it is a combination of frozen specifications, architecture planned in advance (you need overall vision of the project), prototyping, many checklists and agile Ganttified sprints.

But never “agile planning”, whatever that means.

Never start everything at the same time!

Big teams go large here.

I have seen things… whole development teams coding frantically, directed by the PM, before the architect joins the project.

Someone hired them early and they must work (write code!) as they are being paid to warm up their chairs.

The train can not stop: timber!

First, solve the problem.Then, write the code.John Johnson

Practical tip: print this slide and paste it to your computer screen.

But we cost more and take longer than building a house!

The average time to build a family house is 6 months.The perception of cheapnes is the origin of our trouble as it provokes multiple cycles of changes (together with developers reinventing the wheel in each

project). We do not have the restrictions of the physical world: when something is done it cannot be changed. That helps the house builders.

Plain bad planningIt is really difficult to manage and plan projects.

Sometimes I have the impression that…Those who can, code. Those who can’t, manage.And that is not a good strategy for success.

A good PM is priceless. And very rare.

Plans are of little importance, but planning is essential.Winston ChurchillDo not mistake software planning with project delivery planning. The first one must be done in advance while the former may be done in an agile way as the project progresses. I am afraid all planning is treated in the same fashion by the agile crowds.

Mainly software planning

No battle plan survives contact with the enemy.Helmuth von Moltke the Elder

Always have a plan B.

Use checklists.For absolutely everything.The whole aerospace industry is based around checklists. I am sure we can do very well with them in our industry...

Use Gantt charts.To control the work allocated to tasks and

the dependencies between them.

I am open for suggestions. And not, I do not trust developers, or other people for the matter.

Development, design and content creation work planned for one sprint. You only need colourful post its and a large enough window.

The science behind it

Says who?

Parkinson’s bicycle shed effectEveryone knows better… the simple stuff. And our industry is

perceived as simple (we have been promoting it like that for over a decade). The larger the team or group of people involved the more difficult to control. And subordinates will not question their

masters… we are doomed!

https://en.wikipedia.org/wiki/Law_of_triviality

Pareto principle (80-20)80% of effects come from 20% of the causes

20% of code has 80% of errors20% of features provide 80% of functionality

... 20% of staff create 80% of problems

For many events approximately...

https://en.wikipedia.org/wiki/Pareto_principle

Metcalfe's lawThe cost of interrelated or connected operations increases exponentially with the number of connections:- Testing becomes more expensive as the software grows and

relations increase between components.- Team management gets more complicated fast with the number

of members (meetings, communications between parties, etc).

https://en.wikipedia.org/wiki/Metcalfe's_law

Parkinson’s lawWork expands so as to fill the time available for its completion.

So if a developer “thinks” that he has a sprint worth of time it will take him one sprint to deliver. But if he thinks he has three days…

You need a Gantt chart to figure this out.

https://en.wikipedia.org/wiki/Parkinson%27s_law

What do I need?

And what I do not need

You need testing.Automated testing.

Choose your flavour and tool and master them (TDD, BDD, PHPUnit, SimpleTests, Selenium, etc). Get used to coding tests as you code features (or before, up to you). Tests are your QA metric, your

benchmark for improvements, your professional standard seal. And, for God's sake, test patches and bug fixes. Identify your smoke

tests and keep them updated.

The only valid QA measurements are maintainability and reliability.

No one will care about feature xif the application or site keeps crashing.

And no one will miss feature x if the application runs smoothly and costs peanuts to maintain.

The central enemy of reliability is complexity.Geer et al.

Get the idea. It is easier to say than to do.

Simplicity is prerequisite for reliability.Edsger W. Dijkstra

Another way to say it.It is important, so let's repeat it.

Bultaco Sherpa. My father’s motorbike.Simple, robust, reliable, maintainable. Indestructible.From 1970 to date (46 years in the family!). https://ca.wikipedia.org/wiki/Bultaco_Sherpa_T

Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.Rick Osborne

In case you didn’t get it.

Cost of (unit) testing?Just a practical trick to

estimate the testing routines required for your code.

N-Path complexity.Number of unique paths in a routine.Minimum number of tests required to completely test a routine.

Got it? Me neither.

entity_load()NPath: 2

http://blog.ircmaxell.com/2013/05/development-by-numbers-slides.html

Great talk. View it.

Great!Just 2 tests required :)

Use PHPMD (Mess detector) to calculate it.(see PHPQA slide for tool info).

drupal_http_request()NPath: 25,303,344,960

http://blog.ircmaxell.com/2013/05/development-by-numbers-slides.html

WTF?

If you like this, you will love CRAP!(Change Risk Analysis Predictions)

You do not need CI.

In essence it is a bunch of scripts executed automatically on code commits. You can do it manually on demand and still be cool.

You do not need the cloud.Local is good (if you can share it).

See DrupalVM later.

But backup your stuff!

Good and now is better that perfect and tomorrow.

In code, testing and love.

Tomorrow never comes.

Share your work with other professionals.We can’t do everything under the sun. Outsource work to professionals: development work or supporting tasks (accountancy, contracts, etc).

Specialization leads to efficiency. Efficiency leads to profits.

If you think it's expensive to hire a professional to do the job, wait until you hire an amateur. Red Adair Whatever you outsource will

affect your delivery with your client. Find good collaborators and be fair.No client is worth few pennies saved.

Prototype in Drupal.Turn web building process around.

Start with Drupal, then Design and finish with Drupal.

I have been championing this for over 7 years.

I am not alone. At last!See following slides

https://www.phase2technology.com/blog/static-prototyping-and-keeping-drupal-simple-kds/

http://pronovix.com/blog/prototyping-drupal

Understand the basicsExample: do not misunderstand technical debt with writing bad code knowingly. Code must be always written as best as we can, with the best understanding of the project at the point. Debt comes from

partial understanding of what you are coding (the project).

https://http2.github.io/How many web devs do you know that do not understand

the HTTP protocol? And how many are stuck in 1.0?

https://en.wikipedia.org/wiki/Web_cache Understanding web caching is paramount.

https://en.wikipedia.org/wiki/Technical_debt Everyone misunderstandsthis concept big time.

Sell!Do as little development as possible and as much selling as bearable.

Find your unfair advantage.WHat do you do well? What is it that you do much much better than others? Transform it in an advantage. Find a way of using it to

surpass your competitors, to stand out.The more unfair (for others) the better (for you).

Go and find it!

Every client thinks he is an unique unicorn.And no one is.

No one.As small guys we are in a great position to manage their delusional dreams. We are their experts, so behave as such and guide the client through a doable plan to succeed. We control the product here. Avoid development spirals.

In big projects with big teams there are too many delusional unicorns to tame.

Not even you, I am afraid. Be pragmatic here.

Manage expectations. Communication is key.

Better to say anything early than late.Bad news delivered frequently and with plenty of time are much much better than saved until the very last minute. Most people are quite reasonable if given time to react and respond. Don’t be afraid of

delivering bad news. Shit happens. The sooner we find out the better we are positioned to fix it.

Time to market beats features hands downNow and flawed is much better than never and perfect.

Cut down marginal-benefit features ruthlessly. And ship early.Focus on the 20% that will give you the 80ish%.

Build a streamlined product and sell it again and again. That is the key to profit.

Companies profit on top of imperfect products.

The whole planet is doing it.Why don’t we do it in “digital” industries?

Why can’t we?

Ditch your marketing departmentMarketing is for when the clients find you. It is lazy selling. It promises everything at the expense of development costs.

What small guys need is a hardcore salesperson to sell your “product” to as many clients as possible. They need to be convinced rather than

promised any fancy dreams. Think of profitable actions here..

Lie about your professional standards.Do not let the client to dictate them.

This is the only industry where the client decides about the professional standards of the suppliers.

If the client or manager knows you will spend time testing your code, he will try to remove it from the project. Testing is part of your professional standards and responsibility. They can not be removed.

This is insane, madness!

http://www.todayonline.com/world/asia/photos-show-felled-building-taiwan-quake-had-tin-can-fillers

February 2016, Taiwan. A new building collapsed like an accordion during a moderate earthquake killing 170 people.It had “tin reinforced” concrete.No engineer could have ever agreed to do that due to professional standards. Draw your conclusions here.

Tools

Know your tools well.The tools you use do not matter. What matters is how good you are

with them. So… use the web server that you know well!Consider the cost of continuously adopting the “latest buzz thing”:

learning curve, slow development, inefficient development, technical debt, short shelf life of new rapidly changing technologies.

The technology you use doesn’t matter as much as what you can do with it. Stop running after the best new thing.

https://www.digitalocean.com/community/tutorials/apache-vs-nginx-practical-considerations

Miscellaneous tools

https://github.com/ansible/ansible Provisioning at its best. Can’t live without it.

http://phpqatools.org/ Every PHP developer should know these. And use them.

http://robo.li/ Deployment, installation, tests running, home baked CI...

https://thephpleague.com/ S-W-E-E-T-!

https://www.jetbrains.com/phpstorm/ Bye bye Eclipse. Paid for, but so good.

https://bitbucket.org/ Free private repositories. Say no more.

https://www.dropbox.com The cheapest and easiest local backup.

Drupal development tools

http://drushcommands.com/ Easy to read. Great source to explore drush doc.

http://www.drush.org/en/master/make/ Just build Drupal sites with make. Now!

http://www.drupalvm.com/ Your local VM as 1,2,3! Packed with goodies.

https://reload.github.io/deployotron/ Very interesting. Seems abandoned?

https://www.drupal.org/project/profiler_builder A helper for your profiles.For lazy devs or newbies.

https://www.drupal.org/project/master Manage config per environment. Yes!

https://www.drupal.org/project/paragraphs Everyone is doing paragraphs.Beware of its limitations, by design.

Debugging tools

https://xdebug.org/ This is a must. Run it from your IDE for maximum gain.

http://phpdbg.com/ Were you aware of this?

https://addons.mozilla.org/en-US/firefox/addon/firephp/

The most effective debugging tool is still careful thought, coupled with judiciously placed print statements. Brian W. Kernighan (1979)

In case you areold fashioned.

https://blackfire.io/features My next toy.

Paperwork tools

https://stuffandnonsense.co.uk/projects/contract-killer/ Check it now. Go, do it!

Testing tools

https://www.drupal.org/project/drulenium Wraith in Drupal.Does it get better than this?

https://www.phptesting.org/ If you really need CI in house. DIY. PHP, yeah!

https://httpd.apache.org/docs/2.4/programs/ab.html The classic benchmarking tool.

https://redmine.lighttpd.net/projects/weighttp/wiki Benchmarking.

https://github.com/wg/wrk More benchmarking.

Design tools

http://styletil.es/ Please, please, please designers: embrace this.And atomic design.

http://pencil.evolus.vn/ My tool of choice for UX mockups, although not perfect.

https://www.lucidchart.com/ It just works. Tidy diagrams.

http://uxmyths.com/ Fun and very useful.

Which Drupal?

Hobby Amateur Professional Enterprise

D6

D7

D8

Drupal reach

Will amateurs ever tinker with Drupal 8?

Drupal size

D6

I know it all :)

D7

Drupal size

D6 D7

I still know it all :)

D8D7

Drupal size

D6 D8D7

WTF? WIll I ever know enough of it?The end of rockstars and solo heroic devs..

D8B1D7

Drupal size

D6 B1D7 D8

This is Backdrop. There is still hope to know it all!

D6

D7

D8

B1

Drupal size Each “dot” represents a “book of code” of 760 pages, with 72 lines of code per page.

I totally dominate this!

24 volumes of code of 760 pages each! WTF? How many developers do I need in my team to master it?

Drupal size

D 6.37 D 7.42 B 1.3.2 D 8.0.3

Files 500 1,100 2,100 12,300

Folders 60 150 330 3,600

Lines 55,000 177,000 230,000 1,300,000

Pages 760 2,500 3,200 18,000

Volumes 1 3 4 24

Calculated using cloc and tree and rounding the numbers for clarity. Each page has 72 lines of code.Drupal 8 includes vendor folder.

988,000 w/o vendor

https://backdropcms.org/ The moral is to pick the right tool for your size.

I would rather use Backdrop than Wordpress… don’t quote me on this.

Server monitoring

https://sensuapp.org/

https://mmonit.com/monit/

Logs aggregators

https://www.elastic.co/products/logstash DIY. Complicated but... powerful!

https://papertrailapp.com/ Very easy to set up. Good, not great.

https://www.graylog.org/ Very interesting. DIY solution.

https://www.loggly.com Very easy to set up. Drupal modules to help.

Searching

https://www.elastic.co/products/elasticsearch Once you go elastic, there is no other search so fantastic!

Thanksjorge.lopez-lago@onload.uk.com

kurkuma at Drupal.org

Skull icons from http://www.dlanham.com/ui/sticker/