TAMING THE TIGERDEALING WITH PHYSICS IN PROGRAMMING

THE LAWS OF PHYSICS ALWAYS APPLY

• CPUS use electricity and produce heat

• Even computers “in the cloud” are on physical hardware

• There is a limit to the amount of throughput a nic can push

• The larger the data the longer it takes to move it, and the more surface it takes to store it

ARRAYS ARE EVIL

• There are other ways to store data that are more efficient

• They should be used for small numbers of data

• No matter how hard you try, there is C overhead

USE THE ITERATION, LUKE

• Lazy fetching exists for database fetching – use it!

• Always page (window) your result sets from the database – ALWAYS

• Use filters or generators to format or alter results on the fly

STREAM YOUR DATA

• Work on chunks at a time

• Seek back and forth through data if necessary

• Use PHP streams as they were meant to be used

STREAMS: COMPUTING CONCEPT

Definitions• Idea originating in 1950’s

• Standard way to get Input and Output

• A source or sink of data

Who uses them• C – stdin, stderr, stdout

• C++ iostream

• Perl IO

• Python io

• Java

• C#

WHAT IS A STREAM?

• Access input and output generically

• Can write and read linearly

• May or may not be seekable

• Comes in chunks of data

WHAT USES STREAMS?

• EVERYTHING

• include/require _once

• stream functions

• file system functions

• many other extensions

ALL IO

Attach Context

Stream Transport

Stream Filter

Stream Wrapper

HOW PHP STREAMS WORK

USING STREAMS

WHAT ARE FILTERS?

• Performs operations on stream data

• Can be prepended or appended (even on the fly)

• Can be attached to read or write

• When a filter is added for read and write, two instances of the filter are created.

USING FILTERS

THINGS TO WATCH FOR!

• Data has an input and output state

• When reading in chunks, you may need to cache in between reads to make filters useful

• Use the right tool for the job

PROCESS WITH THE APPROPRIATE TOOLS

• Load data into the appropriate place for processing

• Hint – arrays are IN MEMORY – that is generally not an appropriate place for processing

• Datastores are meant for storing and retrieving data, use them

OFFLOAD WORK

• Put work items in queues and inform the user when they’re completed

• It’s not realistic to expect complex reports to be done in seconds, physics apply here too

• Caching complex work items is a good way to balance offloaded work with immediate results

COMMUNICATE WITH OTHER PROCESSES

• Microservices are in essence jobbed systems communicated via http

• You can overload them to work via unix sockets as well

• Rachet or other websockets solutions allow for heavy work with multiplexed communication

• PHP can run in daemons, and even listen and communicate over sockets

NETWORK SOCKET TYPES

• Stream

• Connection oriented (tcp)

• Datagram

• Connectionless (udp)

• Raw

• Low level protocols

DEFINITIONS

• Socket• Bidirectional network stream that speaks a protocol

• Transport• Tells a network stream how to communicate

• Wrapper

• Tells a stream how to handle specific protocols and encodings

USING SOCKETS

THAT SOCKETS EXTENSION…

• New APIS in streams and filesystem functions are replacements

• Extension is very low level

• stream_socket_server

• stream_socket_client

About Me

http://emsmith.net

auroraeosrose@gmail.com

twitter - @auroraeosrose

IRC – freenode – auroraeosrose

#phpmentoring

https://joind.in/talk/f88ef