Your Inner Sysadmin

Chris Tankersley

@dragonmantank

SunshinePHP 2015

SunshinePHP 2015 1

SunshinePHP 2015 2

Who Am I

• PHP Programmer for over 10 years

• Sysadmin/DevOps for around 8 years

• Using Linux for more than 15 years

• https://github.com/dragonmantank

SunshinePHP 2015 3

Here Be Dragons

SunshinePHP 2015 4

Traditional Lamp Stack

SunshinePHP 2015 5

Our Players

SunshinePHP 2015 6

And of course…

SunshinePHP 2015 7

The Server

• /bin - Essential user executable files

• /boot - Stuff that makes the OS boot up!

• /dev - Special device stuff you probably won't touch

• /etc - Configuration files

• /home - User home directories

• /sbin - System binaries

• /usr - Multi-user apps and utilities

• /var - Data usually lives here

SunshinePHP 2015 8

Installing Software

• Compile software from scratch

• Use the package manager (yum/apt)

SunshinePHP 2015 9

Learn to love the Command Line

SunshinePHP 2015 10

Learn a CLI text editor

• vi/vim

• emacs

• nano

SunshinePHP 2015 11

Authentication and Authorization

SunshinePHP 2015 12

sudo

You can give admin access to users (or groups of users) without giving them root.

SunshinePHP 201513

# Add sudo access to a single user to run as rootdragonmantank ALL=(ALL) ALL

# Add sudo access to a full group%admin ALL=(ALL) ALL

You can even restrict what commands the users can run

# Restrict web developers to only restart Apache and MySQL%webdevs 192.168.1.0/255.255.225.0=(root) NOPASSWD:/usr/sbin/service apache2 restart, /usr/sbin/service mysql restart

Jailing Users

Keeps people from getting to things they shouldn't. Protects the users from themselves.

SunshinePHP 2015 14

Jailed Shells

Gives users a full shell but not the entire file system. You can pick and choose what programs the user can have access too. Jailkit makes this incredibly easy to set up.

SunshinePHP 2015 15

Jailed SFTP

Locks the user to a specific base path, but doesn’t give them a shell, much like FTP. You get the security of SSH though! It does require a system user however.

SunshinePHP 2015 16

Jailing SFTP

# In /etc/ssh/sshd_config

Subsystem ftp sftp-internal

# At the bottom of the file

Match User jailedsftp

ChrootDirectory /some/path

AllowTCPForwarding no

X11Forwarding no

ForceCommand internal-sftp

SunshinePHP 2015 17

Docker

SunshinePHP 2015 18

If you do it the non-Docker way

Scripting Languages

SunshinePHP 2015 19

Bash

Most servers use bash as the default shell. Most shells understand bash's syntax. If you find yourself running the same commands over and over, throw it in a bash script.

SunshinePHP 2015 20

Python

Ships with most distros. Great for when you need more power than what bash has.

SunshinePHP 2015 21

PHP!

Leverage your PHP skills to write shell scripts.

• Symfony Console Component

• Aura CLI

SunshinePHP 2015 22

Locking Down your Code

SunshinePHP 2015 23

Running Apache as a different user

MPM-ITK

SunshinePHP 2015 24

MOD_RUID2

<IfModule mpm_itk_module>AssignUserId [user] [user]

</IfModule>

RMode configRUidGid myuser mygroupRDocumentChRoot /var/www/vhosts/domain.com/ www/public

PHP-FPM

user = myuser

group = mygroup

chroot = /path/to/my/chroot

SunshinePHP 2015 25

Logs

SunshinePHP 2015 26

Logrotate

Rotates logs out for organization (or other purposes)

SunshinePHP 2015 27

weeklyrotate 4createinclude /etc/logrotate.d/var/log/wtmp {

monthlyminsize 1Mcreate 0664 root utmprotate 1

}

Logwatch

Script that runs every so often and scans a bunch of logs so you get a pretty e-mail with a summary of events

SunshinePHP 2015 28

--------------------- httpd Begin ------------------------

0.17 MB transferred in 792 responses (1xx 0, 2xx 786, 3xx 0, 4xx 6, 5xx 0)199 Content pages (0.09 MB),593 Other (0.09 MB)

Requests with error response codes400 Bad Request

/w00tw00t.at.ISC.SANS.DFind:): 1 Time(s)404 Not Found

/MyAdmin/scripts/setup.php: 1 Time(s)/phpmyadmin/scripts/setup.php: 1 Time(s)/w00tw00t.at.blackhats.romanian.anti-sec:): 1 Time(s)/webdav/: 2 Time(s)

---------------------- httpd End -------------------------

OSSEC

Actually a Host Intrusion Detection system, but it does this by watching logs. Will alert you immediately to problems, and even shut down the attacks.

SunshinePHP 2015 29

OSSEC HIDS Notification.2012 Oct 24 11:38:10

Received From: maple->/var/log/auth.logRule: 5712 fired (level 10) -> "SSHD brute force trying to get access to the system."Portion of the log(s):

Oct 24 11:38:09 maple sshd[1062]: Failed password for invalid user alias from 199.167.138.44 port 59988 ssh2Oct 24 11:38:07 maple sshd[1062]: Invalid user alias from 199.167.138.44Oct 24 11:38:06 maple sshd[1059]: Failed password for invalid user recruit from 199.167.138.44 port 59884 ssh2

Preventing Intruders

SunshinePHP 2015 30

hosts.deny and hosts.allow

Set of files to allow or deny access to the machine or certain apps/ports on the machine

SunshinePHP 2015 31

IPTables

A firewall that is generally available on Linux machines that can be configured many different ways to allow or block or mangle traffic

SunshinePHP 2015 32

OSSEC

IDS that was logs and will use hosts.deny and iptables to block stuff automatically for you!

SunshinePHP 2015 33

Configuration Management

SunshinePHP 2015 34

What is Configuration Management?

Process by which you figure out what goes on your servers, how you want them set up, and keeping track of that information. Files are usually stored in source control on one server and pushed to clients.

SunshinePHP 2015 35

Why do you need it?

• Ever needed to keep track of when files get changed?

• Ever needed to roll back a change?

• Ever needed to push the same change to a bunch of servers

• Ever needed to set up a server exactly the same way as another server?

SunshinePHP 2015 36

General CM Workflow

SunshinePHP 2015 37

Write a Manifest file

Client checks and compiles the manifests

Client makes changes based on manifests

Ansible

• https://serversforhackers.com/getting-started-with-ansible/

SunshinePHP 2015 38

Puppet

• http://www.erikaheidi.com/page/vagrant

SunshinePHP 2015 39

Server Monitoring

SunshinePHP 2015 40

Quick Poll

• Who here knows that their server is up right now?

• Are all of the required services running?

• Are there enough resources currently available?

SunshinePHP 2015 41

Service Monitoring with Monit

SunshinePHP 2015 42

Host Monitoring with Icinga

SunshinePHP 2015 43

Software Tools

SunshinePHP 2015 44

tmux/screen

Command line multiplexer

SunshinePHP 2015 45

tail

Look at the newest entries in a log, or even watch log files as they are generated

SunshinePHP 2015 46

curl

Command line program for transferring data via a URL

SunshinePHP 2015 47

iftop

Displays a breakdown of bandwidth usage by host

SunshinePHP 2015 48

htop

Slightly better interface for checking memory and CPU usage

SunshinePHP 2015 49

tcpdump

Allows you to view and record data transmitted over the network. Couple this with wireshark and you can inspect the packets!

SunshinePHP 2015 50

Servers for Hackers

Chris Fidao

@fideloper

http://serversforhackers.com

SunshinePHP 2015 51

Questions?

SunshinePHP 2015 52

Thank You!

http://ctankersley.comchris@ctankersley.com

@dragonmantank

https://joind.in/13421

SunshinePHP 2015 53