Join the fightAgainst email spam!

1

Why would we this?4 People waste their time sorting SPAM

4 Lost money by phishing emails

4 banks, creditcards, invoices

4 No trust in their real message

4 Google force you to do!

2

Safer Internet DayFebruary 9, 2016

3

4

5

6

Who is sending emailsfrom there applications?

7

Who is runninghis own emailserver?

8

Who is in chargeof the DNS-records?

9

Who recognize

this situation?

10

My email to bob@example.com

has not arrived.1

Our client(s)

11

My email has not arrived..Lots of reasons

4 The code doesn't send the email

4 The server IP-adres is on the (RBL) blacklist

4 The receiver server doesn't trust your IP-adres

4 The content is marked as SPAM

4 The email policy is not configured or not optimal

12

My email has not arrived..What can we do about it?

4 Check the function of the script

4 Check the server IP-adres on the (RBL) blacklist

4 Submit for removal

4 Checking the email policies [SPF/DKIM]

4 Using email services providers

13

How we did it the old days2016-04-01 05:00:13 [1487] 1Ov4tU-0000Nz-Rm H=mailhost.domain.com [208.42.54.2]:51792 I=[67.215.162.175]:25 Warning: "SpamAssassin as theuser detected message as NOT spam (0.0)"2016-04-01 05:00:13 [1487] 1Ov4tU-0000Nz-Rm <= maillinglist@domain.com H=mailhost.domain.com [208.42.54.2]:51792 I=[67.215.162.175]:25 P=esmtps X=TLSv1:AES256-SHA:256 CV=no S=21778 id=384a86a39e83be0d9b3a94d1feb3119f@domain.com T="Daily Science Maillinglist: Chameleon" from for user@example.com2016-04-01 05:00:14 [1534] 1Ov4tU-0000Nz-Rm => user F= P= R=virtual_user T=virtual_userdelivery S=21902 QT=6s DT=0s2016-04-01 05:00:15 [1534] 1Ov4tU-0000Nz-Rm Completed QT=7s

14

Email service providers

15

16

17

18

19

20

21

22

Email authentication

23

Email authentication1. SPF

2. DKIM

3. DMARC

24

SPF

25

SPFSender Policy Framework

26

SPF4 Created in 2003

4 Which mail servers are used to send mail from your domain

4 Publish an SPF record in our DNS records

4 Technical method to prevent sender address forgery

27

SPFThis technology requires two sides to play together

1. The domain owner, publishing an SPF record

2. The receiving server, checking for domain SPF records

28

SPFIf the message comes from an unknown server, it can be considered as fake and could be rejected.

29

SPF record - JCIDLet's look at an example

jcid.nl. TXT "v=spf1 include:spf.jcid.nl include:_spf.google.com include:spf.mandrillapp.com include:_spf.exactonline.nl -all"

30

SPF record - SweetLake PHPThe parts of the SPF record mean the following:

sweetlakephp.nl. TXT "v=spf1 +a +mx include:spf.mandrillapp.com ~all"

4 v=spf1

4 a

4 149.210.152.247

4 mx

4 mx.transip.email

4 include:spf.mandrillapp.com

4 ~all

31

SPF mechanisms

32

SPF mechanisms4 Domains define zero or more mechanisms.

33

SPF mechanismsall | ip4 | ip6 | a | mx | ptr | exists | include

34

SPF mechanismsMechanisms can be prefixed with one of four qualifiers:

"+" Pass"-" Fail"~" SoftFail"?" Neutral

35

SPF mechanismsThe default qualifier

"+", i.e. "Pass".

36

SPF - The "ip4" & "ip6" mechanismip4:<ip4-address>ip4:<ip4-network>/<prefix-length>

ip6:<ip6-address>ip6:<ip6-network>/<prefix-length>

37

SPF - The "ip4" & "ip6" mechanism"v=spf1 ip4:192.168.0.1/16 -all"

Allow any IP address between 192.168.0.1 and 192.168.255.255.

"v=spf1 ip6:1080::8:800:200C:417A/96 -all"

Allow any IPv6 address between 1080::8:800:0000:0000 and 1080::8:800:FFFF:FFFF.

38

SPF - The "a" & "mx" mechanismaa/<prefix-length>a:<domain>a:<domain>/<prefix-length>

mxmx/<prefix-length>mx:<domain>mx:<domain>/<prefix-length>

39

SPF - The "include" mechanisminclude:<domain>

Example

include:spf.mandrillapp.com

40

SPF - The "include" mechanismExact Online Example

ip4:xxx.xxx.xxx.xxx ip4:yyy.yyy.yyy.yyy ip4:zzz.zzz.zzz.zzz

41

SPF record - SweetLake PHPsweetlakephp.nl. TXT "v=spf1 +a +mx include:spf.mandrillapp.com ~all"

42

SPF mechanismsThe default qualifier

"+", i.e. "Pass".

43

SPF record - The "all" mechanismsweetlakephp.nl. TXT "v=spf1 +a +mx include:spf.mandrillapp.com ~all"

44

SPF -all

45

SPF -allStopping email forgery

46

SPF stats - All domains

SPF -all, 1 April 2016 SPF -all - Stats.

47

SPF stats - Domains with SPF record

SPF -all, 1 April 2016 SPF -all - Stats.

48

SPF - The "all" mechanism"v=spf1 mx -all"

49

SPF - The "all" mechanism"v=spf1 -all"

50

SPF - The "all" mechanism"v=spf1 +all"

51

SPF results

52

SPF resultsAn SPF record can return any of these results:

1. Pass------------2. Fail3. SoftFail------------4. Neutral5. None------------6. PermError7. TempError

53

54

SPF result1 - Pass (accept)

Received-SPF: pass (bob.example.org: domain of alice@example.com designates 192.0.2.1 as permitted sender) receiver=bob.example.org; client_ip=192.0.2.1; envelope-from=alice@example.com; helo=mailout00.controlledmail.com;

55

SPF result - ReceiverReceived-SPF: pass (bob.example.org: domain of alice@example.com designates 192.0.2.1 as permitted sender) receiver=bob.example.org; client_ip=192.0.2.1; envelope-from=alice@example.com; helo=mailout00.controlledmail.com;

receiver=bob.example.org

the host name of the SPF client

56

SPF resultReceived-SPF: pass (bob.example.org: domain of alice@example.com designates 192.0.2.1 as permitted sender) receiver=bob.example.org; client_ip=192.0.2.1; envelope-from=alice@example.com; helo=mailout00.controlledmail.com;

client_ip=192.0.2.1;

the IP address of the SMTP client

57

SPF resultReceived-SPF: pass (bob.example.org: domain of alice@example.com designates 192.0.2.1 as permitted sender) receiver=bob.example.org; client_ip=192.0.2.1; envelope-from=alice@example.com; helo=mailout00.controlledmail.com;

envelope-from=alice@example.com;

the envelope sender mailbox

58

SPF resultReceived-SPF: pass (bob.example.org: domain of alice@example.com designates 192.0.2.1 as permitted sender) receiver=bob.example.org; client_ip=192.0.2.1; envelope-from=alice@example.com; helo=mailout00.controlledmail.com;

helo

the host name given in the HELO or EHLO command

59

SPF result2 - Fail (reject)

Received-SPF: fail (bob.example.org: domain of alice@example.com does not designate 192.0.2.1 as permitted sender)

3 - SoftFail (accept but marked)

Received-SPF: softfail (bob.example.org: domain of transitioning alice@example.com does not designate 192.0.2.1 as permitted sender)

60

SPF result4 - Neutral (accept)

Received-SPF: neutral (bob.example.org: 192.0.2.1 is neither permitted nor denied by domain of alice@example.com)

5 - None (accept)

Received-SPF: none (bob.example.org: domain of alice@example.com does not designate permitted sender hosts)

61

SPF result6 - PermError (unspecified)

Received-SPF: permerror -extension:foo (bob.example.org: domain of alice@example.com uses mechanism not recognized by this client)

7 - TempError (accept or reject)

Received-SPF: temperror (bob.example.org: error in processing during lookup of alice@example.com: DNS timeout)

62

Recap

63

64

DKIM

65

DKIMDomainKey Identified Mail

66

DKIMDigital signature

67

Why DKIM?DKIM is an important authentication mechanism

68

DKIM4 Email receivers

4 Phishing emails (banks, creditcard, invoices)

4 Email senders

4 No trust in their real message

69

DKIMTwo proposals took shape, 2005

1. Yahoo’s DomainKeys

2. Cisco’s Identified Internet Mail

70

DKIMBoth proposals were based in the use of

“ Public Key Cryptography ”

71

DKIMMid 2005, the IETF (Internet Engineering Task Force), submitted the draft “ DomainKeys Identified Mail — DKIM ” specification.

72

How does DKIM work?

73

How does DKIM work?1. Author wishes to send an email to a recipient

2. They (their mailing software) calculate a crypto signature

4 that covers the relevant parts of the message using the Private Key.

3. The signature is placed in the email header

4 and the message is then sent normally by the mail server.

4. At any point in travel the signature is validated using the public key.

5. If any part of the message covered by the signature was manipulated

4 the signature won’t validate and the recipient will be alerted.

74

How does DKIM work?4 Public Key Cryptography like SSH

4 Private key v.s. Public key

4 DKIM uses DNS to publish the Public Keys

75

76

DKIM headerDKIM-Signature: v=1; a=rsa-sha256; c=simple/relaxed; d=jcid.nl; s=mandrill; t=1399817581; bh=Pl25…dcMqN+E=; h=Message-ID:Date:Subject:From:To:MIME-Version:Content-Type; b=Xp/nL93bv6Qo73K…KmskU/xefbYhHUA=

77

DKIM header - Versionv=1

This indicates the DKIM version in use.

78

DKIM header - Algorithma=rsa-sha256

The algorithm suite that was used to generate the crypto signature.

Current two specification defines

4 rsa-sha1

4 rsa-sha25679

DKIM header - Canonicalizationc=simple/relaxed

Note that the c= fragment defines two algorithms.

80

DKIM header - Domaind=jcid.nl

81

DKIM header - Selectors=mandrill

82

DKIM header - Selectortxt:mandrill._domainkey.jcid.nl

v=DKIM1;k=rsa;p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCrLHiExVd55zd/IQ /J/mRwSRMAocV/hMB3jXwaHH36d9NaVynQFYV8NaWi69c1veUtRzGt 7yAioXqLj7Z4TeEUoOLgrKsn8YnckGs9i3B3tVFB+Ch/4mPhXWiNfN dynHWBcPcbJ8kjEQ2U8y78dHZj1YeRXXVvWob2OaKynO8/lQIDAQAB

83

DKIM header - Timestampt=1399817581

84

DKIM header - Body partbh=Pl25…dcMqN+E=

85

DKIM header - Header listh=Message-ID:Date:Subject:From:...

86

DKIM header - Datab=Xp/nL93bv6Qo73K…KmskU/xefbYhHUA=

4 The crypto signature data itself, encoded in Base64 and possibly with whitespace inserted to conform to line length limitations.

87

DKIM resultsThe possible results for your DKIM test are:

1. Pass2. Fail3. None4. Policy5. Neutral6. TempError7. PermError

88

DKIM results - PassThe message was signed, the signature or signatures were acceptable, and the signature(s) passed verification tests.

89

DKIM results - FailThe message was signed and the signature or signatures were acceptable, but they failed the verification test(s).

90

DKIM results - NoneThe message was not signed

91

DKIM results - PolicyThe message was signed but the signature or signatures were not acceptable.

92

DKIM results - NeutralThe message was signed but the signature or signatures contained syntax errors or were not otherwise able to be processed.

93

DKIM results - TemperrorThe message could not be verified due to some error that is likely transient in nature, such as a temporary inability to retrieve a public key. A later attempt may produce a final result.

94

DKIM results - PermerrorThe message could not be verified due to some error that is unrecoverable, such as a required header field being absent. A later attempt is unlikely to produce a final result.

95

MoneyBird - SPAM

96

MoneyBird - Inbox

97

Cal Evans

98

Recap

99

100

DMARC

101

DMARCDomain-based Message Authentication,

Reporting & Conformance

102

DMARC4 Created in 2007 by PayPal, and Yahoo!

4 Later Gmail joined

103

What is DMARC

104

What is DMARCRemove the guesswork

105

What is DMARCReport back to the sender

106

107

DMARC record - JCIDLet's look at an example

_dmarc TXT "v=DMARC1; p=none; pct=100; rua=mailto:re+oqz4ekvxqt0@dmarc.postmarkapp.com; sp=none; aspf=r;"

108

DMARC record - Versionv=DMARC1

This indicates the DMARC version in use.

109

DMARC record - Percentagepct=100

Percentage of messages subjected to filtering

110

DMARC record - Aggregate reportrua=mailto:aggrep@example.com

Reporting URI of aggregate reports

111

DMARC record - Failure Reportsruf=mailto:re+oqz4ekvxqt0@dmarc.postmarkapp.com

Reporting URI for forensic reports

112

DMARC record - Policyp=none

Policy for domain

4 none

4 quarantine

4 reject

113

DMARC record - Sub-domain Policysp=none

Sub-domain Policy

114

DMARC record - Alignmentadkim=s

Alignment mode for DKIM- r = relaxed (default)- s = strict mode

115

DMARC record - Alignmentaspf=r

Alignment mode for SPF- r = relaxed (default)- s = strict mode

116

Recap

117

118

DMARCAggregate report

119

DMARCZIP file

google.com!jcid.nl!1455062400!1455148799.zip

with XML aggregate report

google.com!jcid.nl!1455062400!1455148799.xml

120

DMARC report<?xml version="1.0" encoding="UTF-8" ?><feedback> <report_metadata> <org_name>google.com</org_name> <email>noreply-dmarc-support@google.com</email> <extra_contact_info>https://support.google.com/a/answer/2466580</extra_contact_info> <report_id>4151131448954607551</report_id> <date_range> <begin>1455062400</begin> <end>1455148799</end> </date_range> </report_metadata> <policy_published> <domain>jcid.nl</domain> <adkim>r</adkim> <aspf>r</aspf> <p>none</p> <sp>none</sp> <pct>100</pct> </policy_published> <record> <row> <source_ip>31.3.97.173</source_ip> <count>1</count> <policy_evaluated> <disposition>none</disposition> <dkim>fail</dkim> <spf>fail</spf> </policy_evaluated> </row> <identifiers> <header_from>nonstopdeals.prod.jcid.nl</header_from> </identifiers> <auth_results> <spf> <domain>nonstopdeals.prod.jcid.nl</domain> <result>none</result> </spf> </auth_results> </record></feedback>

121

DMARC report<?xml version="1.0" encoding="UTF-8" ?><feedback> <report_metadata> <org_name>google.com</org_name> <email>noreply-dmarc-support@google.com</email> <extra_contact_info>https://support.google.com/a/answer/2466580</extra_contact_info> <report_id>4151131448954607551</report_id> <date_range> <begin>1455062400</begin> <end>1455148799</end> </date_range> </report_metadata></feedback>

122

DMARC report<?xml version="1.0" encoding="UTF-8" ?><feedback> <policy_published> <domain>jcid.nl</domain> <adkim>r</adkim> <aspf>r</aspf> <p>none</p> <sp>none</sp> <pct>100</pct> </policy_published></feedback>

123

DMARC report<?xml version="1.0" encoding="UTF-8" ?><feedback> <record> <row> <source_ip>31.3.97.173</source_ip> <count>1</count> <policy_evaluated> <disposition>none</disposition> <dkim>fail</dkim> <spf>fail</spf> </policy_evaluated> </row> <identifiers> <header_from>nonstopdeals.prod.jcid.nl</header_from> </identifiers> <auth_results> <spf> <domain>nonstopdeals.prod.jcid.nl</domain> <result>none</result> </spf> </auth_results> </record></feedback>

124

DMARC reportI'm in control

125

DMARC - Tools1. Postmark App

2. Dmarcian

126

Postmark DMARC monitor

127

128

Dmarcian

129

130

Overview DNS records JCIDSPF @ TXT v=spf1 include:spf.jcid.nl include:_spf.google.com include:spf.mandrillapp.com include:_spf.exactonline.nl -all

DKIM google._domainkey TXT v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC+w63i8quIsOR09AfNup5pyt/jsSmKo/iQnOkT8EI1LOn6daR1GqR+5... mandrill._domainkey TXT v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCrLHiExVd55zd/IQ/J/mRwSRMAocV/hMB3jXwaHH36d9NaVynQFYV8N...

DMARC _dmarc TXT v=DMARC1; p=none; pct=100; rua=mailto:re+oqz4ekvxqt0@dmarc.postmarkapp.com; sp=none; aspf=r;

131

How to start your own?4 Deploy SPF & DKIM

4 Publish a DMARC record with the “none” flag set for the policies (monitor mode)

4 Analyze the data and modify your DMARC policy

4 from “none” to “quarantine” to “reject”

132

Any questionsAbout the theory?

133

MXToolbox

134

135

136

137

138

Delivered-To: info@jcid.nlReceived: by 10.194.81.166 with SMTP id b6csp2710139wjy; Thu, 3 Mar 2016 03:07:28 -0800 (PST)X-Received: by 10.28.177.134 with SMTP id a128mr347820wmf.55.1457003248665; Thu, 03 Mar 2016 03:07:28 -0800 (PST)Return-Path: <ramon@delafuente.nl>Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com. [2a00:1450:400c:c09::236]) by mx.google.com with ESMTPS id b71si9817151wmd.46.2016.03.03.03.07.28 for <info@jcid.nl> (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Mar 2016 03:07:28 -0800 (PST)Received-SPF: pass (google.com: domain of ramon@delafuente.nl designates 2a00:1450:400c:c09::236 as permitted sender) client-ip=2a00:1450:400c:c09::236;Authentication-Results: mx.google.com; spf=pass (google.com: domain of ramon@delafuente.nl designates 2a00:1450:400c:c09::236 as permitted sender) smtp.mailfrom=ramon@delafuente.nl; dkim=pass header.i=@delafuente.nlReceived: by mail-wm0-x236.google.com with SMTP id l68so29526516wml.0 for <info@jcid.nl>; Thu, 03 Mar 2016 03:07:28 -0800 (PST)DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delafuente.nl; s=google; h=date:from:to:message-id:in-reply-to:references:subject:mime-version; bh=FSN7Fi9D6o/kqt5G7qz43kmPTxT4eFBfPTd+OGfRiZ4=; b=RDqYimIWeTNR13wseVHStCgo+iVXpE5LeUFSpmJETvVC2OnxuEBOF9vlF5JfWjJ4C5 nheVvDqWUSRHo06kcZ+IgsWSGCIDUNrn14y065xCD9CTYCZcmuKWJyZhfYiSQco3GDiO SVGnW36e3toxNzAtsPyhiN7Xt++euRCgoYbv8=X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version; bh=FSN7Fi9D6o/kqt5G7qz43kmPTxT4eFBfPTd+OGfRiZ4=; b=WutHJxu1kCncM3pWRitfDiiNouwzedP7o6Ta7lfeRz5FGTfuuv1aQcURtZWtaXKp8S YKlRPZa5VeQHuzerxsQrtKwTqHB2+N3FtQWmQVIdBQS+JRZ9tXeka3qeiLSRTqdI6huZ lN6XgaF80KedTJqh1etPpMa92C+qbYbMhXmhacUhanfUdwWXQs7gIeOOds4YXK3hEgbT mp9jU9ajA9sQumWUa5upPyw5DdKuSpiRt70J5BIU5DFgCXSBcdmxfiWaOYvnqRssSERD 6xdYKT8RnetKFn7h+gGDVjs4texPN1Inmek4tUIpIdq0a/hv5av8AJj/TCJiCNylJzCa VxDw==X-Gm-Message-State: AD7BkJL923JBKM2KJibPZmJoZ+9qAnqpVPywwpLQLsMUj+kfIf7dmPNeDOaCv4+cqCOEdA==X-Received: by 10.28.194.132 with SMTP id s126mr5301943wmf.23.1457003248334; Thu, 03 Mar 2016 03:07:28 -0800 (PST)Return-Path: <ramon@delafuente.nl>Received: from FzzBook.fritz.box ([2001:981:fe71:1:b0b1:c9bc:ec89:e494]) by smtp.gmail.com with ESMTPSA id az8sm34038471wjc.17.2016.03.03.03.07.27 for <info@jcid.nl> (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Mar 2016 03:07:27 -0800 (PST)Date: Thu, 3 Mar 2016 12:07:26 +0100From: Ramon de la Fuente <ramon@delafuente.nl>To: info@jcid.nlMessage-ID: <etPan.56d81aee.66334873.1174c@FzzBook.fritz.box>In-Reply-To: <372a6aae6fcf4240ca8698381aed29ba@mijn.jcid.nl>References: <372a6aae6fcf4240ca8698381aed29ba@mijn.jcid.nl>Subject: Re: [#SFB-667-90513]: SweetLakePHP - Join the fight against email spam!X-Mailer: Airmail (249)MIME-Version: 1.0Content-Type: multipart/alternative; boundary="56d81aee_74b0dc51_1174c"

139

140

141

Mail tester

142

143

144

The practice - domains

145

Thank you!

146

Jeffrey CafferataTwitter handle: @jcid

147