Black Ops of Fundamental Defense:Introducing the

Domain Key Infrastructure

Dan KaminskyChief Scientist

Recursion Ventureshttp://www.recursion.com

So. Another DNS Talk.

• There may have been some…consequences…of the last one

• After 15 years of work, the DNS root is signed– Even more surprisingly…– It’s actually a good thing.

A Very Common Scene

[That’s A Strange Username…]

[Heh Wait, That Worked?]

[What’s that in authorized_keys2?!]

Redefining The Possible

• We’ve been trying to authenticate (federate) from one domain to another for years– DNSSEC makes it easy.– This is the power of the Domain Key Infrastructure

• We can’t do this if DNSSEC is hard to deploy– So is it possible to make DNSSEC easy?– Yes.

• But I get ahead of myself.

There Are Four Audiences For This Talk

• The User• The Buyer• The Builder• The Breaker

To The Users

• DKI (Domain Key Infrastructure) will let you know, when you receive a mail from your bank, that it is actually from your bank.– We have been talking about secure email for over a

decade– We apologize. We have failed you.

• We ask you too many questions.• We make awful demands upon your memory• We blame you when things go wrong

• Our failure comes not from lack of trying, but from taking the wrong approach.

To The Buyers

• DKI is going to increase your budget.– Ten years ago, your community spent hundreds of millions

of dollars trying to implement Public Key Infrastructure– On the one hand, it did not work.– On the other, do you think we need strong authentication

any less now, than we did a decade ago?• Information Technology has only gotten more important, not less.

– The question is: Is this pendulum swing, with the DNSSEC root being signed, finally the one that will solve this problem?

– We believe so.

To The Builders

• DKI will make your security products scale– How much stuff have you sold that sits on customer

shelves, because it worked in the lab but failed large scale deployment?

– How many devices have you really been able to ship with certificates?

– How many sacrifices have you had to make in your products, that in the end equated to disabling the security in the first place?• Ahem, devices that don’t even validate the identity of the

SSL peer they’re communicating with

To The Breakers

• You are the most important group here.– People think code is secure until proven otherwise– You know better.

• “Holy crap! Recursion Ventures just made DNSSEC OpenSSH Pre-auth!”– Thank you for noticing. Lets get to work.

Towards Radical Transparency

• Recursion Ventures will be actively supporting an aggressive public audit of all DNSSEC and DKI technologies– Justin Ferguson (jf / @not_me) is auditing LDNS,

NLnet Labs’ excellent DNSSEC library– His report will be released publicly in a few weeks

(by September 1st, 2010, probably earlier)– Initial findings are mostly positive, with some finds• Nlnet Labs is being very supportive of this effort

Some Ground Rules [0]

• No Religion But One– We don’t care about Not Invented Here• Some of the coolest things in this talk were not my idea• We have an Internet to fix

– Bigger than any one person– …or one organization– …or one community– …or one country

Some Ground Rules [1]

– We can’t care about style.• Skype’s Law: The Internet was frozen in 2001. Deal with

it.• Theoretical elegance is great, and there are times where

it’s important to “take a stand”– But it’s gotta work.– And it’s gotta work well. Not just barely.

• Systems that barely work are barely deployed.– 1M SSL endpoints.– Half of them don’t even pretend to be secure.

• Corollary: We can’t care about historical precedent– Historically, we have not achieved success.

Ops Is King

• We do care about Operations– We will not win by calling people Lazy and Stupid

• Great for our egos – defines us as intelligent and industrious – but the customer remains 0wned

– We will not win through moralization• “You’re bad people! You release broken code!”

– We will really not win through regulation• Product liability is the end result of no market forces

differentiating secure code from bad

– We will win the old fashioned way: By delivering a better product to market, as judged by the people who actually have to run with it

Timelines

• 18 months ago, we declared at Black Hat DC:– DNSSEC, as an implementation, is an undeployable

train wreck.– It will get better.

• Today is Yesterday’s Future– Look what we have!

Market Survey(Not A Sales Pitch)

• Open Source Servers– Bind 9.7 (“DNSSEC for Humans”)– PowerDNSSEC (Lazy Signing)– OpenDNSSEC (IXFR Secure Slaves)

• Commercial Servers– Xelerance– Secure64– Infoblox

• Managed Service Providers– Afilias / Proteus– Verisign– Dyn– UltraDNS

Why This Matters

• There is a robust market of companies out to make DNSSEC easy to deploy.– This is a sign of maturity– A bevy of suppliers is the mark of a healthy

industry• A lot of skin in this game• A lot of people out to make DNSSEC operations-friendly

• I’m here to show where it’s all going, on the six-to-eighteen month timeline

1) DNSSEC is not that complicated

• Normal DNS:– Ask a question, get an answer– Ask a question, get a referral• Alice: Jenny’s number? Ask Travis.• Travis: Jenny’s number? Ask Charlie.• Charlie: Jenny’s number? 876-5309

Not Too Different

• DNSSEC– Ask a question, get an answer and a signature– Ask a question, get a referral and a signature• Alice: Jenny’s number? Ask Travis™• Travis: Jenny’s number? Ask Charlie™• Charlie: Jenny’s number? 876-5309™

• Is that it?– Mostly

What’s New

• Referrals now contain new keys– Before, you were just told where Travis was

• “Oh, ask Travis, he’s down by the water fountain”

– Now you’re told how to recognize him• “He’s the guy with the dreadlocks.”

– Who is now the new Chief Hardware Officer of Recursion Ventures

– Welcome aboard, neighbor ;)

– Computers can make stronger identifiers than people can, so we use crypto• But it’s just the same

Is That It?

• There’s magic here and there– Saying “Jenny has no number” has some magic (NSEC/NSEC3)– Records can expire (time exists)– Keys can lead to other keys (KSK/ZSK)

• Some of the magic is optional– More than you’d think

• All of the magic can be implemented in an easily deployable manner– Easiest of course is a managed service or a “one click” device– But, failing that, lets deploy DNSSEC.

• Right now.

A Simple Bind9 Install With A Handful Of Small Zones

Step 1: Change The Port To 50053

Step 2: Launch Phreebird, the Recursion Ventures DNSSEC Server

Step 3: There is no step 3

I ask for keys, I get keys!

OK, maybe there’s a Step 3:.org needs to know

Go To GoDaddy

Click “Manage DNSSEC”

Push the DS

Click OK

Wait 30 seconds

• …

And Look!

It works!(ahem, end to end)

Why This Works

• Phreebird is an online key signer– Like SSH, SSL, and IPsec, it depends on a signing

key being available on demand• Alternative: Magic key sits in a vault

– When a request comes in, it figures out right then and there what response needs to go back out

– Phreebird caches responses, so that 1,000 requests for the same name don’t require 1,000 hits against the encryption engine

Offline Key Signing

• DNSSEC was designed to allow offline key signing– Age: When DNSSEC was designed, processors were

slow and hardware RSA accelerators didn’t exist– Scale: DNSSEC needs to work for both the root –

which needs maximum security – and .com – which needs maximum performance

• Not every server should be run like the root– It’s a big deal that every server can be, though

Reality

• A large part about what made DNSSEC complicated to deploy, was the requirement for offline key signing– Offline keying systems are an order of magnitude

more complicated• PGP

Further Precedent

• We did not invent online keysigning– Obviously, there are the other protocols

• DNSSEC has quietly been including support for online signing for years– Thanks Ben Laurie!

• Some precedent even for DNSSEC servers signing on demand– PowerDNSSEC by Bert Hubert

Phreebird Is A Proxy

• Phreebird uses the existing DNS framework – whatever it may be – and enhances it with DNSSEC responses– No operational impact: Manage your DNS as you always

have, it’s just signing all its responses now– No configuration: There is enough information in any DNS

request to cobble together the appropriate DNS response– Always returns the right answers – even if they change a lot!

• Implementation notes– Today: UDP Port Forwarder– Tomorrow: Linux Mangle Table (won’t even have to change

the port – DNSSEC on, DNSSEC off)

Phreebird is fast

• Thank you, Linux Kernel Developers and LibEvent developers– Niels Provos is a badass– Few hundred lines of code == DNS server that runs

at 60,000 qps on stock fast hardware• So says author of evldns

– We won’t be so fast, since we have a backend to talk to and some records to sign• But we’re pretty clearly faster than almost any name

server we’re put in front of

Phreebird is open

• You’d just reimplement it anyway • Should have code out today, but demos took

precedence over release– Send an email to info@recursion.com if you want a

pre-preview copy with known (and horrifying) bugs– Code should be out in next few weeks (with pen test

report)– Remember, six to eighteen months – this is where

things are going, not where things are at

Phreebird does a lot of things I don’t have time to tell you about today

• There is a lot of obscura in the DNSSEC realm that we’ve been filtering through– How do we handle nonexistent records dynamically?– How do we tunnel trusted records to registries when the

registrars in front of them don’t implement DNSSEC?– How do we manage rollover and expiration?– How do we keep clocks in sync, especially given the chicken-

and-egg relationship between NTP and DNSSEC?• The full version of these slides will contain answers to

these questions, but right now we want to demonstrate the value of this platform conclusively.

Towards TheDomain Key Infrastructure

• Perhaps DNSSEC is easy to deploy. But is it useful?– Distributed authentication is only interesting if it

provides end to end semantics– “My desktop to your server”– Isn’t DNSSEC only designed to secure the links

between recursive name servers, and not endpoints like desktops?

The Basic Mode: The AD Bit

• DNS responses have a bit referred to as AD– Meaning: “The name server you were speaking to

validated the DNSSEC status of this record”• So?– Starbucks, I like your coffee– I don’t trust you to tell me the appropriate

certificate for my bank• No application on earth is going to alter the

user experience based on the AD bit

The Normal End To End Modes

• Chasing– Follow up, from www.foo.com to foo.com, from

foo.com to .com, from .com to root– You only talk to your local DNS server– Problems• Might get blocked by local resolver• Requires lots of round trips

– Fixable using “SuperChase” – tell the local resolver “don’t just give me the immediate, bottom of the chain signature – give me all the information needed”

About Ten Lines Of Code For End To End! (in LDNS)

Tracing: Going Down The Stack

• Chasing: Given a domain, go up to root• Tracing: Given root, get down to the domain– This is basic recursion, the fundamental system by

which DNS servers work normally– Unbound is the nameserver built upon ldns– Unbound can be integrated via libunbound, and is

itself only a few lines of code as well

Tracing in LibUnbound

• Even fewer lines of code!– Create an unbound context (ub_ctx_create)– Add a Trust Anchors file (ub_ctx_add_ta_file)• Containing just the root

– Resolve a domain (ub_resolve)– If result->secure==1, read the output from result-

>data.

Issues With Tracing

• 1) Entirely bypasses the local cache, increasing load on the root and TLD servers– Somewhat acceptable if the cache on the end host

is very long lived– Almost entirely unacceptable if it’s short lived /

flushed for each call– (This was the problem with DNSCurve – if you

used it to achieve end to end trust, the end hosts needed to talk to the roots directly in order to function)

Those Again

• 2) The middleboxen are still a problem– They do bad things to a lot of traffic! What can

you do?– Respect Skype’s Law

Wrapping: DNS over HTTP

• Not complicated – instead of doing DNS over UDP packets that might get intercepted, talk to a custom DNS server that exposes an HTTP endpoint– GET requests w/ Base64 encoded DNS requests– 8 bit clean responses

• Phreebird implements this

Performance Data

• So, how much of a perf hit does DNS take, running over TCP and then HTTP?

Well, it ain’t slower.It might even be faster.

• # DNS over UDP./queryperf -d target2 -s 184.73.1.213 -l 10… Queries per second: 3278.676726 qps

• # DNS over HTTPab -c 100 -n 10000 http://184.73.1.213/Rz8BAAABAAAAAAAAA3d3dwNjbm4DY29tAAABAAE=… Requests per second: 3910.13 [#/sec] (mean)

Could Be Wrong!

• Paul Vixie thinks I am– “Being wrong just means the world is more

interesting than you thought it was.”• Even with a significant penalty, superchase

over HTTP should work reasonably well though

• Recursion Ventures will be hosting a DNS over HTTP service in the Cloud within the next few weeks

Wrapping: Brett Watson’s Observation

• Brett Watson is a really smart Kiwi– He was quite the skeptic about DNSSEC– So was I, so we got along well– His exact quote: “You have to be willing to

separate the content of DNS from the transport of DNS”• This is a fairly profound point

• Led to an interesting concept

X.509

• X.509 normally carries chains to one of a few hundred CA roots, through possibly one of some unknown thousand god-mode Intermediates– This is part of why X.509 didn’t work– Other parts

• Unable to reliably delegate – you can’t get a cert for .domain.com that lets you sign other certs

• Unable to exclude – if Verisign gives you a cert for a domain, so can everyone else

• See 2009 “Black Ops of PKI” for details

X.509 Reloaded

• X.509 could also carry the DNSSEC chain.– SSL already moves X.509– DNS over X.509 over SSL– Take that, hotel miniboxen

• Also, super high performance!

• Implementing this requires:– Extracting all the keys of the full trust chain

• Not too hard – build a trust chain in ldns, then iterate through it extracting all unique keys

– Validating the morass of key material you’re left with• That’s being a bit tricky – requires some rearchitecture

So Adam Langley at Google sent me a private unofficial build of Chrome…

That certificate was self signed……with a DNSSEC chain embedded.

NOTE

• This is an unofficial private build of Chrome!• Google is not at all committed to DNSSEC, DKI,

or X.509 Certificate embedding!• This is just Adam and I seeing what is possible

• And now, I think it’s a good idea to start talking about actual applications.

Where Do We Implement The DKI?

• PhreeShell: Federated Identity For OpenSSH– This was the demo at the start of the talk– Based on the idea that end nodes should validate

identities, not public keys– Today: Identities instead of keys in

authorized_keys2– Tomorrow: LDAP backend (similar to FedSSH)• “Let everyone at support.vendor.com into all the

machines in the vendor.com group”

The Dirty Secret Of Federation

• People have been selling this stuff for years– But nobody’s deploying it in large amounts– OPS IS KING

• Three choices– M to N complexity: Every group has painful and expensive

meetings with every other group– The Risk Of The Kingmaker: One group is trusted by all

others as the identity manager, and that one abuses his role (lets not name names)

– Key Bleed: Everybody has to trust way too many keyholders not to abuse their powers.

The Fourth Path

• The Silent Overseer– DNSSEC: A single root keyholder, incredibly

constrained by both external political constraints and a technical delegation system designed to suppress operational dependency

– DKI: Federation that will actually work

So, do we add ldns/libunbound to each package, one by one?

• Eventually, possibly• But in the short term? To prove value?• On Linux/Unix, SSL is handled via OpenSSL– Specifically, X509_verify_cert– A nice and self contained library call…hmm…

PhreeLoad: Adding DNSSEC Verification To OpenSSL Apps Via LD_PRELOAD

• Prior Work: libval_shim from Russ Mundy @ Sparta– Great work!– Two major differentiators

• 1) Written before the root was signed, so no provisions for chasing a signature down to the root

• 2) Validated the results of a DNS query – which might just be an IP address, attackable via other means

• PhreeLoad operates at a different layer– Given software that’s attempting to achieve end-to-end

security, replace/augment the auth layer with DNSSEC

Curl (without Phreeload)

Curl (With LD_PRELOAD)

Debugging – Just Making Sure This Thing Is Working

The (DELIBERATELY INCORRECT) Schema

• # dig +short _ssl_absolute.www.hospital-link.org TXT

• "'5e0905b0eafd35d59f1b178727d4eaadd06c415d'"

…and if we break DNS…

• _ssl_absolute.www.hospital-link.org. IN TXT 'AAAA05b0eafd35d59f1b178727d4eaadd06c415d'

We break curl.

(When fixed, curl works. So does wget)

All OpenSSL apps means All OpenSSL Apps

• Postfix• Postgres• MySQL• Apache– Want to start working on client certificates that

actually work? Much easier now that we have a signed root

– Welcome to DKI

But Lets Keep The Focus On Browsers

• OpenSSL is great, but browsers don’t use it– IE: CryptoAPI– Firefox / Chrome: NSS

• Also, no LD_PRELOAD on Windows– *whistles innocently*

• What to do?

HTTPS Proxying• PDP @ GNUCitizen put

together httpservers.py, which uses Browser Secure Proxy functionality to inject into HTTPS sessions (assuming appropriate keys are set up)– Based on the work of UZUKI

Hisao, Andres Riancho, and Dave Aitel

• What is sent:CONNECT www.site.com:443 HTTP/1.1

Phoxie: Remote Validation Proxy For Production Browsers

• 1) We can provide the client a custom root certificate• 2) Before the SSL connection is established, we get the

name of the domain the client would like to connect to• 3) For each connection, we can provide a custom root-

signed certificate– Wildcards don’t work nearly as well as they used to– Sorry about that

• 4) The browser will always trust us. We, however, will only proxy if the real server passes SSL validation– Of course, we have PhreeLoad up and running, so SSL validation

includes “DNSSEC is happy”

Browser Lock In IE

Browser Lock In Firefox

Browser Lock In Chrome(Not a Private Build!)

A Neat Trick

• Remember “No Religion”?– There’s something very interesting we can do once

we’ve outsourced validation– It doesn’t involve DNSSEC at all– Just DNS (and even then, barely)

• Wouldn’t it be nice if we could bootstrap trust, with nothing but a domain name?

Self Certifying URLs(Inspired by SFS)

It even works by IP!

It’s just a toy…

• …with a rather strange history– SFS-HTTP: Securing the Web with Self-Certifying URLs– Authors: Michael Kaminsky, Eric Banks– Their model:

http://www.amazon.com:we36hsq8xwgam3tcgzkuay8gy7rb3y8h

– They had a weiiiiiiiird proxy that did caching even if there wasn’t a built in URL

• This has scalability issues– Links work! Securely!– But, man, if the SSL cert is ever lost…the links are dead forever

• Or you get a crappy UI saying “Oh heh! Would you rather try this other cert?”

– Still. COOL.

So, can we get rid of the CA’s?• No. The CA’s are critical, now more than ever.• There are DOMAINS

– www.bankofamerica.comwww.proctorandgamble.comwww.yahoo.com

• There are BRANDS– Bank of America

Proctor And GambleYahoo Inc.

• Brands are a closed namespace.Domains are an open namespace.Certificate Authorities are in the position to overlay a closed, validated namespace on top of the open, delegated hierarchy.– This sounds easy until you realize branding is international and you don’t know

all of them.

Extended Validation…

…has a problem.

• 2008: Adam Barth and Collin Jackson: Beware of Finer Grained Origins

• 2009: Mike Zusman and Alex Sotirov: Sub-Prime PKI: Attacking Extended Validation SSL– “SSL Rebinding”

• Basic concept: EV only protects you against phishing attacks (www.bank--of--america.com). If an attacker actually gets a real www.bankofamerica.com certificate, EV offers no defense– Content from the DV certificate can intermix with content from the

EV certificate– (There’s more, but this is the core fault that makes it impossible to

build a secure EV site.)

DKI Has The Fix:The Exclusionary Property

• DKI is built on DNSSEC.DNSSEC is built on DNS.DNS excludes.– When your DNS domain is registered through eNom, GoDaddy

can’t interfere with it.– When your X.509 certificate comes from Verisign, GoDaddy can

interfere with it – it can issue one of its own.• DKI by its very nature declares the canonical certificate (or

cert set) for www.bankofamerica.com– If that certificate happens to be EV, great!

No other certificate, even if chaining to a proper root, will validate.

What About The Invisible Domains?

• Sites are not servers– One “website” assembles content from sometimes dozens

of servers– If the content retrieved is more complicated than a simple

image, effectively all browsers will require it to be delivered over SSL

– …but not EV-SSL, because there’s no branding associated (the server name is hidden).

• Zusman and Sotirov demonstrated: Compromise the invisible domain, get content in w/ the green bar intact

We Fix Those Too

• First, we make it really amazingly easy to deploy SSL, even on CDNs– There are some issues with virtual hosting for SSL,

requiring crazy certificates that list out all possible domains

– In the DKI model, many domains can share the same certificate because the lookup is against the domain, not the IP

• Second, exclusion works for the invisible domains too– Breaking random CA’s won’t work anymore

But Before We Finish

• I do believe we promised to do something about email.

• Don’t you hate having to deal with locating PGP keys?

• Wouldn’t it be great if you could securely retrieve them via DNSSEC?

PhreeGP: Porting End-To-End DNSSEC Lookups To GPG

Some Problems

• GPG really does not want to trust a random key that comes in over DNS (a reasonable thing )

• PGP signatures don’t appear to actually contain the email address they’re signatures for

• GPG doesn’t let you declare the name of the email address you’re trying to verify on the command line

• PKA support (which involves looking up a key in DNS) is only written into the encrypt path

• Some pretty deep hacking of Enigmail is going to be necessary to make this work

DKIM?

• Scheme for retrieving public keys (not certificates) for domains out of DNS

• Pretty trivial to port end-to-end validation into it, and get domain-to-domain trust

• But there’s no user experience, and it can’t function end-to-end– It’s an anti-spam system, not a replacement for

generic secure email

S/MIME

• Doesn’t have a very good reputation, but…• …it is the most widely deployed secure email

system on the planet– Outlook– Thunderbird– (There’s a S/MIME plugin for Gmail but it doesn’t

verify signatures, so it doesn’t count.)• Just has some key management problems

Lack Of Exclusion Kills S/MIME

• Even if you have the greatest Enterprise CA in the world, you have to hope no other CA issues a free certificate with your name in it– Many CA’s issue email certificates freely, just

depending on domain validation • Can we fix this?– Actually…can we?

APIs

• We were able to hijack OpenSSL because of LD_PRELOAD• We were able to hijack the browsers because of Secure Proxy• How are we supposed to hijack CryptoAPI?– The only extensibility point lets us reject certificates for

being revoked– It doesn’t let us accept certificates that are, say, self-signed– We could write an Outlook extension, but man, that’s

hard.– (There’s another trick, but it’s too hideous to even

mention.)• I guess we’re hosed, right?

This is what a bad idea looks like.(In the Michael Jackson sense.)

API Hooking:Not Just For World Of Warcraft Anymore

• It’s possible to inject your code into any process, to alter how it works– At best, this is usually done for instrumentation– At worst, it’s used for cheating at games or extracting data

maliciously• We’re going to use it to make Outlook better.

– PhreeCAPI: DNSSEC Validation for CryptoAPI– CryptoAPI has a function called CertGetCertificateChain– If it returns True, the chain is valid– The call receives all sorts of context about how its called,

like purpose etc.

See? You can see it in the debugger!While it’s running inside of Outlook!

So, we have this signed email from dan@the-bank.org(Not exactly the most compelling image)

Suppose We Added A DKI Checker

We just added the checker, didn’t put any records into the DKI

Well, there’s the fingerprint of the canonical certificate…

So, lets put this (DELIBERATELY OVERSIMPLISTIC) thing in the DKI

• dan._smpka.the-bank.org. IN TXT '460c1be3f86d39f537864700560f37aef6ce3775'

Now we have mail from the bank, signed by the only key in the world that can sign it.

Ahem.

• Now, perhaps, can we apply more than 15 pixels for security and identity?

This is where EV should go next.

• The user experience around secure email has been minimal because our faith – both in being right, or in being wrong – has been so small

• Now, we can strongly identify email senders• Furthermore, we can strongly identify the

brands associated with those senders• When you receive an email from the bank,

someday soon, you will know it came from the bank.

Conclusion:DKI Is Coming.

• We’ve shown DNSSEC can be deployed in 2 minutes.• We’ve shown that it can achieve end to end security, even

through the weird hotel miniboxes we need to deal with.• We’ve shown how it allows federation of SSH and key retrieval

for GPG.• We’ve shown how it can be integrated into any OpenSSL based

app using LD_PRELOAD• We’ve shown how nicely real world browsers can play with

DNSSEC• We’ve shown how EV, and the Certificate Authority system, still

makes sense in a DKI world – validating brands, for both web sites and emails.