Spam:Spam: Ready, Ready, Fire,Fire, Aim! Aim!Spam:Spam: Ready, Ready, Fire,Fire, Aim! Aim!
APCAUCE / APRICOTKuala Lumpur – 2004
Dave CrockerBrandenburg InternetWorking
<http://brandenburg.com/current.html>
APCAUCE / APRICOTKuala Lumpur – 2004
Dave CrockerBrandenburg InternetWorking
<http://brandenburg.com/current.html>
D. Crocker APCauce/Apricot – KL,200422
Goal and DisclaimerGoal and DisclaimerGoal and DisclaimerGoal and Disclaimer
Spam is complicated and simplistic solutions will be damaging
Email is more complex than people usually realize Spam is a social problem Technical solutions need to follow the social
assessment No single action will eliminate it and nothing will
“eliminate” it
After working on email for 30 years I feel a bit proprietary about it
Spam is complicated and simplistic solutions will be damaging
Email is more complex than people usually realize Spam is a social problem Technical solutions need to follow the social
assessment No single action will eliminate it and nothing will
“eliminate” it
After working on email for 30 years I feel a bit proprietary about it
D. Crocker APCauce/Apricot – KL,200433
What We Will DiscussWhat We Will DiscussWhat We Will DiscussWhat We Will Discuss
The problem
Our reactions to it
Technical environment
Proposals
Making choices
The problem
Our reactions to it
Technical environment
Proposals
Making choices
D. Crocker APCauce/Apricot – KL,200444
Setting the ContextSetting the ContextSetting the ContextSetting the Context
© 1975(!)Datamation
© 1975(!)Datamation
This? Oh, this is the display for my electronic junk mail.
D. Crocker APCauce/Apricot – KL,200455
We We DoDo Have A Problem! Have A Problem!We We DoDo Have A Problem! Have A Problem!
We do not need to cite statistics
It is clear we have a dire problem now!now!
It is clear the situation is getting worse, quickly
It is like moving from a safe, small town to a big (U.S.) city
Nothing Nothing has yet reduced global spam!
We do not need to cite statistics
It is clear we have a dire problem now!now!
It is clear the situation is getting worse, quickly
It is like moving from a safe, small town to a big (U.S.) city
Nothing Nothing has yet reduced global spam!
We must distinguish
Local, transient effects that only move spammers to use different techniques, versus
Global, long-term effects that truly reduce spam at its core
We must distinguish
Local, transient effects that only move spammers to use different techniques, versus
Global, long-term effects that truly reduce spam at its core
D. Crocker APCauce/Apricot – KL,200466
Dangerous LogicDangerous LogicDangerous LogicDangerous Logic
““We have to do something now!”We have to do something now!”(Ignore any side-effects, or dismiss them as minor.)
““Maybe it’s not perfect…Maybe it’s not perfect…but at least we’re taking some action!”
““What have we got to lose?”What have we got to lose?” ““At least it reduces the problem…At least it reduces the problem…
for now.” ““We must replace SMTP…We must replace SMTP…
even though we don’t know what we want to do ““We can do something in the interim…”We can do something in the interim…”
““We have to do something now!”We have to do something now!”(Ignore any side-effects, or dismiss them as minor.)
““Maybe it’s not perfect…Maybe it’s not perfect…but at least we’re taking some action!”
““What have we got to lose?”What have we got to lose?” ““At least it reduces the problem…At least it reduces the problem…
for now.” ““We must replace SMTP…We must replace SMTP…
even though we don’t know what we want to do ““We can do something in the interim…”We can do something in the interim…”
“…“…but this is but this is urgenturgent!!”!!”
D. Crocker APCauce/Apricot – KL,200477
HysteriaHysteria Also Can Destroy Also Can Destroy EmailEmailHysteriaHysteria Also Can Destroy Also Can Destroy EmailEmail
30 years of experience making Internet changes
Risky, difficult, expensive and slow Always has unintended consequences (usually bad) Service providers have highly variable operations Changes to infrastructure require caution!Changes to infrastructure require caution!
Changes need to produce direct benefit Directly affect key problem or directly improve Directly affect key problem or directly improve
serviceservice Orchestrated inter-dependent changes do not work
30 years of experience making Internet changes
Risky, difficult, expensive and slow Always has unintended consequences (usually bad) Service providers have highly variable operations Changes to infrastructure require caution!Changes to infrastructure require caution!
Changes need to produce direct benefit Directly affect key problem or directly improve Directly affect key problem or directly improve
serviceservice Orchestrated inter-dependent changes do not work
D. Crocker APCauce/Apricot – KL,200488
Wheel of Spam (Mis)FortuneWheel of Spam (Mis)FortuneWheel of Spam (Mis)FortuneWheel of Spam (Mis)Fortune
Control of spam Cannot be “surgically” precise Must balance the wheel Needs range of partial
solutions Different techniques for near-
term vs. long-term, except that near-term never is
Heuristics Long lists complicated Complicated Be careful!
Control of spam Cannot be “surgically” precise Must balance the wheel Needs range of partial
solutions Different techniques for near-
term vs. long-term, except that near-term never is
Heuristics Long lists complicated Complicated Be careful!
PoliticalPolitical
LegalLegal
SocialSocial
HumanHuman
AdministrationAdministration
TechnicalTechnical
ManagementManagement DeploymentDeployment
Many FacetsMany Facets
D. Crocker APCauce/Apricot – KL,200499
But What Is Spam, Exactly?But What Is Spam, Exactly?But What Is Spam, Exactly?But What Is Spam, Exactly?
Still no pragmatic, community definition!
Unsolicited commercial or bulk
Anything I don’t want Anything you don’t want me to receive(?)
How can we formulate Internet-wide policies
When we cannot formulate a common, Internet-wide definition?
Still no pragmatic, community definition!
Unsolicited commercial or bulk
Anything I don’t want Anything you don’t want me to receive(?)
How can we formulate Internet-wide policies
When we cannot formulate a common, Internet-wide definition?
Try a pragmatic approach
Focus on core, identifiable characteristics
Ignore the rest, for now
For example, specify1) Type of targeted spam2) How it is occurring3) How the mechanism will
fix the problem4) Dependencies, before
mechanism will work
Try a pragmatic approach
Focus on core, identifiable characteristics
Ignore the rest, for now
For example, specify1) Type of targeted spam2) How it is occurring3) How the mechanism will
fix the problem4) Dependencies, before
mechanism will work
And why do we And why do we stillstill need this slide? need this slide?
D. Crocker APCauce/Apricot – KL,20041010
Different SpammersDifferent SpammersDifferent responsesDifferent responsesDifferent SpammersDifferent SpammersDifferent responsesDifferent responses
“Accountable” spammers Legitimate businesses engaging in aggressive
marketing Need formal rules to dictate constraintsNeed formal rules to dictate constraints
“Rogue” spammers Actively avoid accountability Likely to always have “safe haven” Not always seeking money Need to treat them like virus and worm attackersNeed to treat them like virus and worm attackers
“Accountable” spammers Legitimate businesses engaging in aggressive
marketing Need formal rules to dictate constraintsNeed formal rules to dictate constraints
“Rogue” spammers Actively avoid accountability Likely to always have “safe haven” Not always seeking money Need to treat them like virus and worm attackersNeed to treat them like virus and worm attackers
D. Crocker APCauce/Apricot – KL,20041111
Email is Email is HumanHuman Messaging MessagingEmail is Email is HumanHuman Messaging Messaging
Richly diverse Content Authorship Sources Patterns of use
Spontaneous Serendipitous
Timely Delay hurts
Richly diverse Content Authorship Sources Patterns of use
Spontaneous Serendipitous
Timely Delay hurts
Do not assume precise
Usage scenarios Access Tools Service operations
Do not penalize legitimate users
Or, at least, keep the pain to a minimum
Do not assume precise
Usage scenarios Access Tools Service operations
Do not penalize legitimate users
Or, at least, keep the pain to a minimum
D. Crocker APCauce/Apricot – KL,20041212
Email Points of ControlEmail Points of ControlEmail Points of ControlEmail Points of Control
FilteringFilteringFilteringFiltering
OriginatorUser Agent
Origin MailTransfer Agent External Mail
Transfer Agent
ReceiverUser Agent
Receive Mail Transfer AgentExternal Mail
Transfer Agent
PricePriceAccountabilityAccountability
FilteringFilteringEnforcementEnforcement
PricePriceAccountabilityAccountability
FilteringFilteringEnforcementEnforcement
Gory detail: http://www.ripe.net/ripe/meetings/ripe-47/mailflows.pdfGory detail: http://www.ripe.net/ripe/meetings/ripe-47/mailflows.pdf
D. Crocker APCauce/Apricot – KL,20041313
Proactive Controls – Proactive Controls – PreventionPreventionProactive Controls – Proactive Controls – PreventionPrevention
AccountabilityContent: Sender/authorMail: Sending MTAAccess: Sending provider
Access provider controls Rate-limit Limit outbound ports (eg, SMTP’s 25) Redirect through authorized MTA’s Too intrusive and too much inconvenience for
legitimate senders?
AccountabilityContent: Sender/authorMail: Sending MTAAccess: Sending provider
Access provider controls Rate-limit Limit outbound ports (eg, SMTP’s 25) Redirect through authorized MTA’s Too intrusive and too much inconvenience for
legitimate senders?
D. Crocker APCauce/Apricot – KL,20041414
Proactive Controls – Proactive Controls – PreventionPreventionProactive Controls – Proactive Controls – PreventionPrevention
Charging – Sender pays feeSender pays fee Some vs. all senders How much? Who gets the money?
Enforcement – Laws and contractsLaws and contracts Scope of control – national boundaries? Precise, objective, narrow?
Charging – Sender pays feeSender pays fee Some vs. all senders How much? Who gets the money?
Enforcement – Laws and contractsLaws and contracts Scope of control – national boundaries? Precise, objective, narrow?
D. Crocker APCauce/Apricot – KL,20041515
LegalLegalLegalLegal
Constituencies in the debateBusiness providers: Legitimate needDirect marketing: Legitimate need (?)Service providers: Reduce
complaints/costOutraged consumers: Reduce hassles/cost
Core social principles Careless laws alter society and defeat the goal Consider complexity of English plug/socket…
Constituencies in the debateBusiness providers: Legitimate needDirect marketing: Legitimate need (?)Service providers: Reduce
complaints/costOutraged consumers: Reduce hassles/cost
Core social principles Careless laws alter society and defeat the goal Consider complexity of English plug/socket…
D. Crocker APCauce/Apricot – KL,20041616
AccountabilityAccountabilityAccountabilityAccountability
LevelsLevels
1. Identity A label What the label refers to
2. Authentication Validate the identity Who is doing the
validation
3. Reputation Predict behavior, using
history & opinion of others
LevelsLevels
1. Identity A label What the label refers to
2. Authentication Validate the identity Who is doing the
validation
3. Reputation Predict behavior, using
history & opinion of others
Real world Real world systemssystems
Friends, colleagues
Third-party service Trust the rating service? Like credit-reporting
Yourself(!) E.g., pre-authorize email
receipt, after purchase
Real world Real world systemssystems
Friends, colleagues
Third-party service Trust the rating service? Like credit-reporting
Yourself(!) E.g., pre-authorize email
receipt, after purchase
D. Crocker APCauce/Apricot – KL,20041717
AuthenticationAuthenticationAuthenticationAuthentication
Channel chain-of-Channel chain-of-trusttrust
Trust via each handling entity
SSL/TLS PPP login SSH
Works well for point-to-point
Channel chain-of-Channel chain-of-trusttrust
Trust via each handling entity
SSL/TLS PPP login SSH
Works well for point-to-point
Object origin Object origin validationvalidation
Message validated Channel is irrelevant S/MIME, PGP
Works well for store-and-forward
Object origin Object origin validationvalidation
Message validated Channel is irrelevant S/MIME, PGP
Works well for store-and-forward
D. Crocker APCauce/Apricot – KL,20041818
Security ModelsSecurity ModelsSecurity ModelsSecurity Models
ObjectObject ChannelChannelSecureSecure
MailMailMailMail
SecureSecure
MailMailSecureSecure
MailMail
MailMail
MailMail MailMailMailMail
MTAMTA
MTAMTAMTAMTAMTAMTA MTAMTA
MTAMTAMTAMTASecureSecure
SecureSecureSecureSecure
MTAMTASecureSecure
MTAMTAMTAMTA
SecureSecure
MTAMTAMTAMTAMTAMTASecureSecure
MTAMTASecureSecure
D. Crocker APCauce/Apricot – KL,20041919
Reactive Controls – Reactive Controls – FilteringFilteringReactive Controls – Reactive Controls – FilteringFiltering
DetectionSource: Good/Bad senderDestination: Honey pot, attracts
spammersContent: Advertising, pornographyAggregate traffic: Massive bulk mail flow
Action Divert, delete or return Label and deliver Notify administrator
DetectionSource: Good/Bad senderDestination: Honey pot, attracts
spammersContent: Advertising, pornographyAggregate traffic: Massive bulk mail flow
Action Divert, delete or return Label and deliver Notify administrator
D. Crocker APCauce/Apricot – KL,20042020
Source InformationSource InformationSource InformationSource Information
TypeType MeaningMeaning Current Current ValidationValidation
MTA IPMTA IP SMTP client Net validates addressNet validates address
EHLO DomainEHLO Domain SMTP client DNS match actual IPDNS match actual IP
Provider IPProvider IP Site of SMTP client DNS in-addr.arpaDNS in-addr.arpa
Mail-FromMail-From Bounces address NoneNone
FromFrom Author NoneNone
SenderSender Posting agent NoneNone
ReceivedReceived Handling sites NoneNone
TypeType MeaningMeaning Current Current ValidationValidation
MTA IPMTA IP SMTP client Net validates addressNet validates address
EHLO DomainEHLO Domain SMTP client DNS match actual IPDNS match actual IP
Provider IPProvider IP Site of SMTP client DNS in-addr.arpaDNS in-addr.arpa
Mail-FromMail-From Bounces address NoneNone
FromFrom Author NoneNone
SenderSender Posting agent NoneNone
ReceivedReceived Handling sites NoneNone
D. Crocker APCauce/Apricot – KL,20042121
Proposals – Proposals – Out of BandOut of BandProposals – Proposals – Out of BandOut of Band
Legal efforts define Common use of term “Spam” Requirements when sending classes of mail Remedies for violations
Administration Exchange filtering rules Exchange incident (abuse) reports Are abuse desks used, useful?
Legal efforts define Common use of term “Spam” Requirements when sending classes of mail Remedies for violations
Administration Exchange filtering rules Exchange incident (abuse) reports Are abuse desks used, useful?
D. Crocker APCauce/Apricot – KL,20042222
Proposals – Proposals – AuthenticAuthentic ChannelChannelMTA RegistrationMTA Registration
Proposals – Proposals – AuthenticAuthentic ChannelChannelMTA RegistrationMTA Registration
Presumed-AuthorPresumed-Author MTA IP registered with
Mail-From domain EHLO domain
Registration in DNS New record, or TXT Simple authentication, versus “policy”
Proposals RMX, SPF, LMAP, DMP, DRIP, FSV, Caller-ID
Presumed-AuthorPresumed-Author MTA IP registered with
Mail-From domain EHLO domain
Registration in DNS New record, or TXT Simple authentication, versus “policy”
Proposals RMX, SPF, LMAP, DMP, DRIP, FSV, Caller-ID
Provider NetworkProvider Network MTA IP registered
with net hosting it
Registration in DNS in-addr.arpa New record
Proposals MTA Mark, SS
Provider NetworkProvider Network MTA IP registered
with net hosting it
Registration in DNS in-addr.arpa New record
Proposals MTA Mark, SS
D. Crocker APCauce/Apricot – KL,20042323
Proposals – Proposals – AuthenticAuthentic ContentContentCertify the authorCertify the author
Proposals – Proposals – AuthenticAuthentic ContentContentCertify the authorCertify the author
Classic Classic AuthenticationAuthentication
S/MIME – OpenPGP Classic public key
service Message content only
Challenge-Response Block until response to
challenge received Patented
Classic Classic AuthenticationAuthentication
S/MIME – OpenPGP Classic public key
service Message content only
Challenge-Response Block until response to
challenge received Patented
Good-GuyGood-Guy Validate identity Certify reputation Proposals
Challenge-Response Project LUMOS TEOS DomainKeys
Good-GuyGood-Guy Validate identity Certify reputation Proposals
Challenge-Response Project LUMOS TEOS DomainKeys
D. Crocker APCauce/Apricot – KL,20042424
Evaluating EfficacyEvaluating EfficacyEvaluating EfficacyEvaluating Efficacy
Adoption Effort to adopt
proposal Effort for ongoing use Balance among
participants Threshold to benefit
Impact Amount of Net affected Amount of spam
affected
Adoption Effort to adopt
proposal Effort for ongoing use Balance among
participants Threshold to benefit
Impact Amount of Net affected Amount of spam
affected
Robustness How easily
circumvented
Test scenarios Personal post/Reply Mailing List Inter-Enterprise
Robustness How easily
circumvented
Test scenarios Personal post/Reply Mailing List Inter-Enterprise
Look with a Look with a veryvery critical eye! critical eye!
D. Crocker APCauce/Apricot – KL,20042525
Evaluating OA&MEvaluating OA&MEvaluating OA&MEvaluating OA&M
Operations impact on… Adopters of proposal Others
Internet scaling – What if… Used by everyone Much bigger Internet Individual vs. Group use
System metrics Cost Efficiency Reliability
Operations impact on… Adopters of proposal Others
Internet scaling – What if… Used by everyone Much bigger Internet Individual vs. Group use
System metrics Cost Efficiency Reliability
Look with a Look with a veryvery critical eye! critical eye!
D. Crocker APCauce/Apricot – KL,20042626
SummarySummarySummarySummary
Spam is a complicated topic It needs to be treated with all due respect
Many factors, proposals, and constituents
Complicated considerations and effects
On the Internet, interim never is Deploy strategic solutions
Spam is a complicated topic It needs to be treated with all due respect
Many factors, proposals, and constituents
Complicated considerations and effects
On the Internet, interim never is Deploy strategic solutions