Upload
ashley-carroll
View
225
Download
4
Embed Size (px)
Citation preview
Exchange deployment at CERN Exchange deployment at CERN and new ideas for SPAM fightingand new ideas for SPAM fighting
Michel Christaller, Emmanuel Ormancey, Alberto Pace
CERN Mail infrastructureCERN Mail infrastructure
14 Servers 8 “Mailbox” stores, 2 Public Folder Stores, 2 Front-end servers, 2 Spare
IMAP (secure), POP (secure), MAPI and secure HTTP MAPI with Outlook on Windows/Mac MAPI open (in theory) outside CERN using Microsoft ISA Server IMAP and POP work with almost any client HTTP works with any Web browser Collaborative tools available with MAPI and HTTP
Office XP recommended for collaborative features Not possible to switch Outlook 2000 from IMO to CW Allows multi protocol (pop, imap, mapi, webdav) All information stored at server level, no more PST file problems
Office 2003 being evaluated MAPI over HTTP Seamless connected/disconnected/online/offline feature Optimized for slow network connections
Migration overviewMigration overview Nothing changes for the user
Legacy ServerNew Server
user.mailbox.cern.chMail Server
Mail ClientMail User
The server is replaced,The server is replaced,Nothing changes for the clientNothing changes for the clientAdditional interfaces availableAdditional interfaces available
imap
mapihttp
imapspops
webdav
Migration: what is doneMigration: what is done
User are invited to migrate by filling a migration form The password is kept on the new service and synchronized
with the windows password Unresponsive users are forced to migrate and the password is
reset
All folders and mails are copied from the old servers to Exchange
Mail Forwarding configuration is kept if any Mailbox is not functional during at most 10
minutes, while rebuilding configuration files
Migration WorkflowMigration Workflow
Migration Form
Mailbox migrated
Keep password typed in migration form
Nice and Mail password synchronized
Mailbox migrated
Password reset
Nice and Mail password synchronized
“Ask for migration”
Accept / Delay
FormReminder Mail (3) Accept
After n remindersForce migration
No answerClick on link
Click on link
Migration StatusMigration Status 10000 Exchange Users, 14774 Total Only inactive and a few “non cooperative” users remaining Cleanup: More than 700 Mail accounts deleted following user approval
Current statusCurrent status
1 year of production Exchange software stable and scalable No major disaster, only normal hardware failures, solved
in operational delays Usage: 50 % Outlook XP, other 50 % with IMAP, POP and
HTTP access 1’000’000 Incoming mails per week, 30% is Spam
Next step, currently in testNext step, currently in test
Move SMTP Gateways to Exchange Implement automatic anti flood system
Any server, sender or recipient sending or receiving more than 500 mails in 5 minutes will be banned (numbers to define)
Only solution to improve quality of service, and reduce impact of loops on “regular” mails
Migrate Mailing lists system from majordomo to Exchange You will hear about this next year
Spam Fighting at Cern
Evolution
Legacy systemLegacy system
Sendmail checks: Lists of banned IP addresses, domains, subject,
senders or recipients, and words Header “consistency” tests (i.e. message id format)
Mail rejected if identified as Spam Heavy manual work:
Update local banned lists from abuse reports Remove entries when users report false positive
rejections
Current serviceCurrent service
Existing market products were reviewed: Technology too young Results are not accurate Missing a per user basis configuration
While the market consolidates … CERN developed his own Anti-Spam filter
Based on SpamAssassin Less effort than running after immature commercial
technology Now in production for 1 year Easy to modify and update detection techniques
How it worksHow it works
The anti-spam filter calculates the probability for a message to be spam Regular expressions “Intelligent” content parsing Statistical heuristics (Bayesian Filters)
The user sets the threshold at which he wants spam to be rejected Rejected message can be seen by the user
(CERN Spam folder) Per user configuration (!) Allows rejection of foreign languages mail
(Chinese, Korean, Russian, Japanese, Arabic, etc …)
User configurationUser configuration
Filtering levelFiltering level
Language-based rejectionLanguage-based rejection
EfficiencyEfficiency
Roughly 160 000 Incoming mails per day Spam filter detects from 25% to 35% as spam
EfficiencyEfficiency
False positives are very low Except for commercial lists (spam that you want) White lists at user level can be configured to prevent this
Good spam detection Statistics are hard to build Standard mailbox filtering statistics:
30 to 40 Spams filtered per day 1 or 2 Spams still go to the INBOX per week
Could still be improved with some optimization Not enough for some users with “public” email
address Old email address or published email address are more
targeted for Spam
Current evolutionCurrent evolution
Spammer techniques always follow anti-spam techniques
New detection mechanisms work only for a few months
Needs a full time work to have a constantly “up-to-date” filter
Only viable long term solution is to accept only mails from people you know: ICQ (and other messenger systems) already have this feature Accept only messages from people in my contact list Adding someone to the contact list requires validation
New feature (in test)New feature (in test) Good Mails not matching the
user’s white list are quarantined Mail is sent to sender requiring
action to validate himself
Once validated, sender is added to white list, mails are moved back to Inbox
Move to Inbox.Quarantine
Quarantine level
Inbox
Move to Cern Spam
Delete
Spam Filter level
Delete if evident spam level
Mail to sender for validation
What’s next ?What’s next ? Join forces against Spam
Share rules, regular expressions patterns and Bayesian statistics dictionary with other organizations
Central antispam configuration with Live Update like antivirus definitions is the solution. Therefore …
Long term goal: use a commercial product Like for antivirus products, only a full time working team
will provide up-to-date filters
In addition …In addition …
Within Exchange, mail is authenticated Not possible to forge To: or From: fields Delivery and Read receipts are reliable A platform for workflow application
Extend this towards the internet Mail messages digitally signed with guaranteed origin
and dates (See my presentation on PKI this Thursday)
ConclusionConclusion
Users are profiting from the new collaborative services Shared calendar (already used by 1500 accounts) Tasks, workflow Web and webdav interfaces
Spam is a serious issue Towards accepting only authenticated/verified mail There is a future for commercial products in this area