Re: [SAGE] ISP class egress anti-spam filtering

[Brad Knowles <brad@shub-internet.org>]

On 11/16/07, Neil Neely wrote:

>  First I should ask:  Are there any free open source ISP class egress
>  anti-spam filtering solutions out there that anyone has ever utilized?
>  I've seen some links to a handful of vendors doing this in general,
>  but haven't looked too closely.

At AOL, our answer was to transparently proxy all outbound port 25 
traffic from our clients through a set of dedicated servers, and then 
we actively asked all the known blacklist operators to include the IP 
addresses of these machines on their blacklists.

So far as I know, this pretty much completely eliminated the 
complaints about AOL customers that were directly spamming sites out 
on the Internet, because the Abuse Team at AOL is pretty good at 
shutting down real AOL customers who abuse other sites, and this 
eliminated their alternatives.

Of course, this caused certain other classes of problems -- lots of 
people I know of had AOL accounts that they could use as remote 
access around the world, and then submit their legitimate e-mail 
(possibly via their "home" server) while travelling abroad.  When we 
shut down outbound port 25, these people were completely shut out 
unless they could switch to an alternative port -- like the 
Submission protocol on port 587.


When I was working at Skynet SA/NV (the largest ISP in Belgium, with 
about a million customers as of the time I left), our answer was to 
use certain blacklists on our outbound mail servers, and we did not 
auto-whitelist our own customers on these machines.

This meant that if an external operator of a particular blacklist got 
a spam report regarding one of our customers, they could add that IP 
address to their blacklist and we would honor that on our outbound 
mail servers, even though the IP address in question is one of our 
own customers.

>  I believe this problem space is relatively unexplored, and I would suggest
>  to this group of people that this is the next frontier on the anti-spam
>  war that we should collectively work towards.

Agreed.

>  A free open source  relatively easy to deploy tool named spamassassin
>  came along a number of years ago and launched a revolution in ingress
>  spam filtering technologies, and we now have many, many wonderful tools
>  at our disposal that help reduce the flood of bad coming into our
>  networks.

SpamAssassin is one tool.  It's a hammer.

When all you have is a hammer, everything looks like a nail.


I believe that there are other things out there that are not nails, 
and we should not be abusing the one tool we've found so far to treat 
them as nails.

>             What I would love to see is a similar revolution occur with
>  egress filtering.

I think we need to be careful when choosing terms.  When you use the 
term "egress filtering", I believe that many ISPs will think of just 
the sorts of things that have been extensively talked about at NANOG, 
and we won't have any more success with that concept than they have.

You don't want to take a term that is already known and thought 
poorly of, and then apply that to yourself out of ignorance.

>  The problem space is really quite different from ingress filtering,

Agreed.

>                 In ingress filtering you know the destination, so you
>  can simply quarantine those dubious messages for users to review - in
>  egress filtering you don't necessarily have a clue who the sender is
>  (which is a core part of the problem obviously).  What is the best way
>  to deal with this?  There are many other challenges relating to
>  egress filtering, and we collectively desperately need better answers
>  to these problems.

Well, we presumably do know where the message is coming from.  They 
may not be the original sender, but they presumably are our customer 
and we know who they are.

>  The theory I am operating on is that egress filtering could follow the
>  same path as ingress filtering - the introduction of a free open source
>  solution that was relatively easy to deploy started a revolution so that
>  now everyone does it.  If it were easy for ISP operators to truly police
>  their users than we would do it.  If high quality egress filtering
>  were ubiquitous and relatively inexpensive it would become far easier to
>  hold ISP's accountable if they weren't doing this.

At the network layer, the folks on NANOG have put together some 
pretty good tools to make this fairly easy, and yet the vast, vast 
majority of ISPs don't pay any attention what so ever to network 
layer egress filtering.  So, just putting the toolsets out there 
isn't enough.

You've got to have the motivation to use the tools, too.  This might 
be a positive "carrot" motivation, or a negative "stick" motivation, 
or both.


At the higher layers, I agree that this is a different class of 
problems, and while there are some pieces of information we don't 
have because the direction has changed, I think we also need to take 
advantage of the fact that there are other pieces of information we 
do have which we do not have for ingress, and we can use this to our 
advantage.

What those kinds of tools might look like, I have no idea.  I'll be 
interested to see what we can find.

-- 
Brad Knowles <brad@shub-internet.org>
LinkedIn Profile: <http://tinyurl.com/y8kpxu>