D&C GLug - Home Page

[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]

Re: [LUG] OT: abusive IP address list processing or blocklist for web spam

 

On 17/01/13 00:40, Martijn Grooten wrote:
>
> I've never really looked at comment spam, but to me it always sounds
> like something that is potentially unblockable based on the source, as
> there's no way to distinguish between a human leaving a comment from
> their home PC and a compromised home PC that's part of a
> comment-spamming botnet.

These are generally not compromised end user PCs but hosted servers.

But I think it moot.

If a users PC is compromised we'd rather they cleaned it up before we
let them create an account on our service (given that people we don't
want will then potentially have access to our service).

So I'd have no trouble blocking using the XBL for example, except it
doesn't stop the abuse I want to stop.

The other problem I see trying to stop abuse of web pages is many of the
mobile companies proxy HTTP traffic, in the same way many ISP aggregate
outbound SMTP. So the block lists are polluted with IP addresses of
large proxies (Orange India, Blackberry etc were examples I spotted).

I think both are broken designs as they mix up (convolve) good and bad
origins. e.g. I am left with accept all O2 traffic, or reject all O2
traffic, rather than being able to block troublesome IP address
specifically.

This is the analogous criticism made of the same approach to control
SMTP spam, it moves the spam filtering to the sending ISP, and so if the
ISP is not good at it, or ceases to be good at it, you end up preferring
they hadn't bothered. Meantime proxies complicate delivery and generally
make stuff harder for everyone for no gain.

The pattern to use here if the ISP must intervene is probably the one
used by Virgin (badly) for implementing censorship, which is you allow
all clients to contact all servers directly and you introduce the proxy
when you identify a problem (e.g. suspected spewing of SMTP, or
suspicious levels of web activity, or servers allegedly hosting child
porn for Virgin media).

On a similar note some of this traffic is from Tor. I did wonder if all
of it is, but not sure how I would tell.

> But then perhaps using a botnet for comment spam is a waste of
> resources. It's probably not something that generate a huge income for
> the crooks behind it.

Whether the abuse I see is profitable is not my concern.

Although given the resources they are prepared to dedicate to it I
suspect it must be immensely profitable.

I know the specific abuse they are doing is no longer profitable as
they'll find next time they check robots.txt, but in the mean time I
halved the work load of our server by rejecting one specific user agent
string at 11:30 this lunchtime (who uses Firefox/8.0 these days anyway -
well a handful of over confident Macintosh users - but I only blocked
the Windows XP version with not extras).

-- 
The Mailing List for the Devon & Cornwall LUG
http://mailman.dclug.org.uk/listinfo/list
FAQ: http://www.dcglug.org.uk/listfaq