Seite 1 von 1

Poisioning

Verfasst: 30. Mai 2015, 13:38
von ktsaou
Hi,

I am trying to suggest FireHOL users the blocklists they should use.

In the github repo https://github.com/ktsaou/blocklist-ipsets is my progress so far. There, I have collected all the blocklists I could find and I compared them to understand how they work.

For the moment I have concluded to a Level 1 blocklists which are the ones with minimal false positives. I plan to add these to firehol with a simple command, like 'protection level1'.

Now, I am trying the build Level 2 protection, the Essentials.

Regarding blocklist.de, my main concern is poisoning. Since you collect mainly fail2ban alerts from a community of users, how can you be sure that a group of them are not attackers trying to poison your lists?

To my understanding, you are doing a marvelous job (I do use your list on production systems and your list did help me a lot during an attack I had a month ago - and at the same time I only had just one of my users reporting that he/she is blocked due to your list).

As I see it, you are a lot more effective than openbl.org (check your overlap with openbl_1d (1day) here: https://github.com/ktsaou/blocklist-ipsets#openbl_1d - 90% of openbl_1d appears in your list, but openbl_1d is less than 1% of your list).

Also, it appears your are a lot more effective in attacks apart SSH. Check this: https://github.com/ktsaou/blocklist-ipsets#blocklist_de. Your list matches about 25% of stopforumspam_1d, 35% of bruteforceblocker (ssh), 40% of shunlist, 15% of open proxy lists, 15% of projecthoneypot, etc. You are doing an astonishing job!

However, unlike, for example, openbl.org which controls the attacked hosts (currently 39 of them, according to their site), so poisoning is out the picture, you may at some point be hit by attackers trying to poison your list.

So, my question is this: how do you handle a possible intended poisoning of your list?

Thanks!

Costa

Re: Poisioning

Verfasst: 30. Mai 2015, 19:20
von Martin
Hello Costa,

we check each report automatically, that he have enough loglines and on some service like "mail" where is only a info-Mail for the Provider, we check the Logs to some Types like "droped mail, because spam", or "unknown user" and more.
If the Check failed, we drop the Report and the IP comes not in our System.
On wordpress-bruteforce like, it needs more then 5 POST-Requests in the Logs to /wp-admin or /wp-login.php or /xmlrpc.php
Reports without Logfiles or for Services which we could not reported, was also directly dropped and the IP does not comes in our System.

Randomly, we look at the sended Reports too.
Also we check the Answer from the Providers if he complaint about a report from us. Then we disable the User or Server and inform him, to fix it.

We whitelist Proxy-Server, so that he are only in the List, but we dont send a Report out.
And for VPN-Server, we whitelist them too, but we dont list them.
Also we use the TorProject-Blacklist for TorExit-Nodes and dont list them and also for IPs which was in the SpamHausWhitelist or in the dnwsl.org List.


The rate of false reports or false-positives is very, very slow. From 2.000.000 reports, for 3-4 Complaints we got feedback, that the Report was not ok.
The most false-positives, comes, if a User send us a Report about "badbot"-Service, but it was only 404-Errors. This send us a Mail, that a Report was false. So we check it and contact the User, that he use a false service.
If he dont fix it, the Server will be disabled for reporting or the complete will be blocked and he cannot send us longer Reports.



We forwarding our Data to stopforumspam for the BadBots and Regbots automatically, so that our Data was in the data from stopforumspam too.
The IPs, which we have reported, we forwarding it also to https://www.check-and-secure.com/start/ because that the User can see if there IP infected and he dont received an Report from there Provider.



We have currently over 2.400 Users and from there, was 5 big Hosting-Company which send us Reports and we have a lot of Honeypots. There makes round about 35% of the Reports. The other 65% comes from the other Fail2Ban-Users.
The complete counts of Server which send us there Data was over 8.000 (in the public list was only about 2.500 servers, the other was counted only as 1 Server, because behind there was 100 up to 2.000 Servers).

If you have more Questions, please answer!

Re: Poisioning

Verfasst: 31. Mai 2015, 13:16
von ktsaou
Thanks Martin,

I agree you are doing a great job and from your response I see you track false positives to clean the list.

Do you have a policy regarding reports for Dynamic IP users? As I see it, the biggest problem with false positives, is dynamic IPs, where a bad (or infected) guy disconnects and an innocent new user gets the bad IP.

The problem with false positives and dynamic IPs gets even worst for mobile operators, who NAT hundreds of thousands of mobile devices to just a class-B network. This means that at the same time, an IP can be bad and good.

As I understand it:

- pbl.spamhaus.org
- dul.dnsbl.sorbs.net
- nomail.rhsbl.sorbs.net
- noserver.dnsbl.sorbs.net
- dyna.spamrats.com

provide indications of dynamic IP users. I say indications, because the purpose of these lists is different.
Of course, even these lists, are not complete. For a few countries they are updated, for others they are not.

By the way, additionally to swl.spamhaus.org and list.dnswl.org which you already use, good IP reputation may be obtained from:

- hostkarma.junkemailfilter.com (response 127.0.0.1)
- rep.mailspike.net (responses 127.0.0.16 to 127.0.0.20)

So the question is this: Do you have a policy regarding Dynamic IP users? Do you treat them separately somehow?

Thanks!

Costa

Re: Poisioning

Verfasst: 1. Jun 2015, 20:30
von Martin
Hello Costa,

no we make no different between static or dial-up-IP.
Every IP which was reported are 48 Hours in the Export-List and RBL-Servers listen.
After them, we dont get new Attacks, the IP is not longer listen as blocked.
But a User/Provider can every time us the delist-link under https://www.blocklist.de/en/delist.html and remove there IP earlier before the 48hours was over.
But if a User removed a IP and we got a new Attack, the IP is listen again up to 48 Hours or to the next delist-request (works automatically, no manuell action is needed).