apache.txt is mainly static

[ktsaou]

Hi,

I believe the apache blocklist is mainly static data.

Take a look at this chart:

http://iplists.firehol.org/?ipset=block ... apache#age

About 85% of it does not expire. Never.

Check also the list's retention policy at:

http://iplists.firehol.org/?ipset=block ... #retention

Most of the IPs passing through the list do expire in logical durations.

But not this 85% that is currently listed.

Check also the age of IPs in all.txt here:

http://iplists.firehol.org/?ipset=blocklist_de#age

Around half of it is also stale.

On Jan 6, 2016 the following were added to apache.txt (and are still listed).
Here in CIDR notation (I grouped them to CIDRs to limit the size of the post):

Code: Alles auswählen

5.9.25.65
5.9.25.66/31
5.9.25.68/30
5.9.25.72/29
5.9.25.80/30
5.9.25.84/31
5.167.64.0/21
46.118.112.135
50.7.240.10
77.109.139.87
82.221.99.224/28
91.215.155.32
95.134.130.182
95.141.17.0/24
95.143.192.159
96.44.142.250
96.47.224.42
96.47.225.0/24
108.62.56.0/21
113.212.69.0/24
113.212.70.0/24
173.234.225.0/24
173.234.226.0/23
176.9.219.38
176.100.75.27
176.221.42.32
178.137.16.0/24
188.95.234.6
188.143.232.1
188.143.232.2/31
188.143.232.4/30
188.143.232.8/29
188.143.232.16/28
188.143.232.32/27
188.143.232.64/26
188.143.232.128/25
188.143.233.0/24
188.143.235.21
192.251.226.0/25
192.251.226.128/26
192.251.226.192/27
192.251.226.224/28
192.251.226.240/29
192.251.226.248/30
192.251.226.252/31
192.251.226.254
194.71.223.0/24
194.71.224.0/23
195.254.134.10
195.254.134.194
216.151.130.0/24
216.151.137.0/24
216.151.138.0/24
216.152.249.0/24
216.152.252.0/24

The above matches 9.010 unique IPs.
8.989 of these IPs were added to all.txt on Mar 8, 2016.

My data show that on Mar 8, all.txt was somehow reset - at 07:10 GMT was listing 70.000 unique IPs, then at 07:42 went down to 55.000 IPs and a few minutes later went back to 70.000 IPs.
So, most probably these data were listed in all.txt before Mar 8.

Similarly, on Jan 6, apache.txt was reset - at 17:28 GMT was listing 18.500 unique IPs, then at 17:56 was listing 614 IPs, then at 18:14 was back to 18.500 IPs.
So, most probably these data are there for a long time before Jan 6.

To my understanding, apache.txt is mainly static data.
It does not come from a fail2ban process.
You have blacklisted all these IPs permanently.

Regards,

Costa

[Martin]

Hello,

only the following data was static, permanently added only on the all.txt, mail.txt, apache.txt (since years ago):

Code: Alles auswählen

            $input2 .= netz('188.143.232.0', '188.143.233.255'); # leonburgnet ru
            $input2 .= netz('178.137.16.0', '178.137.16.255');
            $input2 .= netz('95.141.17.0', '95.141.17.255');
            $input2 .= netz('108.62.56.0', '108.62.63.255'); # nobistech keliix06
            $input2 .= netz('173.234.225.0', '173.234.227.255'); #nobistech keliix06
            $input2 .= netz('5.9.25.65', '5.9.25.85'); # pixray.com
            $input2 .= netz('113.212.69.0', '113.212.69.255'); # xeex.in
            $input2 .= netz('113.212.70.0', '113.212.70.255'); # xeex.in
            $input2 .= netz('216.151.137.0', '216.151.137.255'); # xeex.in
            $input2 .= netz('216.151.138.0', '216.151.138.255'); # xeex.in
            $input2 .= netz('216.152.252.0', '216.152.252.255'); # xeex.in
            $input2 .= netz('216.152.249.0', '216.152.249.255'); # xeex.in
            $input2 .= netz('216.151.130.0', '216.151.130.255'); # xeex.in
            $input2 .= netz('82.221.99.224', '82.221.99.239');  # Fake Tor-Exits Burratino.net
            $input2 .= netz('96.47.225.0', '96.47.225.255'); #sysop@iptelligent.com
            $input2 .= netz('5.167.64.0', '5.167.71.255'); # ertelecom.ru
            $input2 .= netz('194.71.223.0', '194.71.223.255'); # errsy.com
            $input2 .= netz('194.71.224.0', '194.71.224.255'); # errsy.com
            $input2 .= netz('194.71.225.0', '194.71.225.255'); # errsy.com
            # open proxy
            $input2 .= '176.9.219.38'."\n";
            $input2 .= '46.118.112.135'."\n"; # macht vod.com.au referer spam
            $input2 .= '50.7.240.10'."\n";
            $input2 .= '96.47.224.42'."\n";
            $input2 .= '96.44.142.250'."\n";
            $input2 .= '195.254.134.10'."\n";
            $input2 .= '195.254.134.194'."\n";
            $input2 .= '188.95.234.6'."\n";
            $input2 .= '188.143.235.21'."\n";
            $input2 .= '95.134.130.182'."\n";
            $input2 .= '95.143.192.159'."\n";
            $input2 .= '176.100.75.27'."\n";
            $input2 .= '176.221.42.32';

and 192.251.226.0 - 192.251.226.255 #blutmagie

All other was reported again and again.
The permanently added cannot be removed, because with some of them, we had an dispute and so, these was the only solution, which was acceptably for both sides, to dont received reports and dont fix/stop it.

Please remember, that on the time, the list was generated an a other Process/Cleanup blocks the Tables or the system has too heavy load, it can breakable and not complete. but this is very rare.

[ktsaou]

Martin, I am trying to score all the lists.

For this reason I need some reference lists. Yours (and a few more) are perfect to start with, but only if they do not contain any static data.

Is it possible for you to put all these static data on another IP list, named 'static' or something similar?

If you do this and leave all static data out of your other lists, I could use your data as the basis for scoring other lists.

Thanks!
Keep up the good work!

Costa

[ktsaou]

Your static list is this:

Code: Alles auswählen

5.9.25.65
5.9.25.66/31
5.9.25.68/30
5.9.25.72/29
5.9.25.80/30
5.9.25.84/31
5.167.64.0/21
46.118.112.135
50.7.240.10
82.221.99.224/28
95.134.130.182
95.141.17.0/24
95.143.192.159
96.44.142.250
96.47.224.42
96.47.225.0/24
108.62.56.0/21
113.212.69.0/24
113.212.70.0/24
173.234.225.0/24
173.234.226.0/23
176.9.219.38
176.100.75.27
176.221.42.32
178.137.16.0/24
188.95.234.6
188.143.232.0/23
188.143.235.21
194.71.223.0/24
194.71.224.0/23
195.254.134.10
195.254.134.194
216.151.130.0/24
216.151.137.0/24
216.151.138.0/24
216.152.249.0/24
216.152.252.0/24

It matches 8.754 unique IPs.

[Martin]

You can see the new IPs/Attacks in our graphs under:
https://www.blocklist.de/en/statisticsgraph.html
The manually added IPs are not in the graph.

A new list is not possible, because then, our Users was not longer protected about these bad IPs.

Yes, there was only at the all.txt, mail.txt and apache.txt added.