Forum blocking legit users

[dyn]

I saw this in forum logs, legit hosts blocked by anti-spam system:

Code: Select all

»Blacklist “dnsbl.othello.ch” »Domain “sympatico.ca”
»Blacklist “dnsbl.othello.ch” »Domain “yandex.ru”

After checking the settings i saw this:

spam-domain_rbl.png

Of course i disabled it by setting it to 0, new values are "Weight: 0 (Default: 4)". Not sure why folks @ dnsbl.othello.ch think Yandex and Sympatico, both gigantic ISPs, are spam hosts, but until they figure it out we won't be using their services anymore. So, if you got blocked because of spam filters it was nothing personal, you can re-register or send email to contact[@]vogon.vg if you require further assistance. This is only in case you are blocked by spam filters, please do not email us if you are a spammer who got his account deactivated.

[SD]

Yandex is almost always present on the board. What is it actually doing? I don't really understand these bots tbh, what do they do? Why are they here?

[dyn]

Those are aimbots looking for players to turn them into cheaters.

OK, to be serious, those are web indexing bots, more info http://en.wikipedia.org/wiki/Internet_bot. There is another side to it ... A lot of times bots are used to build private databases which are then used for various purposes, like intelligence gathering and marketing research. This is pretty lucrative biz when done properly. Data-mining companies can earn lot by reselling this data filtered by some criteria to other companies.

[SD]

So how come we allow them on the board? I take it they don't have bot "accounts" but they somehow manage to be "active users" ?

[dyn]

Forum does nothing to attract them, they visit us by themselves. What forum does is just detecting them and then it lists them. It is possible to block crawlers, but then nobody can find you via search engines or when looking for some answer about games or something random. Forum compares known signatures to ~600 known bots in the database and then blocks certain links to them, like Register and Search.

Check viewtopic.php?f=29&t=2256&p=28739#p28729 for list of all bots we can detect. A lot more than original phpBB installation.

[SD]

Aah. I see them like spiders hanging around the house, not doing any harm just kinda being there and checking stuff out

[dyn]

Yea, but when you think about it -- Google is worth billions and all they did initially was spidering the web. Or so they say. Some are of different opinion and not all content is used for web search, it is also what intelligence agencies use to profile people and so on. It's the entire science around it.