A Proposal to Stop Spambots

[dragonprobably]

We've all seen those spammed threads, right? The ones filled with Asain characters and other ASCII oddities. I propose to stop this we let people with a big enough post count can only post threads.

 

[Sonereal]

I second this proposal and propose we go one step further. Only people with a big enough post count should be allowed to post posts as well!

 

[NinjaCow64]

We've all seen those spammed threads, right? The ones filled with Asain characters and other ASCII oddities. I propose to stop this we let people with a big enough post count can only post threads.

This would be all good for Colosseum, but it would be terrible for the normal Civ forums. My first thread was basically "Help my game is bugged". I'm sure most people's first thread on this forum is something like that, or something in the strategy forums saying "Help I don't know how to get out of this bad situation". If we remove that feature for new users then it would be really frustrating. Spambots aren't bad, just ignore them. The current restrictions on PMs and such stop the worst of it anyway.

 

[dragonprobably]

Okay, Ninja has a point.

 

[ori]

yeah unfortunately there are no easy solutions to this that do not essentially shut down the forums.

 

[dragonprobably]

I guess we'll just have to live with it.

 

[ori]

oh gods no - we are trying to remove the workload involved in getting rid of them - they are fast. We have been deleting hundreds of these damn threads every day and in some of the forums they'd essentially occupy the first 10 pages by now and drwon out everything if we did not.

 

[Synobun]

A registration security question does well for preventing a lot of the spambots.

 

[ori]

there is one in place, scratch that its essentially 3 - including a captcha and two other questions requiring understanding the questions posted - the problem is that the latter two require a human once and the captcha seems to be circumventable through automatic means as well.

 

[Synobun]

there is one in place, scratch that its essentially 3 - including a captcha and two other questions requiring understanding the questions posted - the problem is that the latter two require a human once and the captcha seems to be circumventable through automatic means as well.

What's worked for me in the past was to make the human question not something that can be easily deciphered by a computer but is easy to figure out if you know English.

It took over 2108 unique spambot attempts for this question to be broken through once: "If there are three people in a room, how many legs are there?"

Since then I've switched to something else and am currently at 185 unique attempts without a single pass. "There are eight people in a room. Let's add three more. Now let's get rid of six. Now, with the remaining people, how many legs are there?"

I typically have to rotate questions every 6-16 months.

 

[ori]

yeah we have these kinds of questions - the problem is that it requires a human once as long as its not changed to automate afterwards.

 

[Evie]

Working on my own site, we've seen more than ample evidence to conclude that modern spambot factories usually have humans around for the express purpose of registering spambot and bypassing defenses if your site generates enough traffic.

 

[Quintillus]

Out of curiosity, I fired up the registration page in a browser I never use (IE), though only so far as to see the questions and not filling out the form. A few thoughts:

- Have we considered the captchas that show 9 images and require you to select all those that show, for example, waffles? The ones I see currently just require entering one word/number, and those have been around for a long time so the bots may have caught up. ReCaptcha in particular is the one I'm familiar with that shows 9 images; unfortunately I haven't found a demo of an actual challenge on their site. But having tested an implementation of it, it's usually relatively easy for humans to solve unless the human intentionally answers incorrectly (at which point it becomes progressively more difficult). And from what I've heard from others at my company, it's been working at reducing their number of bots so far.

- Do the questions change? When I've reloaded the page, I always get the same questions. If they don't change, the bot-writer could just answer them once and they are no longer a challenge. If there is a bank of, say, 40 questions and you get two at once, that may help. It would still be possible for someone to write a bot that identified all 40 questions and answers, but it would require more effort, in particular because they'd have to have a human encounter all (or at least most) of them first before the bot became effective at answering them.

- The other thing that I don't know if we have is rate throttling by IP address, similar to what we have on searches. Supposing that a bot can only answer our captcha correctly 20% of the time currently, if there's no rate throttling, it can try again as soon as it learns it failed, and succeed in a few seconds. If there were even a fairly small timeout for a failed captcha/incorrect answer, it would decrease the bot efficiency and may discourage the bot authors.

The mods have been doing a really good job of deleting these threads - it's been a long time since I saw any of these threads. But if we're at hundreds per day it's definitely worth investigating whether there are technological measures that could hinder the effectiveness at the bots.

I do agree with NinjaCow64 that disallowing new threads from newly-registered members would be undesirable. Quite a few members do join the site by asking a question in a thread (an admin might be able to provide specific statistics on that as well).

 

[filli_noctus]

Would it be possible to implement a system where posts from new members that contain external links require moderator approval or generate an automatic report? Ideally with a white-list of common safe sites to reduce the extra workload such a system would generate and minimise disruption for legitimate posters. Given that spambots raison d'etre is the posting of links it should cause the majority of them to announce themselves directly to staff with their first couple of posts.

 

[ori]

there is already a rather arcane thing doing this (sometimes) for new users with more links making it more likely something ends up in moderation - the problem is that the moderation queue for these is woefully underattended for time reasons and it would definitely be preferrable to not have new user's posts end up in oblivion or take a day or two (or even just multiple hours) to actually post.
Also against the current spam attack this would not have helped at all as they did not post links that vbulletin recognized as such.

 

[Synobun]

I'll admit I am a bit unfamiliar with vBulletin's moderation abilities, but would it be possible to add a new tier of moderators whose only job is to process and inspect new members and their posts? That way they have zero expectation to manually browse the forums and instead only have to enter the mod panel to see if there's anything requiring a human eye for approval, and it'd also reduce the need for such stringent moderation standards to be met since they aren't involved with moderating the actual community.

In other words, I'm asking if it's possible for there to be a type of "Quality Control" user group added. A comparatively large number of members/moderators who just have a checklist to go over with new members and their posts and then approve/reject. CFC certainly has enough trustworthy members who can follow simple instructions, even if they aren't capable of being a moderator to the extent CFC would ordinarily require.

Just an idea, of course. It would certainly address the time constraint issue for moderating new posts.

 

[Valka D'Ur]

I'll admit I am a bit unfamiliar with vBulletin's moderation abilities, but would it be possible to add a new tier of moderators whose only job is to process and inspect new members and their posts? That way they have zero expectation to manually browse the forums and instead only have to enter the mod panel to see if there's anything requiring a human eye for approval, and it'd also reduce the need for such stringent moderation standards to be met since they aren't involved with moderating the actual community.

In other words, I'm asking if it's possible for there to be a type of "Quality Control" user group added. A comparatively large number of members/moderators who just have a checklist to go over with new members and their posts and then approve/reject. CFC certainly has enough trustworthy members who can follow simple instructions, even if they aren't capable of being a moderator to the extent CFC would ordinarily require.

Just an idea, of course. It would certainly address the time constraint issue for moderating new posts.

What you're describing is a basic "admin approval/validation" requirement. That's the system I use for all the forums I run where there's an issue with spammers. Sure, it means that new registrations have to wait until they're checked out, but I'd rather that than have to put up with cleaning out spambots later. We used to have a daily housekeeping chore at my Doctor Who forum until I told the owner flat-out that we had to enable admin validation and other security measures. I just didn't have time to spend on dealing with dozens of these things per day. And if any registration looked like it might be one or the other, I'd run a check on that to see if it had registered on other sites and if so, if there were indications that it was a spambot or a real person. Yes, it takes time to do that, and on a large forum where there may be a lot of these, it's not something the regular moderators have time to do (or the access, for that matter). I don't allow the moderators on my forums to validate new members; that's strictly an admin job.

 

[Synobun]

That's pretty much what I'm referring to except in the format of a specialized member group that only has access to new member post moderating. No topic controls, no public modding, no admin panel, just plain and simple approval/rejection (with subsequent member controls to ban spammers).

 

[Valka D'Ur]

But the validation queue is part of the Admin Control Panel, at least in every kind of forum software I've ever used as an admin!

 

[Synobun]

But the validation queue is part of the Admin Control Panel, at least in every kind of forum software I've ever used as an admin!

Same here, though my experience with premium software is limited. I was hoping that it was doable with vBulletin either natively or with a plugin.