X-Message-Number: 24117 Date: Thu, 20 May 2004 10:02:40 -0400 (EDT) From: Dan Hitt <> Subject: measuring the effectiveness and consequences of the rating system As most of us are aware, we have a rating system that Kevin put together to control spam-like behavior. He was responding to a specific problem, in which a poster seemed to have cryonet.org in some list of addresses that he sent a mass mailing to two or three times a week. [Just for reference, Randy (http://www.cryonet.org/cgi-bin/dsp.cgi?msg=23980) James (http://www.cryonet.org/cgi-bin/dsp.cgi?msg=23857), Michael L (http://www.cryonet.org/cgi-bin/dsp.cgi?msg=23991), and others have considered the question of whether the content of the messages had any influence on whether the poster's could be understood on spamming. Kevin also discussed this in http://www.cryonet.org/cgi-bin/dsp.cgi?msg=23989.] And even if it weren't an immediate problem, a mature system should be robust against spam-like behavior. Kevin's system allows us to rate each message, and in addition to rating messages off-topic, it provides us with a chance to easily help each other improve by reminding each other to use descriptive subject lines (something Hank made explicit in his recent message http://www.cryonet.org/cgi-bin/dsp.cgi?msg=20448). And some people think it's a good thing to have in place: Steve J (http://www.cryonet.org/cgi-bin/dsp.cgi?msg=24035) likes it, and no doubt lots of others. On the other hand, Kennita and others have pointed out problems with the system (http://www.cryonet.org/cgi-bin/dsp.cgi?msg=24109), and i can't help but worry that we're making cryonet less hospitable (although see Kevin's http://www.cryonet.org/cgi-bin/dsp.cgi?msg=24093 for a contrary view). Of course the choice of what to do is solely Kevin's: he's paid for cryonet both with his money and with is work, and i owe him big-time for making the idea of cryonics as real as breakfast to me, and providing contacts with cryonicists. Posting to cryonet is not a free speech right. But at least as an exercise, i think we can come up with measures as to how effective the system is, and whether and how much it may generate unintended consequences. I have two measures in mind, but i'm sure there are others and better ones. One measure would be the mean time between first appearances of a newcomer. That is, in the one year before the rating system came into being, how many newcomers were there? (I guess at least one, Paul P.) If the number of newcomers is significantly smaller, then i think there's a negative unintended consequence of the rating system. A second measure, this one of the effectiveness, would be the number of posts with poor subject lines (say divided by the total number of posts over some period, or perhaps just from some fixed individual poster). Is that number decreasing? (Measuring the number of off-topic posts is harder to do objectively, i think, but poor subject lines are pretty visible.) What could be done with such metrics? Well, Kennita has privately proposed an alternate system for discouraging spam-like behavior, and it could be tried out (to see, objectively, if it performed better). Perhaps she can post it? In any event, thanks again to Kevin for this substantial piece of work (cryonet), because rating system or no, it has been very beneficial to me, and i hope lots of others in the future. dan Rate This Message: http://www.cryonet.org/cgi-bin/rate.cgi?msg=24117