[R] Rating R Helpers

[Matthew Keller]

Also just MHO, but I think it is a good idea, if for no other reason
than it creates an additional incentive for people to respond and to
do so in a thoughtful way. I think the rating systems on some other
boards, and those used at commercial sites such as Amazon, testify to
the fact that people often do care about their ratings, and this, in
turn, tends to bring the discourse to a higher level. Of course, it
would also help newbies get some idea about how to weight responses -
something that takes quite a long time otherwise (longer than 3 months
I'd argue). However, I'm not sure that the ratings should be
restricted to only those who asked the question given that it isn't
only the questioner who has a valid opinion on responses. Why not open
it to everyone (and increase sample sizes)?

Mark - I agree that the quality of the ratings themselves is going to
depend on the questioner/raters. But the noise will wash out over
time, or in stats speak, your mean rating should be unbiased estimate
of your response helpfulness and its variance should decrease as more
people rate your responses :). I guess that assumes independence
though...

In any event, I'd love to see it happen,

Matt



On Nov 30, 2007 1:53 PM,  <markleeds at verizon.net> wrote:
> >From: "Doran, Harold" <HDoran at air.org>
> >Date: 2007/11/30 Fri PM 02:12:36 CST
> >To: R Help <r-help at stat.math.ethz.ch>
> >Subject: [R] Rating R Helpers
>
> Of course, it's just MHO, but I think your suggestion is more work than neccesary because it becomes pretty obvious who the R-experts are once you've been on the list for more than 3 months.
>
> Also, the rating system wouldn't work well
> because the quality of the rating would depend
> on the experience of the person who asked the
> question. A beginner might think a particular answer
> is good while a more experienced person might think
> it was bad.
>
>                                Mark
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> >Since R is open source and help may come from varied levels of
> >experience on R-Help, I wonder if it might be helpful to construct a
> >method that can be used to "rate" those who provide help on this list.
> >
> >This is something that is done on other comp lists, like
> >http://www.experts-exchange.com/.
> >
> >I think some of the reasons for this are pretty transparent, but I
> >suppose one reason is that one could decide to implement the advise of
> >those with "superior" or "expert" levels. In other words, you can trust
> >the advice of someone who is more experienced more than someone who is
> >not. Currently, there is no way to discern who on this list is really an
> >R expert and who is not. Of course, there is R core, but most people
> >don't actually know who these people are (at least I surmise that to be
> >true).
> >
> >If this is potentially useful, maybe one way to begin the development of
> >such ratings is to allow the original poster to "rate" the level of help
> >from those who responded. Maybe something like a very simple
> >questionnaire on a likert-like scale that the original poster would
> >respond to upon receiving help which would lead to the accumulation of
> >points for the responders. Higher points would result in higher levels
> >of expertise (e.g., novice, ..., wizaRd).
> >
> >Just a random thought. What do others think?
> >
> >Harold
> >
> >
> >
> >
> >       [[alternative HTML version deleted]]
> >
> >______________________________________________
> >R-help at r-project.org mailing list
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Matthew C Keller
Asst. Professor of Psychology
University of Colorado at Boulder
www.matthewckeller.com