These are very good points. Very good points indeed. I wasn't completely comfortable with using averages, since it does have certain implications. Perhaps some kind of "fFavorite" system would work best. Or, maybe it would be more valuable fFor us to plainly state which ones we think are the best of the lot, and hash it out in a more private string of emails until we have arrived at some fFinalists. Speaking of, how many fFinalists do we want? I suggest 4, which seems like "just the right amount." Other have suggested 3 and 5. Maybe the number of fFinalists will be dependent on opinion at the time. On Sat, Jun 5, 2010 at 12:24 AM, Bryan Stout <stoutwb@xxxxxxxxx> wrote: > Hi Scott, > > Thanks for starting the vote collection! I have some suggestions for the > doc: > > 1. Don't say how many will go on to the finals. That hasn't been agreed on > yet. > > 2. I think it might be better for the scores to be hidden -- I know that > seeing others' scores has at least some effect on me, though I try to avoid > it, and I'd like to have mine and others' uninfluenced scores, at least at > first. > > 3. What I would like to see in a table like this is whether a judge *has* > rated any particular game. Seeing how much overall judging has been done > helps us all pace ourselves better, and focus on games that need more > attention. > > 4. It looks like column C is making a sort of average rating. I think that > is actually counterproductive, because the judging algorithm will not choose > based on an average score, but by comparing preferences: whether judge J > rates game X higher than game Y is all that matters, not their absolute > ratings nor the magnitude of the difference in their ratings. So posting an > average rating can be misleading and set up false expectations about who > will make the finals. > > Regards, > Bryan