Your plan sounds good to me, Ryan, except that step 4 needs to be fleshed out more. I think that for an initial pass, it's fine for one judge to eliminate games that obviously aren't worthy of an award, but there should be an intermediate step where remaining games are played by more judges before narrowing down to the finalists. At any rate, I'm in. Who else? Many hands make light work. Bryan On Mon, May 17, 2010 at 9:56 AM, Ryan Hackel <deeplogic@xxxxxxxxxx> wrote: > > Over the weekend, I've been thinking about the challenge of selecting finalists from a large list of candidates. > > If there's one thing I've learned from the IGDC, it's that opening asking the IceFans to play and vote/rate/rank games doesn't generate enough data points to be meaningful. A strong IGDC had about 30 votes, IIRC, and some had as few as 12 votes. And remember that was for a pool of 8-12 contestants, not 50 or more. So here's my counter-proposal: > > 1) We form a panel of judges, all volunteers. Each judge should be an Icehouse community veteran, or at least experienced in a wide variety of Icehouse games. I think 5-8 judges would be best, but more is better. > > 2) The candidate games (currently ~50) are divided equally and randomly among the panel of judges. For example, if there were 8 judges for 50 games, then each judge would have 6 games assigned to him, and a few judges would receive a seventh game. A judge should not receive a game he designed. If a judge is unwilling or unable to play a particular game, he can ask his fellow judges to swap it for a different one. > > 3) Each judge then plays the games assigned to him. He takes notes on the factors of the game that impressed him, and what he found lacking. We could try to codify things like originality, replay value, quality of presentation and the like, but I have learned from professor evaluation forms that sometimes a blank sheet of paper and the freedom to just write an critique can be best. If a judge fails to review his assigned games, then the panel assigns the his remaining games amongst the other judges. If no judge wants to review a particular game, then said game should be dropped from finalist contention. > > 4) The judges meet and discuss the games they played, and their opinions from such sessions. This can be done face-to-face, if possible, or through any online collaboration tool of choice. The judges then form a consensus of which games deserve to be a finalist, and a list of finalists is announced on the mailing list, and on the IceWiki. Preferably, this should be done a week or two before Origins, if not earlier. This provides adequate time to pyramaniacs to familiarize themselves with the finalists, and prepare to play them. > > 5) In the weeks leading up to Origins, and during Origins itself, the Icehouse community plays the finalist games, and sends their opinions on the games to the panel of judges. Toward the end of Origins, the results are weighed by the panel of judges, and the winner is announced. > > If a large pool of candidates is to be parsed down to five or fewer finalists, we need to divide and conquer. I don't think anyone is up to the task of trying out 50 new games, but finding five or ten pyramaniacs to try a few games is likely. By splitting the work up amongst a panel of dedicated judges, each game gets played, and at least one strong opinion is formed for each and every game. Ultimately the judges will decide who the finalists are, and who the winner will be, and the meeting of the minds should be a fruitful one. But the general fan community should be able to weigh in their opinions as well. Fans should have a chance to try out the finalists and influence the judges' decision, and there is no better time to do that than at Origins, the annual Icehouse Convention. > > If all goes well, we can have a similar contest for a "best of 2010" award, with new or returning judges. But we're just weeks away from Origins right now. If we're going to have enough lead time to choose finalists in time for Origins, we need to get cracking now. > > ---Ryan > > > > -----Original Message----- > From: "Bryan Stout" [stoutwb@xxxxxxxxx] > Date: 05/14/2010 01:53 PM > To: "Icehouse Discussion List" <icehouse@xxxxxxxxxxxxxxxxxxxx> > Subject: Re: [Icehouse] 2009 pyramid game awards: who wants to help > > Yes, please! Any help is greatly appreciated. I have a couple of > games I added last year myself, but I don't think we need to worry > about such matters during the initial pass. We will discuss it more > later, of course. > > Thanks, > Bryan > > On Fri, May 14, 2010 at 12:16 PM, Ryan Hackel <deeplogic@xxxxxxxxxx> wrote: >> I like the idea of a Rookie of the Year award, bestowed upon a fan-made game designed in year X and determined at Origins X+1. >> >> I also like the Hall of Fame idea. Inclusion into the Hall should be difficult but definitive. >> >> I'd be willing to make a preliminary pass at the 2009 roster to pose my list of finalists. I do have a horse in the race (Dectana) so I will leave it up to de-facto organizer Bryan Stout as to whether i should advance my opinion for finalists. >> >> ---Ryan