I'm definitely not going to be at Origins (a long way from Japan), but I'd be willing to help with the preliminary judging. Distance could also be a problem when it comes to meeting and discussing the games... I'm not likely to find more than one opponent, but both of us have tried a fair number of icehouse games, so if there are two-player games, we'll be happy to try them out and give some feedback. When browsing through the list, I also noticed that games in preliminary development were included. I think those could be dismissed from the start, which would reduce the number of games to judge by a significant amount. Julien > Send Icehouse mailing list submissions to icehouse@xxxxxxxxxxxxxxxxxxxx To subscribe or unsubscribe via the World Wide Web, visit http://lists.looneylabs.com/mailman/listinfo/icehouse or, via email, send a message with subject or body 'help' to icehouse-request@xxxxxxxxxxxxxxxxxxxx You can reach the person managing the list at icehouse-owner@xxxxxxxxxxxxxxxxxxxx When replying, please edit your Subject line so it is more specific than "Re: Contents of Icehouse digest..." Today's Topics: 1. Re: 2009 pyramid game awards: who wants to help (Ryan Hackel) 2. Re: 2009 pyramid game awards: who wants to help (Bryan Stout) ---------------------------------------------------------------------- Message: 1 Date: Mon, 17 May 2010 09:56:31 -0400 From: "Ryan Hackel" <deeplogic@xxxxxxxxxx> Subject: Re: [Icehouse] 2009 pyramid game awards: who wants to help To: icehouse@xxxxxxxxxxxxxxxxxxxx Message-ID: <20100517095631.19711@xxxxxxxxxxxxxxxxxxxxxxx> Content-Type: text/plain; charset=UTF-8 Over the weekend, I've been thinking about the challenge of selecting finalists from a large list of candidates. If there's one thing I've learned from the IGDC, it's that opening asking the IceFans to play and vote/rate/rank games doesn't generate enough data points to be meaningful. A strong IGDC had about 30 votes, IIRC, and some had as few as 12 votes. And remember that was for a pool of 8-12 contestants, not 50 or more. So here's my counter-proposal: 1) We form a panel of judges, all volunteers. Each judge should be an Icehouse community veteran, or at least experienced in a wide variety of Icehouse games. I think 5-8 judges would be best, but more is better. 2) The candidate games (currently ~50) are divided equally and randomly among the panel of judges. For example, if there were 8 judges for 50 games, then each judge would have 6 games assigned to him, and a few judges would receive a seventh game. A judge should not receive a game he designed. If a judge is unwilling or unable to play a particular game, he can ask his fellow judges to swap it for a different one. 3) Each judge then plays the games assigned to him. He takes notes on the factors of the game that impressed him, and what he found lacking. We could try to codify things like originality, replay value, quality of presentation and the like, but I have learned from professor evaluation forms that sometimes a blank sheet of paper and the freedom to just write an critique can be best. If a judge fails to review his assigned games, then the panel assigns the his remaining games amongst the other judges. If no judge wants to review a particular game, then said game should be dropped from finalist contention. 4) The judges meet and discuss the games they played, and their opinions from such sessions. This can be done face-to-face, if possible, or through any online collaboration tool of choice. The judges then form a consensus of which games deserve to be a finalist, and a list of finalists is announced on the mailing list, and on the IceWiki. Preferably, this should be done a week or two before Origins, if not earlier. This provides adequate time to pyramaniacs to familiarize themselves with the finalists, and prepare to play them. 5) In the weeks leading up to Origins, and during Origins itself, the Icehouse community plays the finalist games, and sends their opinions on the games to the panel of judges. Toward the end of Origins, the results are weighed by the panel of judges, and the winner is announced. If a large pool of candidates is to be parsed down to five or fewer finalists, we need to divide and conquer. I don't think anyone is up to the task of trying out 50 new games, but finding five or ten pyramaniacs to try a few games is likely. By splitting the work up amongst a panel of dedicated judges, each game gets played, and at least one strong opinion is formed for each and every game. Ultimately the judges will decide who the finalists are, and who the winner will be, and the meeting of the minds should be a fruitful one. But the general fan community should be able to weigh in their opinions as well. Fans should have a chance to try out the finalists and influence the judges' decision, and there is no better time to do that than at Origins, the annual Icehouse Convention. If all goes well, we can have a similar contest for a "best of 2010" award, with new or returning judges. But we're just weeks away from Origins right now. If we're going to have enough lead time to choose finalists in time for Origins, we need to get cracking now. ---Ryan -----Original Message----- From: "Bryan Stout" [stoutwb@xxxxxxxxx] Date: 05/14/2010 01:53 PM To: "Icehouse Discussion List" <icehouse@xxxxxxxxxxxxxxxxxxxx> Subject: Re: [Icehouse] 2009 pyramid game awards: who wants to help Yes, please! Any help is greatly appreciated. I have a couple of games I added last year myself, but I don't think we need to worry about such matters during the initial pass. We will discuss it more later, of course. Thanks, Bryan On Fri, May 14, 2010 at 12:16 PM, Ryan Hackel <deeplogic@xxxxxxxxxx> wrote: > I like the idea of a Rookie of the Year award, bestowed upon a fan-made game designed in year X and determined at Origins X+1. > > I also like the Hall of Fame idea. ?Inclusion into the Hall should be difficult but definitive. > > I'd be willing to make a preliminary pass at the 2009 roster to pose my list of finalists. ?I do have a horse in the race (Dectana) so I will leave it up to de-facto organizer Bryan Stout as to whether i should advance my opinion for finalists. > > ---Ryan > > > > -----Original Message----- > From: "Bryan Stout" [stoutwb@xxxxxxxxx] > Date: 05/14/2010 11:16 AM > To: "Icehouse Discussion List" <icehouse@xxxxxxxxxxxxxxxxxxxx> > Subject: Re: [Icehouse] 2009 pyramid game awards: who wants to help > > Thanks for responding, Jeff. ?After saying "Who would like to help?", > I was thinking "I find the sudden silence ver-r-r-r-r-r-ry > reassuring." > > I agree with the idea of "no award" being possible. ?Your wanting an > Excellence award is similar to an idea I was going to propose anyway, > so I might as well mention it now: > > Starting next year, which will be a big year for pyramid games from > the Labs, I propose that the Icehouse community open up an Icehouse > Games Hall of Fame, which will contain games that have stood the test > of time and are still judged excellent. ?Ten years as the minimum time > since invention strikes me as a good amount of time, and both official > Looney Labs games and fan-made games would be eligible. ?The Hall of > Fame will open with a few select classic games, and every year more > can be added -- probably one game, but perhaps more, perhaps none -- > as more games reach the age that they can be considered. > > Back to your thoughts, Jeff. ?I think it would be good to have both > types of awards: a Hall of Fame Award for true excellence over the > years, and an annual Apex Award (or whatever its name will be) for the > best new game(s) of the year -- which could still have No Award if no > good fan-made game was made that year. > > Now, back to *my* original question: would anyone like to help with > selecting the finalists for an award this year, either with an initial > pass, or an intermediate pass? > > Even if you all don't have time to help judge, your thoughts about the > awards are desired. ?Let's get this off the ground! > > Regards, > Bryan > > > On Fri, May 14, 2010 at 7:55 AM, Jeff Wolfe <jwolfe@xxxxxxxx> wrote: >>> 1. ?Who would like to help go through these games? >>> 2. ?How shall we rate them? >> >> When I originally proposed the idea, I envisioned it as an Excellence award. >> I think we should look at it through the lens of "if you could publish one >> game (and only one game), what would it be?" ?I suspect we will not have >> much trouble screening out most of the entries, because they are not yet of >> publishable quality. ?I feel strongly that this shouldn't just be "what was >> the best of the bunch?" ?If they're all crap, there should be no award. >> Even if they're all decent but none of them are great, there should be no >> award. ?I'm inclined to say that we should leave it open to all unpublished >> games, not just ones developed in the past year. ?That would make it harder, >> but not necessarily too much harder. ?Of all the unpublished games out there >> I know about, there are only a handful that I would consider to be in the >> neighborhood of excellent. >> >> So here's the vision: In five years, the award is well established enough >> that Games Magazine (or a similar publication) prints a small article about >> it and publishes the rules for the winner. ?Mainstream exposure. >> >>> 3. ?How many finalists shall we have? >> >> There should be 3 finalists. ?We are asking people to fit these games into >> their convention schedule, so there should be as few as possible while still >> giving them a reason to vote. ?The finals should be a simple selection. ?We >> don't want to make it complicated for people. ?We want to give them every >> reason to participate, and no excuse not to participate. ?If we want it to >> truly be about Excellence, then we should include "none of the above" in the >> voting options. >> >>> 4. ?When should the finalists be determined? >> >> Ideally, it should be a month in advance. ?We probably don't have time to do >> it that soon, so I'd say at least two weeks. ?We need time to prepare >> materials for the public portion of the process. >> >> >> _______________________________________________ >> Icehouse mailing list >> Icehouse@xxxxxxxxxxxxxxxxxxxx >> http://lists.looneylabs.com/mailman/listinfo/icehouse >> > _______________________________________________ > Icehouse mailing list > Icehouse@xxxxxxxxxxxxxxxxxxxx > http://lists.looneylabs.com/mailman/listinfo/icehouse > > > _______________________________________________ > Icehouse mailing list > Icehouse@xxxxxxxxxxxxxxxxxxxx > http://lists.looneylabs.com/mailman/listinfo/icehouse > _______________________________________________ Icehouse mailing list Icehouse@xxxxxxxxxxxxxxxxxxxx http://lists.looneylabs.com/mailman/listinfo/icehouse ------------------------------ Message: 2 Date: Mon, 17 May 2010 10:46:47 -0400 From: Bryan Stout <stoutwb@xxxxxxxxx> Subject: Re: [Icehouse] 2009 pyramid game awards: who wants to help To: Icehouse Discussion List <icehouse@xxxxxxxxxxxxxxxxxxxx> Message-ID: <AANLkTin_UDyCqb-4p3a3o2N53tuXgVyU573--kV_-ARn@xxxxxxxxxxxxxx> Content-Type: text/plain; charset=ISO-8859-1 Your plan sounds good to me, Ryan, except that step 4 needs to be fleshed out more. I think that for an initial pass, it's fine for one judge to eliminate games that obviously aren't worthy of an award, but there should be an intermediate step where remaining games are played by more judges before narrowing down to the finalists. At any rate, I'm in. Who else? Many hands make light work. Bryan On Mon, May 17, 2010 at 9:56 AM, Ryan Hackel <deeplogic@xxxxxxxxxx> wrote: > > Over the weekend, I've been thinking about the challenge of selecting finalists from a large list of candidates. > > If there's one thing I've learned from the IGDC, it's that opening asking the IceFans to play and vote/rate/rank games doesn't generate enough data points to be meaningful. ?A strong IGDC had about 30 votes, IIRC, and some had as few as 12 votes. ?And remember that was for a pool of 8-12 contestants, not 50 or more. ?So here's my counter-proposal: > > 1) ?We form a panel of judges, all volunteers. ?Each judge should be an Icehouse community veteran, or at least experienced in a wide variety of Icehouse games. ?I think 5-8 judges would be best, but more is better. > > 2) ?The candidate games (currently ~50) are divided equally and randomly among the panel of judges. ?For example, if there were 8 judges for 50 games, then each judge would have 6 games assigned to him, and a few judges would receive a seventh game. ?A judge should not receive a game he designed. ?If a judge is unwilling or unable to play a particular game, he can ask his fellow judges to swap it for a different one. > > 3) ?Each judge then plays the games assigned to him. ?He takes notes on the factors of the game that impressed him, and what he found lacking. ?We could try to codify things like originality, replay value, quality of presentation and the like, but I have learned from professor evaluation forms that sometimes a blank sheet of paper and the freedom to just write an critique can be best. ?If a judge fails to review his assigned games, then the panel assigns the his remaining games amongst the other judges. ?If no judge wants to review a particular game, then said game should be dropped from finalist contention. > > 4) ?The judges meet and discuss the games they played, and their opinions from such sessions. ?This can be done face-to-face, if possible, or through any online collaboration tool of choice. ?The judges then form a consensus of which games deserve to be a finalist, and a list of finalists is announced on the mailing list, and on the IceWiki. ?Preferably, this should be done a week or two before Origins, if not earlier. ?This provides adequate time to pyramaniacs to familiarize themselves with the finalists, and prepare to play them. > > 5) ?In the weeks leading up to Origins, and during Origins itself, the Icehouse community plays the finalist games, and sends their opinions on the games to the panel of judges. ?Toward the end of Origins, the results are weighed by the panel of judges, and the winner is announced. > > If a large pool of candidates is to be parsed down to five or fewer finalists, we need to divide and conquer. ?I don't think anyone is up to the task of trying out 50 new games, but finding five or ten pyramaniacs to try a few games is likely. ?By splitting the work up amongst a panel of dedicated judges, each game gets played, and at least one strong opinion is formed for each and every game. ?Ultimately the judges will decide who the finalists are, and who the winner will be, and the meeting of the minds should be a fruitful one. ?But the general fan community should be able to weigh in their opinions as well. ?Fans should have a chance to try out the finalists and influence the judges' decision, and there is no better time to do that than at Origins, the annual Icehouse Convention. > > If all goes well, we can have a similar contest for a "best of 2010" award, with new or returning judges. ?But we're just weeks away from Origins right now. ?If we're going to have enough lead time to choose finalists in time for Origins, we need to get cracking now. > > ---Ryan > > > > -----Original Message----- > From: "Bryan Stout" [stoutwb@xxxxxxxxx] > Date: 05/14/2010 01:53 PM > To: "Icehouse Discussion List" <icehouse@xxxxxxxxxxxxxxxxxxxx> > Subject: Re: [Icehouse] 2009 pyramid game awards: who wants to help > > Yes, please! ?Any help is greatly appreciated. ?I have a couple of > games I added last year myself, but I don't think we need to worry > about such matters during the initial pass. ?We will discuss it more > later, of course. > > Thanks, > Bryan > > On Fri, May 14, 2010 at 12:16 PM, Ryan Hackel <deeplogic@xxxxxxxxxx> wrote: >> I like the idea of a Rookie of the Year award, bestowed upon a fan-made game designed in year X and determined at Origins X+1. >> >> I also like the Hall of Fame idea. ?Inclusion into the Hall should be difficult but definitive. >> >> I'd be willing to make a preliminary pass at the 2009 roster to pose my list of finalists. ?I do have a horse in the race (Dectana) so I will leave it up to de-facto organizer Bryan Stout as to whether i should advance my opinion for finalists. >> >> ---Ryan ------------------------------ _______________________________________________ Icehouse mailing list Icehouse@xxxxxxxxxxxxxxxxxxxx http://lists.looneylabs.com/mailman/listinfo/icehouse End of Icehouse Digest, Vol 60, Issue 13 ****************************************