Revamping the judging process

[Jam Stunna]

I've mentioned this to a few other people, but I want to open it up to the WWYP community as a whole.

I think it would be a good idea to take a look at the judging process. This is the first WWYP that I've seen where the scores were submitted on time, and Tom, Virg and Matt are to be commended on this. However, Tom has already stated that he will not be judging the next contest, and for good reason. Writer's want to write, not just judge. This is the reason why I always declined to judge.

The way it's set up now, we are always in the search for people who are willing to judge, and will do so in a timely fashion. The first-place finisher is almost punished by being excluded from the next contest if they choose to judge. It seems a little unfair. Plus, with the current system we're pretty much gimping the competition by keeping some of the best writers out due to judging commitments. I really want to compete against Wobbles next time, and everyone else.

Now I'm not trying to be antagonistic or anything like that, but I think it would be worthwhile to explore new ways to judge the contests, instead of piling it on Virg everytime .

My suggestion
I used to run a writing contest on ocremix.org, and we had a system where everyone judged everyone else's pieces. This way everyone was invested in the judging process. Everyone had two votes: one for first place, and one for runner-up. You could not vote for yourself of course,I think this would be better, so that everyone can participate and everyone will review each other's work, instead of just two or three people.

This can be adapted to fit in more with the style of WWYP, but I think it might work for us. I'm interested to hear other people's take on this.

 

[demoncaterpie]

My only problem is how long that would take. I mean, it took months for three judges to read everyone's stories, but a dozen of them?

There's also the chance that a person might enter, forget about the contest, and not come back for a while. It's a good idea in theory, but you're also trusting us a little too much

 

[Rapid_Assassin]

That sounds like an excellent idea. If someone enters, and forgets they entered, we could PM them to remind them. If they still don't vote by a deadline, maybe disqualify them from winning or something...

Votes could run on a point based system so there's less potential for ties. Such as 1st place = 3 points, 2nd = 2 points, 3rd = 1 point.

 

[Jam Stunna]

It worked before when we did it. And like Rapid_Assassin said, if you don't submit your scores on time, you're disqualified.

 

[Tom]

Personally, the most important factor in the contests is the critiquing. Scores are nice, prizes are nice too. But I would like to think that people enter so they can get better at writing, and that comes from the experience and the critiques.

As long as everyone allots an ample amount of time to the judging process, so that they actually critique the work and give feedback, then I wouldn't mind seeing a contest organized like this one.

I also think that, to keep the grading altruistic, you shouldn't be able to see the scores that other people give. Maybe we should PM them all to a willing party that did not enter the contests. That way we don't have someone voting for two less than impressive stories just so they have less competition. =/ which would be despicable.

I'd like to see what the other previous judges think about this.

I really do like to write.

 

[Eor]

I don't like this. I like what we have now, and don't see a big reason to change it. I'd rather not have my piece judged by people going against me, or by people I don't believe can write well at all.

 

[Jam Stunna]

I understand your concerns, but what about people like Virg, Matt and Tom that are stuck judging? Is this system fair to them? And what's the point of a competition where the best writers can't enter it because they're the judges?

 

[Eor]

They're not stuck judging, they can always back out. Hell, Virg and Tom where added because other judges backed out. Matt is the only one who has been constant with this, and that's because he wants to. I think this is because you've only been here for the last half of the contest, but we originally had Scav and Redcell as judges, DCP has judged several...if they want to back out they can, there's nothing stopping them. There will be people who will take over.

 

[Blackadder]

I guess I'm kind of on the fence for this one.
It sounds like it could work in theory, but as Demo mentioned, it has a lot of trust placed in it. And some people (Like me) are just no good at saying if a story is good or not. The judges we have in WWYP tend to be people that really know about Tropes and styles and what's good and what isn't.

Point in case, you put up that story recently. About the man and the woman that went to that old childhood hide-out area near the water?
I thought it was fine, but Tom found the subtle and sudden change in the flow of the story. "The colour exploded..." or similar, and you corrected it. That's the kinda eye for writing I think is needed in a WWYP.

Then again, the current way of judging tends to take its time. It's sweet-arse time. I guess your way could work though...
But I still like the current way.

...Or do I!?!?!?

 

[Jam Stunna]

Here's how I see it: A crappy review is still better than no review. If we open up the judging process, we would get more reviews for each piece (most of the entries of this last contest only got the three judge critiques). I'm with Tom, it's really not even about winning to me. The main reason I entered these contests in the first place is because this seems to be the only part of the creative minds forum where someone will read what you write.

Would people really skew their scores? Are the prizes even worth it? I don't think so, but I honestly could be wrong.

My biggest goal with this is to allow anyone who wants to write to enter. Eor mentioned new judges to replace Virg, Tom, etc. But what about when those new judges want to write? If I have to choose between judging and writing, I choose writing ever time. I certainly realize that my system may not be the best one, but all I really want is a way to let writers be writers. Does anyone have any ideas on how we can to that?

 

[Eor]

If those new judges want to stop, then probably an old judge will come in, or somebody new.

I'm going to go all out and say that I hate this idea. I only want to be judged (with comments, mind you) by people who I trust. This is why Uncle Kenny was not allowed to be a judge.

 

[Tom]

i think the least we can do is try it.

write with your power?

 

[Blackadder]

I think the least we can do is try it.

Write with your power?

Well, I guess I'd be up for a test of it. I'm still a little on the fence about it all, but I'm more than willing to support a trial run of the idea.

Test FTW? Test FTW.

 

[Tom]

we could call it the round table! :D

 

[Jam Stunna]

I think we should give it a try as well. Round table sounds like a cool name.

 

[Matt]

I do approve (and so does Crimson King!).

The new and improved WWYP contest is coming to a theater near you very soon!

 

[Blackadder]

I do approve (and so does Crimson King!).

The new and improved WWYP contest is coming to a theater near you very soon!

I approve Of This Message, all it stands for, and the idea of another WWYP soon. In Jam's idea form. Squeeeee!!

Just make it super-awesome.

 

[Virgilijus]

Scav and I are discussing the matter as I type.

If we cannot find an answer...well, that would be expected, wouldn't it?

 

[Jam Stunna]

Scav and I are discussing the matter as I type.

If we cannot find an answer...well, that would be expected, wouldn't it?

Answer to what exactly?

 

[Virgilijus]

Jam, quit pointing out my vagueries!

>:^(

 

[Jam Stunna]

I still love you in a platonic way.

 

[Tom]

I still love him in a... well, in what way can a giraffe love a rubber man?

 

[Virgilijus]

Hopefully not very violently

But back on topic, I'll edit in my ideas after Scav and I talk some more.

 

[McCloud]

So we are to be knights of the round-writing-judging-table?

 

[Scav]

Here's the way I see it, written quickly before my comp runs out of battery.

WWYP is about two things:

1) Getting people to write. This is accomplished by deadlines and with competition.

2) Giving people good, constructive criticism, so that they can improve.

This is why we've always had a small number of judges, and always had a numerical scoring system. And, of course, it's why there's a deadline. The CM is great and all, but us young writers are at our most creative when given a target.

I guess, in a way, that's what WWYP is. A bunch of targets.

A round table is contrary to those goals. Yes, it allows for criticism and spurs writing, but it becomes more casual. We might decide that's what WWYP needs: to be more casual. But right now, a round table judging system is more within the philosophy of the CM.

Virg and I have a few ideas for reworking the scoring. Most of it haves to do with... doing away with the scoring entirely. Giving the judges even more leeway to talk about the strengths and weaknesses of a story. In particular, getting rid of that ugly 10-30 point range, where different scores don't actually mean anything. Ironically, making the scoring more arbitrary will make it LESS arbitrary.

And hopefully, less time consuming and less arduous

More to come. I like what y'all are doing with this, and it thrills me to see the writing community wanting to keep WWYP going.

 

[Jam Stunna]

I agree with pretty much everything said. Let's see what else other people can come up with.

EDIT- I really don't care about the judging. All I want is a system where I can compete against the best writers here, who are usually relegated to judging. I just think it would be amazing to have a contest with Tom, Matt, Scav, Virg, Wobbles, and all the other winners competing. Like an All-Star edition or something.

 

[Eor]

Weren't we going to have a two-division wwyp once? That'd be pretty cool.

 

[raul]

What about this: A two division WWYP, sam prompt for both divisions, but the usual judges have their division and those of us that aren't usually judges have a division. The judges obviously still decide the winner of the non-judge division with whatever system they create and then the non-judges turn around and do the same thing based on the judgement system used for the non-judges to decide a winner.

OK I said judge too mnay times, a summary:

2 Divisions--1 for non-judgers, 1 for usual judgers

Usual judgers create a system of scoring, use it to determine winner of the non-judge division
Non-judgers use same system to pick a winner from the division that is made of up usual judging members.

Thats my idea.

 

[Scav]

The two division WWYP is still a possibility, but it has a different goal

So here's what Virg and I bantered about as a possible judging process.

1) The biggest change: no more scores. At least, not visible ones.

When I've been involved with contests and campus magazines, they do have a scoring rubric, but it serves as a guide for the judges rather than a determiner of a winner. When you submit a story to a magazine, you are *very* fortunate if you get a rejection letter with advice. Because of the volume of entries, the majority get a form letter.

In the Scirg (Vav? Virv? Visc? Scarg? Scurvy?) system, scores still exist. The varied talent levels and differing tastes of the judges mean we want to avoid all three sitting down to annoint a winner. We risk more opinionated judges bending the other two to his ruthless, Mattastic (Mattastecizing?) whim. Scores are now handled behind the scenes.

Judges still have to do writeups, but they are much more free-form. Judges are free to point out anything they like. They're encouraged to touch on the bricklaying aspects -- style, form, spelling, plot, etc. Judges submit their score to that round's Head Judge, who is responsible for tallying them and posting the results.

2) Fewer stories will actually be "scored."

EVERY story will have a critique. However, each judge will select their 5 favorites. The judge will only score these 5.

This mimics another magazine practice. Many publications have "first readers" that go through each round of submissions. They choose, say, 1 in 10 stories, based on what has the chance to "make it." This way, the editors don't have to read 1000 stories a month.

I should stress that we wouldn't be doing this to eliminate "chaff." Every story will be edited and critiqued. What I've noticed in a year and a half of WWYP (has it been that long?) is once you get below the top 5, the usefulness of scoring falls off a cliff. Is a 22 REALLY worse than a 25? Or did one of the judges just go by a gut instinct on style?

Virg said it best: "Telling someone that he's a 3-point-worse writer than someone else feels like lying. Especially when judging is so unscientific..."

3) Drastically shorter writing period.

Virg and I didn't agree on this, per se, but I feel it's important to mention.

Does anyone REALLY do any writing during weeks 2 and 3? The vast majority of stories drop in the first week (thanks to sudden inspiration or "hey look the prompt matches a story I have lying around), or the last week. Those middle two weeks are spent brainstorming. Sometimes. Usually, they're spent posting "ohhh I have a great idea, maybe I'll write it."

No more johns the writing period should be shortened to 2 weeks (giving at least 2 full weekends.) The judging period, 1.5-2 weeks.

4) Option: staggered contests.

What if the official "writing period" was only two weeks, but you knew about the prompt 3 weeks in advance? WWYP winners would be able to choose a prompt AND write in it, because everyone knows it 4-5 weeks before the deadline. It would go like this:

Jan 23: Announce WWYP7 prompt.
Post WWYP7 topic.
Announce WWYP8 prompt (one line description.)

Feb 3: WWYP7 Due. Judging begins.

Feb 10: WWYP7 judging due. Winners announced.

Feb 13: WWYP8 Topic posted.
(Judges, prizes, and detailed description.)
WWYP7 winner announces WWYP9 prompt (one line description).

----

Summary:

1) No more scores
2) Every story gets a critique, but only 5 per judge get ranked
3) 2 week writing period
4?) Staggered prompts.

Each contest has 3 "winners" (orange names), and 1 honorable mention. We can also post a "Judge's Favorite" next to nominated names that didn't make top 3.

Whew! So who likes Scurvy?

 

[Eor]

So, would the wwyp winner not become a judge anymore?

edit: oh, nevermind.

 

[raul]

I like that alot Scav. I think that idea is great for everyone, judges and non judges alike.

 

[Wobbles]

Do I get to select the WWYP7 prompt? I have an interesting idea, but I'd like to run it by other peoples first.

 

[Virgilijus]

Scurvy rocks

I don't see why the previous contest winner can't be involved in the prompt making business

 

[Scav]

I agree. Given the "nobody writes except for the first and last week" bit, knowing the prompt in advance isn't that much of a help

 

[Mr.Lombardi34]

Personally I don't like Jam's idea .

However, I think scurvy is an excellent idea. I would love to see that put into effect.

 

[pokemonmaster01]

I wasn't a fan of the everybody-judge-everybody system, but I think that the one Scav outlined is good. The other idea seemed too inconsistent.

 

[Santini]

Writer's want to write, not just judge.

Hehe, the smash writer made a grammatical error.

I'm just kidding. Sorry. But seriously, as a writing noob who plans on entering one of the next few, I like the current system. I was actually looking foward to criticism from the great writing gods far more skilled than I.

But if al lot of these writing gods don't like it, mine as well just try something new. Trying it once couldn't hurt.

EDIT: While I would love a 4 week writing period, I could easily go for a 2 week period if it meant more contests (i.e. more chances to get lucky and actually win).

 

[Eor]

The two weeks seems better. Maybe if I have a closer deadline I'd stop putting off every idea I have.

And to be fair, as I thought we where going to do with Jam's idea, I was going to test it with a bear-*****-matt story. If I won (or got higher then last), I'd use that as proof. If I got last, I'd of accepted that it was a good system. And now I'm angry, because I don't know where I can slip that noble-winning plot into anything anymore.

 

[raul]

The two weeks seems better. Maybe if I have a closer deadline I'd stop putting off every idea I have.

And to be fair, as I thought we where going to do with Jam's idea, I was going to test it with a bear-*****-matt story. If I won (or got higher then last), I'd use that as proof. If I got last, I'd of accepted that it was a good system. And now I'm angry, because I don't know where I can slip that noble-winning plot into anything anymore.

Why not a special WWYP entitled "Something Funny ***** Matt"? Who ever has the funniest plot for ****** Matt wins.

 

[demoncaterpie]

Why not a special WWYP entitled "Something Funny ***** Matt"? Who ever has the funniest plot for ****** Matt wins.

Eor would win. Contest over.

As of now, I support the shorter contest thing. If we can keep everything to schedule, than I'm cool with it.