Hello all,

Wave 8 of Exemplar is now open and accepting your recognitions of the amazing things your fellow judges are doing.  This wave will remain open until 11:59 PM PST on January 31st.

If you missed the cut off or had an awesome interaction with another judge at an event/conference/chat room lately, head on over to Judgeapps and enter a nomination now.

Wave 8 will follow the same slot allocation as Wave 7

Recognizing Judge’s Level	L1 Nominations	L2 Nominations	L3 Nominations	Additional “any level” Nominations
Level 2	2	1		2
Level 3	3	2	1	3
RC Extra	4	4	1
GP HJ Extra	1	3	2

The Exemplar Team is hard at work on processing the Wave 7 Nominations, with the expectation to release the nominations in early to mid-December.

Hrm.  Somehow this post doesn’t look long enough.  I should probably talk about something else.

OH!  I know something!

Deferred Nominations

Wave 7 was the second wave where we used deferred nominations; and while the team has been happy overall with how they have turned out, there are a few common points of constructive criticism I’ve heard, so I’d like to address them here.

First, some of our readers might not be aware of what a deferred nomination is. As part of the review process we read and flag nominations that don’t meet our internal guidelines.  Those nominations then go through a second review.  Coming out of that process, a nomination is either accepted as-is, updated following communication with the author, or deferred to next wave. A deferred nomination is not published in the wave it was submitted.  Instead the author gets a single additional slot in the following wave as an opportunity to submit a revised nomination; effectively a “try again” slot.  The goal was to speed up the processing time of nominations by not having to contact the author of every nomination that was flagged.

So, now on to the constructive feedback we’ve gotten on the deferred process.

When the Wave comes out, I can only see my accepted nominations, I can’t tell if the nomination was deferred or something else happened and it disappeared.

Yes.  This is absolutely an issue.  Ultimately we want you to know if your nomination was deferred or not. Updating judgeapps so that you can see the deferred nominations you authored is an eventual goal.  In the meanwhile, we are sending out emails notifying the judges of the deferred nomination along with the nomination text.  These emails do not go out immediately, but do typically go out in the first two days following the release of a wave.

My nomination was deferred, and I don’t know why!

We do not send a specific message with why your nomination was deferred.  Right now, that is a mountain too high to climb. However, we have spent a lot of time describing in general why nominations are deferred in previous blog posts. We highlight nominations that exemplify what we are looking for, and many regions highlight nominations as well.  There are lots of examples out there of what we are/are not looking for.

I’ve responded to every direct emails or IM with an inquiry about why a nomination was deferred and what could be done to make it better.  That might change if everyone sends me an email, but as of right now, if you ask a question, you get an answer.  However, before you do, read your nomination and pretend you are someone else.  Nominations are written with “perfect knowledge”.  This means you know what you are nominating a judge for.  When you say the judge did a “good job” or “does a lot for the community”, you know what exactly what that means; Does the reader of the nomination?

Also, check your spam email folder.  The single largest reason a nomination is deferred, is because we sent an email asking for clarification and didn’t get a response.    

If you had told me ahead of time what the problems were I could have fixed them.

That’s true, and that’s what we are doing in this process.  When a nomination is deferred, it’s not lost.  You receive another nomination slot in the next wave specifically so that you *can* fix it, without holding up the release of the other Exemplar nominations.

Speaking of holding up the release, I sure would like you to get the nominations out faster.  Why even bother trying to contact authors and get any corrected?

This is also a valid point. We do spend a lot of time going over nominations that we believe can be easily fixed without coaching, sending emails out and waiting for replies, and then manually re-entering them into the system.  We could probably eliminate about 3 weeks off the schedule if we skip that entire process and just defer everything that gets disputed.  That sounds great, and might happen in a future wave.  However, the second pass does a few good things.  It gets a second set of eyes on a nomination.  In court, judges tend to be stricter before lunch than after.  I suspect Magic Judges are the same way, and the second pass being a bit apart help remove this phenomena.  There is also value in working with the authors to fix some (but not all) of the nominations. Ultimately, the goal is to have descriptive, significant nominations, and spending some resources on those nominations that are right at the line tends to pay off.   Right now, the process is doing enough good things to justify the delay.  

However, we are looking to make waves faster.  This wave, we are going to reduce the “reply time” from our emails from two weeks to one week.  We have found that a judge typically responses within 2 days or never, so waiting an extra week for an email that will never come doesn’t make much sense.  Next wave, we will also be looking to shave a week off in our internal processes.

Hope that helps answer some questions for you.