(Note: The L3 checklist further changed. This article has only historical value.)

Hi all,

As has long been overdue, the level 3 checklist is modified to better fit the New World Order.

First of all, here’s a summary of the changes. You can find detailed explanations for the changes below, and the new full checklist at the bottom of this post.

• Review check reduced to 6 from 10, but excludes certification reviews
• Certification can be L1->L2
• Region requirements to recommendations removed
• Tweak to recommendation phrasing mandating examples and clarifying co-authoring
• Self-review must grade each Quality
• English requirement reduced to team leading and reading official documents
• Team Leader Certification can replace Team Leader Check
• Community requirement stays

The new checklist applies to all L3 candidates who haven’t yet submitted their checklist. If you’ve submitted yours it’s evaluated using the old criteria.

Rationale for the changes

The main motivation behind the New World Order is recognizing that a judge’s career and progression are modeled after our primary mission which is to judge Magic events. Levels 1, 2 and 3 correspond to an increasing scope of tournaments you’re bound to judge. Level 1 is dedicated to judging of Regular in-store events. Level 2 is about Competitive events and community interactions. Level 3 is about large events and beyond. Why beyond? Level 3 is the last achievement purely dedicated to judging events; after that the path branches out as a judge wants to specialize, for that we have the Advanced Roles of Grand Prix Head Judge, Regional Coordinator, and Program Coordinator. Level 3 is the last judge level where we’re looking for well-rounded judges, skilled in a variety of domains. This is the reason the Qualities cover almost everything a judge is ever expected to be involved in, and the reason the checklist covers so much.

Review check reduced to 6 from 10, but excludes certification reviews

With the transition from regional pre-releases and PTQs to a network of smaller events centered on stores, there’s a reduced amount of events using multiple judges. This makes it more difficult to gather enough observation data to write reviews. A typical way to compensate is to write a small number of substantial reviews and a greater number of shallow and short reviews, just to meet the number of reviews requested on the checklist. This isn’t something we want to encourage. We’d rather see more focus on the quality reviews; so the number is reduced but every one of those reviews should be of high quality.

Certification reviews are very variable. Depending on the relationship between the tester and the candidate, it can be just a stamp of approval (“I asked you if you wanted to test, you did and succeeded”) or the result of longer mentorship where a lot of substance can be put in the review. The latter can often result in a review separate from the certification itself. Because what we’re looking for in reviews is ability to mentor through written means, it’s cleaner to exclude certification reviews altogether.

Certification can be L1->L2

L2 Tester is coming and some L2s will be able to certify other judges for level 2. As we expect this to interest many L3 candidates it seems natural to allow certification of L2 to count towards the 2 in the checklist (they don’t count double though!).

Region requirements to recommendations removed

This requirement had two purposes:
– make sure a candidate has “seen the outside world” and doesn’t just ask their closest L3s for recommendations
– make sure the candidate isn’t trying to mask a deficiency by asking their L3 friends for recommendations instead of more objective judges

With the rationalisation of L3 testing, there is much less concern about deficiencies being unnoticed, so it makes much less sense for someone to recommend a candidate “just because it’s my friend.” If you’re an L3 judge, we trust you to write recommendations to candidates who deserve it, no matter where you live.

As an additional nice side-effect this removes worries about people moving and recommendations suddenly becoming invalid because of that.

Tweak to recommendation phrasing mandating examples and clarifying co-authoring

Analysis of many panels and recommendations led to an unsurprising conclusion: when a Quality is poorly covered in recommendations, the candidate is often weak in that area. That’s why we ask recommenders to put specific examples supporting their view on each Quality. This makes sure that the observation is detailed enough that deficiencies are more easily detected. And as usual it’s much better to learn about a deficiency way before testing so that you can fix it. Mandatory grading follows the same logic: the recommender must observe carefully in order to make a choice. Note that this has already been enforced for a while and the Verification Committee has been asking L3s to provide more information when recommendations weren’t up to standard; this is just an officialization.

Co-authoring recommendations is a solution when a candidate is especially isolated and can’t judge often enough with two different L3s to get two complete recommendations. In those cases it’s ok that one of the two is pieced together by two L3s. In order to prevent the other extreme, many L3s who each write about one Quality, we’re clarifying exactly what’s allowed. We also ask that each observation is clearly attributed to a recommending judge in order to encourage accountability.

Self-review must grade each Quality

We’ve been seeing an increasing number of non-committal self-reviews which either put each quality in both strengths and areas for improvement, or put everything in comments with no grading. This often has the consequence that the observations are very generic and similar to the very definitions of each Quality. This doesn’t help us get a clear, personalized picture of the candidate and also doesn’t help the candidate realize where they are and what are the next steps in their judge progression. Having to decide if a Quality is a strength (in this context, on par with the expected minimum for a Level 3) or an area for improvement really makes you think about your performance and leads to more precise self-evaluation.

When these self-reviews haven’t been informative enough we’ve been asking the candidates to put more detail and to grade the Qualities. This is more an officialization than a real change.

English requirement reduced to team leading and reading official documents

Magic is a global game and we’ve seen a lot of efforts to localize policy documents. This means you can judge effectively on local events without ever needing English. The cards are in your language, the players speak your language, you have studied translated versions of the CompRules and IPG, everything’s great.

As you step up to level 3, you’ll be expected to have regular contact with judges from other regions, in particular on GPs. You’re also expected to be able to lead teams on GPs, which means communicating effectively with your team members, other team leaders, and your head judge who probably has only English as a language in common with you.

You will also be answering complex questions about recently changed policy and can’t rely anymore only on translated versions. In case a difference accidentally appears between a localized document and the official one in English, other judges will rely on you to understand the differences and explain them in your own language.

So these two will require a certain minimum level in English. But something that’s not needed is ability to conduct complex policy discussions in English. That’s why we’re reducing the expected minimum.

Team Leader Certification can replace Team Leader Check

As Team Leader Certification becomes available it will replace the Team Leader Check which is discontinued. The exact details are to be published soon but won’t be terribly different. Either will be valid for your checklist, meaning that if you have a fresh Team Leader Check it’s valid for 3 years as previously advertised.

Community requirement stays

This comes back to my introductory paragraph about the New World Order context. As you move from level 2 to 3, you’re still expected to be skilled in several areas, namely events, policy, and community. So contrary to what was announced in May, we’re not changing the community participation requirement to allow only events, my apologies for the wrong expectations created. Discussions since that announcement made me realize that I underestimated how important it is that everyone contribute at least a bit to community development.

New checklist

Here’s the updated checklist you need to meet in order to apply for level 3, effective immediately:

To be considered as a candidate for advancement to Level 3, the following requirements must be met. References to “last x months” refer to the date of formal submission of the application.

• Must be a Level 2 judge, in good standing, for at least 12 months.
• Must have scored at least 80% on a Level 3 Preliminary Exam in the last 6 months.
• Must have acted as Head Judge or Team Leader for at least 5 Competitive (or higher) REL events, managing at least 2 other certified judges, including at least 2 such events in the last 12 months.
• Must have acted as Head Judge for at least 20 other events, including at least 5 such events from the last 12 months.
• Must have participated extensively in the pre-certification training and mentoring of at least 2 different judges who certified for Level 1 or Level 2 in the last 12 months.
• Must demonstrate communication skills sufficient to act as a Team Leader at the large, international scale, such as Grand Prix events. Must understand English well enough to be up-to-date on official documents.
• Must demonstrate participation in the judge community on a regional or global level beyond just being on staff at events (examples include mailing lists, seminars, articles, projects, etc.).
• Must have written a general (i.e. non-event-specific) self-review in the last 12 months, covering all of the Qualities of Level 3 Judges listed above, graded as Strengths or as Areas for Improvement. If that self-review is more than 6 months old, your application must include a brief update indicating progress on the Qualities of Level 3 Judges.
• Must have submitted at least 6 event-specific (non-certification) reviews on other judges to the Judge Center in the last 12 months, demonstrating the ability to provide accurate and constructive feedback, on both Strengths and Areas for Improvement.
  • Must include at least one review of a Level 1, of a Level 2, and of a Level 3 judge.
  • Must include reviews on judges of at least 2 Regions.
• Must have received two written recommendations from Level 3 judges.
  • Each recommendation must cover at least 7 of the 9 Qualities, each graded as Strength or Area for Improvement. Each quality must be supported by specific examples.
  • One of the two recommendations may be co-authored by two Level 3 judges; in that case, the primary recommender is the judge who enters the review and thus vouches for your Level 3 candidacy. Each observation must be credited to one of the recommending judges.
• Must hold a valid Team Lead Certification or have received confirmation from a GP Head Judge within the last 36 months indicating success in a Team Lead position at a Grand Prix where the checking judge was the Head Judge.

As always I’m open to feedback so don’t hesitate to ask questions, here or at l3.testing.manager@gmail.com.

Daniel