Greetings all,

If you read this on Facebook, you can skip it, it contains no new content.

If not, feel free to dive in.

For context, I wrote this on Facebook after making Level 3 Judge, and getting a number of “congrats, this seems important, but I don’t know why” posts from family and non-Magic friends.

The quick hits version of this article:

• I’ve been in the “closing in” state for moving to L3 for a bit over a year. The final pieces happened in the last three months, and involved 22 pages of essays, taking a fairly difficult 50 question test, and a 3 hour panel with three high-level judges, which was more intense than any job interview I’ve ever had.
• I am now trusted to be awesome at events, but have correspondingly higher expectations. 95% of both were in place before the testing. The number is mostly recognition, and it felt amazing to get it.
• The process was the goal, and the process was awesome, and continues to be awesome, and continues to continue.

I’ve heard the judge program described as a cult of self-improvement that runs tournaments on the side, and nowhere is this more visible than the L3 process right now.

Some Stats on L3+ Judges

• There are just over 4000 judges in the world.
• 116 are L3+
• I am the first L3+ in Minnesota since 1998, ending a 15 year gap. (There is another L3 FROM Minnesota, Tasha Jamison.  She moved to Virginia, made L3, and now lives in Portland.)
• The United States (well, the world) has been divided up into regions by the Judge program, for the purposes of administration and program development.  Minnesota is in the Midwest region, along with the Dakotas, Nebraska, Iowa, Wisconsin, and Michigan.
• There are no L3+s in Iowa, the Dakotas, or Nebraska.
• There is one in Michigan, our regional coordinator Steven Briggs. (And a Judge Emeritus, see here for more details on Judge Emeritus.)
• There is one each of L3,  L4, and L5 in Wisconsin.
• This means I am one of only five L3+ judges in the Midwest region.

What do L3s do in the judge program?

• L3s test L1 judges for L2.  This involves evaluating their knowledge of the game, their skills at running events, and their customer service skills.  Some of this is from a written test.  Some is from an interview.  Some is from working with the L2 candidate in question.
• L3s act as team leaders at Grand Prix events, usually with a team of 5-6 people (Grand Prix are the largest tournaments, with 900+ players and 35+ staff, who are usually L2 or higher)
• L3s act as head judges for larger competitive events regularly, ones that require a staff of 4+ other judges
• L3s review and provide feedback to other judges regularly, and there is an expectation they will least one review per event they work.  Reviews take the form of both face-to-face and written feedback, and are expected to provide concrete examples of things that judge has done well and poorly.
• L3s provide outside of event leadership in some fashion. There are a wide variety of program development things that L3s do.  I mentioned in a prior post James Bennett, who works on JudgeApps.  There are judges that write articles, record videos, put together conferences, handle coordination for the judges in their region, or manage testing of other judges.  This is not an inclusive list, just some examples.
• L3s are expected to work all the Grand Prix in their region.  This year the only Grand Prix in my region is in Detroit, but they move around.  2012 had Lincoln and Minneapolis.
• L3s are expected to go to other Grand Prix events outside their region, as time permits.  This year I attended Denver, Portland, Las Vegas, and Kansas City, and plan to attend Oklahoma City, Washington DC, and if I can make my schedule work, Albuquerque.
• L3s get access to a handful of L3+ specific mailing lists, forums, and other tools and contact methods.  It is kind of a secret club, but the club does things like talk about how to make sure other people know the rules for joining and can be awesome enough to get in.  The opposite of an exclusionary club, though there are entrance requirements.
• L3s are the core of larger event staffs. I’ll talk about those requirements in a sec, but because they are acknowledged leaders with a strong and maintained skillset, they are almost never turned down for events. Talking with a couple of very good tournament organizers, they could not think of a reason for turning down an L3 that applied to one of their events, and if you ask them in advance they will accept you before they do the normal announcement. Most of them will be happy to do so, as this means they can count on you being there for the event.  This means L3s can plan better for events – previously there was uncertainty for me if I would be accepted for an event I applied to work. Now I can contact the TO in advance and they will generally confirm they will put me on for an event, and I can plan around that. I have used this once so far, and plan to do so occasionally in the future if I need to plan things like time off in advance.

What do you need to do to become L3?

• You must judge (and head judge) a minimum number of events.
  • The requirements are not onerous if you are working events regularly, but do require you to be reasonably involved.
• You must certify at least a minimum number of judges.
  • L2s can make L1s.  You can also help certify someone for L2.  I have done both.
• You have to already be doing regular written reviews, with some specific metrics for them.
  • This is simultaneously easy and hard.  I am pushing myself to do more reviews, but am still lacking in them.  Some people basically never review.  Like any skill, it takes practice to get there.
• You need to write an extensive self-review (and once you make L3 refresh it at least yearly), covering the 12 qualities the program wants to see in L3s. These are things like leadership, ability to investigate players causing problems in events, ability to assess other judges, diplomacy, rules and policy knowledge, etc
  • Honesty is easy in this.  The hard part is something I ran into – I did two self-reviews six months apart, and had the same weaknesses in both.  I had not addressed my issues adequately in that time.  This was something I discussed with my panel (more on that later).  They want me to push for improvement and have a plan for fixing those repeated issues.
• You need two recommendations from L3+ judges.
  • These have to refer to your self-review, and cover at least 8 of those 12 qualities above.  I had some awesome reviewers, who gave me tons of good feedback while writing the reviews, and going into the final stages of the L3 process.  This is normal for L3 candidates, but strange in other spheres of life.
• You need to demonstrate skill in team leading, lead a team at a Grand Prix and get a recommendation based on that from the L4+ head judge, and preferably some positive notes from the L3 assigned to shadow you at the event.
  • I’ve seen people have to re-do this.  Team leading is HARD.  Especially at Grand Prix events, all the basic goals of the team will be hit.  You have people that have done the job before, and will do the core things well.  What the recommendation here points to is that you know the “other stuff” – how to provide leadership, how to plan for issues, how to handle the people side, and how to make a team feel like a team and not just some guys doing some stuff.  I’ve worked with fantastic team leaders (Thales Bittencourt, an L3 from Brazil is my idol), and some that are not so good.  (Not calling anyone out, they have all gotten better and I hate pointing fingers about stuff like this.)

That sounds like a lot.  Then you are just made L3 for doing all that?

Nope, there is more.  Those are the pre-test requirements.  They show you are ready to be tested.

• Once you have all the pre-test ducks in a row, you do a written essay-style pre-panel interview with a L3+ you don’t know.
  • Mine had 17 starting questions, and we did another five back and forth. Taking a look at it, it ended up as a 22 page document of solid text.  Talking about specific things are discouraged, but I had to cover every aspect of those 12 qualities above, and there were probing open-ended questions about things like mistakes I have made and my motivations for what I do.
• After that, you take a multiple-choice test with 50 rules and tournament policy questions. 80% is a passing score.
  • This test is hard.  I scored a 92%, which is pretty good.  The rules section covers the pretty complicated Magic rules thoroughly, and the policy section tests your knowledge of how Judges are expected to run events just as deeply.  You have to be a knowledge expert to pass.
• Then comes the fun part. A panel of 2-3 high level judges go over those essay responses, the reviews you have written, your recommendations, and anything else they have gathered on you, talking with you on any of those aforementioned 12 qualities of regional judges they want to see if you are strong or weak on.
  • My panel took three hours.  It felt like a grilling, because they skipped everything I was good at, and concentrated only on my weak points, trying to find out HOW weak.  At the end the “you are easily great in these categories blah blah blah” blew right past, and then there was a rundown of things I should look at improving.  Looking back, that is super affirming – I am good enough not to have to spend a ton of time puffing my ego, and they care enough to let me know they want more from me than just being “good enough” in all 12 categories.
• When the panel evaluates you, you can be weak on no more than two categories, and if you are, or are totally deficient in just one you will need to go through a fixing process and come back with the problems addressed.
  • If you get to the panel, you are 100% assured to make L3 if you want to work for it, as you are given a L3 buddy that works with you on fixing your issues, and you get really incredible feedback.

Whew.  Tons of work.  In the Facebook thread, this was described as “equivalent to a college degree” and Stephen Hagan (L2 from Kansas, PhD in Sociology) told me it is pretty similar to a thesis defense.  Both are apt – I had to defend my position that I deserve L3, and there were a lot of moments in the panel where I thought I had flubbed some answer terribly and it would be a big negative.

As a note, you get really incredible feedback regardless of passing or failing at every stage of the process. This might be the best part of pushing for L3. You get a roadmap to self-improvement, and a ton of dedicated people that want to help you get there.

I personally never had to re-do anything that was not related to time. I let some of my time-based things over-age, notably having to re-take a L3 assessment test which is basically an evaluation rules and policy test. This suggests I was very ready at every stage of the process, just procrastinating on doing things, which should not surprise anyone that knows me.

I’ve already seen some changes from making L3.
I am a lock for two different large events scheduled in Minnesota and Wisconsin coming up in the next couple of months as a result of being L3.
I got some more product from working Grand Prix Kansas City. (Not the goal, but it makes the trip better.)
I have been invited to work on one of the core Judge websites, JudgeApps, hopefully implementing something I suggested in my pre-panel essays. (Look for Judge Achievements coming at some point, fellow judges. They will be super cool, though there are some logistical things related to them and I do need to actually learn the JudgeApps codebase first…)
From the feedback I have gotten I have plans to get better. Things like better review planning, better event management, and an idea for future changes to my region and local area, I have plans, thoughts and ideas I did not have before.

Basically, it is a process that is a lot of work, but not frustrating work. You know the goal is achievable, if you put in the effort and make yourself better. And the making you better is the point. The number is just a number. It shows you got to the one stair, though there are a lot climbing up above, and the climb is the point, not the stair.

But man, it feels good to have the acknowledgement of your peers. And holy cow, this was a great process, with more to come.

Oh, and a disclaimer.  Any errors are mine above.  I had a bunch of feedback on things, and some wrong data on Facebook, and I am sure I have made some other mistakes.  Please don’t hold it against me, but do comment and I’ll fix things.

Thanks tons, everyone, for all the well wishes and congrats.