A Guide to L1 Post-test Interviews

| Evan Cherry
Written by Josh Feingold

Written by Josh Feingold

 

Taking an L1 exam is a formative moment in a judge’s career. The lessons the candidate takes away from the experience will do a lot to shape their progress in the coming months. Whether the candidate passes or fails, the post-test interview is a prime opportunity to guide this individual on to greater success and gather the information for a meaningful review.

 

Step 1: Pass/Fail

Before anything else, tell the candidate whether they passed or failed the exam. Don’t tell them the total score. We’ll get to that later. This may seem obvious, or it may seem like a missed opportunity to hold your candidate in suspense. However, you are going to want the candidate to be able focus their attention on the tasks you are going to present. Having them worry about passing or failing will only detract from the quality of information you would otherwise be able to obtain from the coming steps of the interview process.

If the candidate passed, explain that passing the test is only the first step and now you need to talk about some specifics to help fill out your advancement review. If the candidate failed, let them know that failing an L1 exam is not the end of the road. What you need then is to determine how you can give them good guidance on how to prepare for the next time they test.

 

Step 2: Uncertain Answers

Ask the candidate to go through the exam and point out any questions where they were uncertain about the answer. Note these questions even if the candidate chose the correct answer, as it may be useful to remind the candidate about these rules or policy issues when writing the review. You are looking for two major things during this step:

1. How many of the “uncertain” answers were actually missed?

A substantial mismatch may indicate that the candidate is unable to identify areas in which their competence is limited. If a candidate misses questions that they were certain about, the judge may confidently misunderstand a rule or policy.

2. What reasoning is the candidate applying to pick an answer in uncertain situations?

Most L1 exam content is straightforward logic that governs the way rules and policy work. For example, at Regular REL just fix it unless you really need to DQ. The candidate should understand the basics of casting spells, combat, and steps of the turn. They should apply good fundamental knowledge in approaching unknown questions rather than making stuff up or guessing wildly.

 

Step 3: Incorrect Answers

After discussing “uncertain” questions, you can reveal the total score on the exam. Not providing that information prior to this point preserves the information you would receive in the previous step. If the candidate knows the specific number of missed questions, it may change the way they assesses confidence in the given answers.

Your goal in reviewing incorrect answers is to uncover where they went wrong. These issues should certainly be discussed in your review of the candidate to reinforce their understanding.

 

Step 4: Goals

At this point, the candidate knows the outcome of the exam and has reviewed items about which they were uncertain or incorrect. Where does the candidate go from here?

If the candidate passed, what types of events are they planning to judge? Mostly store-level Regular REL events like FNMs and pre-releases? Other store-level Competitive REL events like GPTs and Invitational Qualifiers? Do they want to start moving toward judging larger Competitive REL events like 10Ks? Are they making plans for L2 at some point, or happy to be an L1 for the foreseeable future?

If the candidate failed, what rules and policy problems did you identify together? When do they want to test again? What is going to be different next time? How will they study and prepare?

Regardless of the outcome, you will want to direct the candidate to some resources beyond the Judge Center practice exams. Fortunately, the judge community is filled with motivated people who are willing to share their knowledge, so many such resources are out there. Here is a short (and not at all comprehensive) list by category of interest:

 

Step 5: The Review

You didn’t think you were done when the interview was over, did you? It’s review time! Let’s start with the basics: enter the review by DCI number, NOT THE CANDIDATE’S NAME. DCI numbers are unique. Make sure you aren’t promoting the wrong “Jim Smith” to L1! Do this also for the candidate’s exam so you can reference the candidate’s answer sheet for their DCI#.

Just because a candidate got a passing score on the exam, you are not obligated to recommend them for L1. For example, if the candidate has excellent rules knowledge but missed every question on Regular REL policy and your interview indicates that this person would make FNM miserable for the players, you probably don’t want this person as a judge. Passing the test is necessary, but not sufficient, to reach L1.

The review is a permanent note of the things you discussed during the interview. You should repeat what was discussed in the interview. You want the candidate to be able to reference all the things you talked about. Some of those items should be:

  • What did the candidate get right on the exam? Are there areas or rules that the candidate knows very well? (Strength) What did they get wrong? Are there obvious problem areas? (Area for Improvement)
  • Does the candidate really understand Regular REL policy and philosophy? (Strength) Did they want to impose punitive fixes? (Area for Improvement)
  • Do they have a good demeanor and rapport with players? (Strength) Is the ruling given in a nervous or condescending tone? (Area for Improvement)
  • Is the candidate a good listener and receptive to feedback? (Strength) Do they get defensive? (Area for Improvement)
  • Did the candidate’s list of uncertain questions match relatively well to the list of incorrect answers? (Strength) Were the incorrect answers a complete surprise? (Area for Improvement)
  • Did the candidate show good reasoning on uncertain answers? (Strength) Did the candidate seem confused about the way things generally work? (Area for Improvement)
  • Did the candidate have clear goals following the test? (This is a Strength, but lack of goals isn’t really an Area for Improvement.)
  • What are the candidates goal’s, and what resources are available to help the candidate reach that goal? If the candidate is very strong, you may want to put these items in the Areas for Improvement section, since these are actually the items they will use for immediate improvement. If the AFI section is already looking a bit hefty, you may wish to put the list of resources in the comments section, following some words of encouragement.

Once the review is submitted, you’ve actually finished the interview process. Now you can sit back, put your feet up, and bask in the warm feeling of knowing that you’ve given this judge candidate (and, hopefully, brand new judge) the most useful testing experience you could provide. As mentor and candidate, you have shared in an intimate process that will enrich both of your experiences in the Judge Program.