The following document contains information originally provided by Justin Turner.

Greetings, Judges! As I talk with you guys and gals at events, and especially post-event, there is a lot of talk about reviews. It seems that this fantastic tool for peer-based improvement goes woefully underused. Any high-level judge will tell you that one of the most important things to do at events is take notes and write reviews, yet it doesn’t seem to happen with nearly the frequency that it should. This article on good reviews aims to change the notion that reviews are “difficult to write” and, I hope, dispel the attitude of “I don’t even know what I would put in the review.”

1. Why Reviews Are Important
2. How to Approach Writing Great Reviews
3. Putting These Ideas into Practice
4. Example of an Effective Review Process

1. Why Reviews Are Important
Reviews are a very good tool for several reasons. Through reviews, we make the program better as a whole. At large events, working with people with whom you aren’t familiar, you have the opportunity to obtain critical advice from peers who are unbiased. You learn different and innovative ways to do the things that you do in an event. In a good review, you should be presented with some great, positive things you did that you can keep doing well and some things that you can improve. In a really good review, you are also presented with solutions to the things you can improve. Taking this information to heart, you can really begin to improve your judging and have even more fun (seems impossible, I know!) than you already are at events by minimizing the “speed bumps.”

2. How to Approach Writing Great Reviews
The easiest way to ensure you are on the path to writing a great review for a fellow judge at an event is to take notes! As you’ll see below, you don’t need to have pages of notes written down—just a few key points. We just take those key points and really flesh them out by talking about “why is this important to an event?” and “how can we make this better between events?” As you are interacting with judges and you see anything they are doing that stands out—whether it is a great thing or something they can work on—just jot down a little reminder. A few words, maybe two sentences at the most, should be all you need to remind yourself of the individual topics you would like to touch on. I want to take you through a typical review that I have both received and seen written about other judges, and suggest some changes that will really help make this tool what it was originally designed to be.

Sample Review Strengths:  Chris and I were on deck checks for this event. Our goal for the day was time extensions of 10 minutes or less. Chris was fast and efficient in sorting a deck and checking it against the decklist. Almost every one of our time extensions came to less than 10 minutes, which was awesome.  Since Chris was a candidate to test for L1, we went over a lot of different scenarios and rules questions. Chris showed a firm grasp of the rules and CR. He would come up with an answer quickly, was correct most of the time, and was able to explain his answers very well.
Areas for Improvement:  I would like to see Chris continue to study the IPG and MTR. I felt like he wasn’t as confident in his policy answers as he was with CR answers. With more work, I easily see Chris becoming a valued member of our judge community.  When giving rulings, I would also like to see Chris use a stronger voice. He is a somewhat soft-spoken person and needs to project more confidence into his voice when dealing with players.
Comments:  Congratulations on passing your Level 1 test, Chris! I look forward to working with you again at other events!

Now, looking at that review, there is some good information there, but we didn’t say why the goal was 10 minutes or less, and we have no specifics anywhere in there. “Study the IPG and MTR” seems like a blanket statement. It isn’t necessarily a bad statement, but some more specifics would really help this judge when reading this review. We especially need to know why each items is important and how it will impact future events. After a little work, here is an amended review, which I feel does a much better job of conveying what we want this judge to know about his or her performance at the event.

Amended Review Strengths:  Chris and I were on deck checks for this event. Our goal for the day was time extensions of 10 minutes or less. Chris was fast and efficient in sorting a deck and checking it against the decklist. Almost every one of our time extensions came to less than 10 minutes, which was awesome. Chris—keep practicing between events. Shuffle up your own decks and continue to improve your technique in sorting decks. Being efficient in this area helps the whole event run smoother and quicker, as we can avoid long time extensions. Since Chris was a candidate to test for L1, we went over a lot of different scenarios and rules questions. Chris showed a firm grasp of the rules and CR. He would come up with an answer quickly, was correct most of the time, and was able to explain his answers very well. Chris showed great initiative in picking up trash and pushing in chairs without having been told to do so. Chris—keep it up! See a problem, come up with a solution, fix it—that’s a great attitude to have.
Areas for Improvement:  I would like to see Chris continue to study the IPG and MTR. I felt that he wasn’t as confident in his policy answers as he was with CR answers. With more work, I easily see Chris becoming a valued member of our judge community. Chris—I hear you’ve been attending Joe’s judge class. I’d encourage you to continue to work with Joe on improving your knowledge and understanding of policy. He is a great resource for you over there. When giving rulings, I would also like to see Chris using a stronger voice. He is a soft-spoken person and needs to project more confidence into his voice when dealing with players. We talked about how a judge can be completely correct, but if he doesn’t sound confident, a lot of players will appeal, which in the end slows the event down. Showing confidence in rulings will help instill respect and trust in judges with the players we are here to serve. We want them to have faith in us to get the call right, and we won’t do that if they are questioning every ruling we make. Be confident, Chris. You know what you’re doing; show it to the player. Perhaps you could have Joe work with you on role-playing situations. As you gain more experience in giving these rulings, it becomes easier, and talking through a situation becomes second nature. It also makes it much easier to deal with difficult players.
Comments:  Congratulations on passing your Level 1 test, Chris! I look forward to working with you again at other events!

Look at what this review has going for it: Here, the candidate learns why the 10-minute limit for time extensions exists and gets some follow-up information on how to practice between events to shorten his deck-check time even more. We also learn that this candidate was able to come up with good, quick answers to rules questions and was really good about making sure the event was looking in tip-top shape! Also, we have some good praise at the end to build confidence in each other, which is very important in one of these reviews.
In the next section, we have the “study the IPG and MTR” blanket statement, which could still use a more specific example, but we also have some motivation and understanding about the judge’s situation. The reviewer learned about the support network behind the reviewee and urged the judge to continue with that support program. Good advice!
We were also able to really flesh out the “be more confident” feedback. Here, we see why it is important during events to be confident in your rulings, and there is even some advice on how to work on that issue between events. Feedback like this is important to judges so they can really grow between events and keep improving in the program. I feel that the review system isn’t used enough as it is, and when it is used, a majority of the reviews look more like my first example than my second example. Even the second example could be fleshed out with more specific examples of situations that happened at the event. I think by looking at the differences between these two sample reviews, we are able to see that the few notes we had scribbled down or stuffed into our memory banks for later writing are actually plenty to write a solid review on. Just make sure you flesh out your points with “why is this important?” and “how do we make this better?” I think if we all tried our best to make sure we were answering these questions with each paragraph we write in a review, we could help each other tremendously with our growth in the program.

3. Putting These Ideas into Practice
Now that we’ve delved into how we can take a few key points and turn them into a really solid review that gives a subject praise and room to grow, along with motivation and the tools to grow, it’s time to go into what exactly those key points should be.
At all events with a judge staff, whether they are PTQs, prereleases or GPs/PTs, you should have a small notebook or some method for quickly scribbling down notes. I generally will not make a page for each judge; rather, I just write down the judge’s name and the key point, then tabulate them later. This helps you save time on the floor and lets you concentrate more on the event than writing down notes. At small events, make sure you spend some face time with each of the judges on staff, including the head judge. Ask questions about how the program is in their area, what their goals are in the program, and what they are doing locally to meet those goals. Also, have some fun and ask some silly rules and policy questions. Most judges became judges because of an interest in the mechanics and rules, and to help players, so asking some scenario-based questions is a great way to bond with fellow judges with whom you haven’t spent a lot of time. About 95% of the time, these conversations will generate at least one thing for you to include in a review. For the other few points you want to write down: Shadow the subject while he makes calls, watch him work the floor, and continue to talk with him throughout the day. The mutual benefit of this communication is one of the best parts of working an event with more than one judge on staff. At large events, do basically the same thing, but try and concentrate on your team first. Those are going to be the judges with whom you’ll be spending the most time, so get to know them and how the program is doing in their area. The categories presented in the review center are basically the criteria with which you should work when thinking of points to write down for a review. The ones with which I predominantly concern myself are; Diplomacy with Players, Educates Fellow Judges, Explains Rulings Clearly, Professionalism, Shows Initiative, Game Rules Knowledge, and DCI Penalty Guideline. Those are the ones that I look out for, and I make notes based on those kinds of topics.

4. Example of an Effective Review Process
Let’s say I am observing judge Peter Venkman at a GP. I talk with Peter and find out he is a Level 1 judge from New York City, and he is aspiring to be a Level 2. I find out he is studying with another judge named Egon Spengler, who is a Level 2. Egon and Peter communicate regularly on Facebook, sometimes on IRC, and also at local PTQs. As we are walking the floor and talking, there is a judge call, and Peter goes to answer it. I quickly follow to observe. In the ruling, Peter is very professional, uses great body language to connect with the player, and makes lots of eye contact. His ruling seems a little rough; it seems that he isn’t very confident in his answer, and the player appeals the ruling. Right here, I already have the basics for a great review for this judge just from that little scenario. I’ll take out my notebook and write something like, “Venkman, professional, good body language, lacked confidence.” That’s all you need. When you read that later, you’ll remember exactly what the situation was, and you can expand on it in your review. However, we don’t have enough to write a really complete review. It looks like we have two positives and one area for improvement so far, so we come back a few rounds later and talk to Peter again. We ask some rules questions that are a little tricky and some policy questions that are “use your judgment” calls. These types of questions are great, especially when you get into the “whys,” because you get to see just how familiar and comfortable the subject is with his or her book knowledge. Let’s say that in this scenario, we determine that Peter is great on the basics, but could use some help on replacement effects. We also determine that Peter has great knowledge of the IPG and comes up with really insightful ways of looking at a scenario and applying the relevant policy. I would write down something like, “Venkman, needs help on replacement effects, great policy.” Now we have everything we need to write a great review, and it only took us maybe ten or fifteen minutes throughout the day to acquire this information! Based on my notes, and taking into account the techniques from the beginning of this article, my review would read something like this:

Improved Review Strengths:  Peter was on paper team with me at GP Carpathia, and he was a great asset on the floor. I had the pleasure of observing him make some rulings throughout the day. He was very professional and used great body language to connect with the players and get on their level. Coupling this with lots of eye contact, I feel the player was able to be himself and was shown that Peter was really there to help him. These small details about rulings are so important to provide the players with a great play experience and to really develop a sense of trust that the judge staff has their best interests at heart and really wants to uphold the integrity of the tournament—as opposed to just being referees handing out penalties. Peter—keep up the good work! I also had the opportunity to speak with Peter about some scenarios that have come up for me in the past that have been tricky to work out, and we had some great discussions. Peter showed a fantastic grasp of the IPG, and was really able to break down for me what his thought processes are when he is deciding what section of the policy is appropriate for a given situation. I learned some great things from Peter that I can apply myself in the future, and I want to thank him for letting me “peer behind the curtains” a bit and see what goes on in his head when awarding a penalty. I also learned that Peter is aspiring to be a Level 2, and I can say his policy knowledge shows that he is putting the effort forth into attaining the knowledge needed to hit his goal.
Areas for Improvement:  On the rulings that I referenced above, one thing I would like to point out is that it seemed like Peter was having trouble delivering rulings with confidence. One time in particular, the player appealed, and it seemed like the only reason was because Peter seemed unsure of himself when speaking. Peter—you already have all the body language down—just build some confidence into your voice! From talking to you throughout the day, it’s obvious to me that you know what you are talking about; show the players that you do as well! This will not only help provide them with great customer service, but will develop the community, because they will start coming to you with other questions and comments when they think you know what you are doing. These are some of the building blocks to a strong Magic community, both locally and globally. We were going through some rules questions and it seemed like the replacement effects section was a little rough. I learned that Peter was working with Egon Spengler, a local L2, and I urge Peter to talk with Egon specifically about that section. Peter—replacement effects are pretty common, and it’s not too uncommon to have multiple replacement effects going on. Once you get the hang of them, though, even very complex game states will become very easy to get to the bottom of. I’m sure that with a little reading and working with your mentor locally, you will easily be able to master this section of the CR!
Comments:  Peter was a great asset all around. I had a great time working with him at this event. I think we were both able to learn a lot from each other, and that is half the reason I make the trips to Grand Prix events. I look forward to working with Peter in the future. Peter—tell me when you test for Level 2, man. Good luck!

As you can see, we were able to build a very constructive review based on two small notes in a notebook in my pocket at this event. Total time invested was roughly fifteen minutes. At a large event, you should easily be able to do this for your teammates and your team lead. In fact, I would try to bring up the fact that you are doing this during your team or staff meeting, and see how many others you can get to do it with you! The more we review each other, the better we will all be, and that can do nothing but help what is already a fantastic community! Keep up the good work, everyone!