Since March 2017 Reviews are hosted on JudgeApps.

The JudgeApps interface features several options within the Reviews dropdown menu, available to all registered users.

“My Reviews” gives you a screen showing the most recent reviews that you wrote, the most recent reviews that you received, and any drafts that you haven’t submitted yet. “Search Reviews” allows you to filter all the reviews you have access to, for example to find reviews of a certain judge, or reviews of judges of a certain level, or even to search exclusively for a certain type of review.

But the biggest changes to our system are in how you write reviews, so let’s go to that screen now.

There a few different types of reviews. Some are purely for feedback, and others are used to update judges’ levels and roles. Here are some review types that might be available to you:

You may not have access to all of those choices – some of them only make sense for judges of certain levels.

• An Evaluation review is the default – it is an evaluation of another judge with no strings attached. It may or may not be associated with an event, and it is not an advancement review or a formal recommendation. All JudgeApps users can write Evaluation reviews.
• A Self-Review is a special type of Evaluation review. It is a review written about yourself. When you choose Self-Review, the “review subject” fields will disappear, because every Self-Review is about yourself, and any review about yourself is a Self-Review. There’s no special baggage here – this can be the Self-Review you submit to the Verification Committee as part of the L3 process, or it can just be a Self-Review that you’re writing for yourself. All JudgeApps users can write Self-Reviews.
• The Advancement Interview review type is used when someone is trying to be promoted. This is used to enter successful advancement attempts, or unsuccessful ones. This is a special type pf review that adds a few new fields. You’ll be asked to enter the candidate’s exam ID, exam score, and the new level you intend to set them to. You’ll also be asked whether the interview was successful. Once you submit this review, if the interview was successful and the exam score is passing, the subject’s level will change, so don’t use this feature lightly – only submit an Advancement review if you intend to change someone’s level. (Note: It will look like you can demote people using this feature. Rest assured that you cannot. For technical reasons, we have to build the list of valid levels before you even choose who you are reviewing – it will make sure that you’ve chosen a valid level before letting you submit the review.)
• An L2 or L3 Recommendation is another special type of review, where Level 2 or 3 judges can choose whether to recommend someone for advancement as part of the advancement process. These special types of reviews ask you whether you are recommending the subject for advancement – you choose Yes or No. This is only a recommendation, it does not actually change anyone’s level. We encourage you to enter both successful and unsuccessful recommendations using this type, to help candidates track their progress.

Here is an example of how you might enter an unsuccessful advancement review. In this situation, the candidate passed the exam but failed the interview.

The first part of the form also asks for the language the review is written in, and the roles that each party was filling at the event in question. If the review isn’t tied to an event, or you can’t find a role that fits, you can leave the roles blank.

The second part of the form asks for the event that the review is from. There are three possible use cases here:

• First, check the dropdown (green) to see if the event is there. This will list all your JudgeApps events, starting from the present and working backwards. If you find the right event and select it, the rest of the boxes will disappear.
• If you can’t find the right event, then you can enter the event details yourself (blue).
• Finally, if the review isn’t connected to a specific event, you can leave this entire section blank.

Which brings us to the review itself. Normal review form, right? Not so much, we’ve highlighted some of the changes.

• In blue: you can add new sections to the review! The buttons at the top will add a set of default sections, either the familiar “Strengths and Weaknesses”, or the Qualities of Level 3 judges. Either of those buttons will also remove any unused sections, to keep the review from getting cluttered up, but don’t worry – the form will not remove any section that you’ve already entered text into. It will only remove empty sections.
• In red: You can also remove unneeded sections one by one.
• In yellow: You can click and drag the sections to reorder them.
• In green: The rating is now per-section, so you can rate someone “Above Average” in Teamwork, but “Below Average” in Policy Knowledge, for example. The ratings are also completely optional – the default is “Not Rating”, which just means you’re choosing not to use this feature.

“Save and Continue Editing” will bring you back to this page. As long as there aren’t any errors displayed, your review has been saved. This system does not save automatically, and if you’re using a computer that is prone to crashing, you might want to use Google Docs or MS Word to prepare your review.

“Save Draft” will bring you to a preview of your review. This shows you all the important details of the review, as well as the contents of the review. As you can see, this is a self-review from an event.

This draft review is your completely private workspace – no one else can see your drafts. At the top, you’ll either see a “Submit” button, or any error messages that are preventing you from submitting the review. Click “Edit Draft” to fix those errors.

Once you click Submit, the system will share the review with the subject. They’ll get an email notifying them that you submitted a review, and you’ll both be able to discuss the review in our new comments section. This is a great way to chat back and forth on the contents of the review – to request clarification, or suggest changes. The other party will also receive a notification when you post a comment.

Once a review has been submitted, it can be edited for 7 days. After that, you will be unable to make further changes.

As with Judge Center, only a limited number of people have access to all submitted reviews. Right now, no one has this level of access globally. Certain JudgeApps administrators will have access to your data; however, they will only use that access to diagnose problems with the site or address technical problems. Regional Coordinators have access to all reviews written by or about judges in their region. Some types of review have special permissions: for example, the judges who are responsible for the GP Team Lead certifications have access to all of the evaluations of GP Team Lead candidates.

This page written by Dan Collins.