{"id":547,"date":"2015-08-19T22:53:48","date_gmt":"2015-08-20T03:53:48","guid":{"rendered":"http:\/\/blogs.magicjudges.org\/bearz\/?p=547"},"modified":"2016-01-01T16:28:19","modified_gmt":"2016-01-01T21:28:19","slug":"guest-post-planning-for-failure","status":"publish","type":"post","link":"https:\/\/blogs.magicjudges.org\/bearz\/guest-post-planning-for-failure\/","title":{"rendered":"Guest Post: Planning for Failure"},"content":{"rendered":"<p>[<em>When I moved to New York City a year ago, Chase Culpon was one of the first judges I met &#8212; and I&#8217;m very glad I did.  In addition to his expertise at events, Chase has taken a huge role in strengthening the judge community in the city, by coordinating social gatherings, organizing carpools to tournaments and conferences, and even starting a support group for judges interested in becoming more fit.<\/p>\n<p>I&#8217;ve always valued Chase&#8217;s insight into tricky topics, so when he approached me a few months ago with an idea for a guest post, I couldn&#8217;t have said yes faster.  I&#8217;m very excited to share his expert perspective about ways we can better prepare ourselves for the challenges of running great tournaments. \u2014Bearz<\/em>]<\/p>\n<p><div class=\"wp-caption alignright judgeimg\"><a href=\"https:\/\/apps.magicjudges.org\/judges\/dci\/3103625099\"><img src=https:\/\/apps.magicjudges.org\/dci\/avatar?dci=3103625099&size=200 alt=\"Chase CulponNew York, New York\"><\/a><p class=\"wp-caption-text\"><em>Chase Culpon<br \/>New York, New York<\/em><\/p><\/div>Hi, I\u2019m Chase&#8211;a judge and <a href=\"http:\/\/www.wnyc.org\/people\/chase-culpon\/\">audio engineer<\/a> in New York City. On the surface, my job is plugging in some good mics and pushing around some faders to make sure events sound great on a live broadcast. What that really means is that, well before the event begins, I design and implement good systems. Deciding what tools are right for each job, establishing best practices, and working with a team are all essential for making live events run smoothly. Great shows and concerts have great systems that keep rolling despite failures, and the same is true for Magic events. In the spirit of <a href=\"http:\/\/blogs.magicjudges.org\/bearz\/entwining-experiences\/\">entwining experiences<\/a>, I\u2019d like to share how we might use some of the tools of my trade to make the best of every Magic event.<\/p>\n<p>One of my favorite frameworks for refining the systems I use professionally is the Failure Mode and Effect Analysis, or FMEA for short. It\u2019s an approach designed to pick apart systems where failures mean life and death. Originally developed by the military in World War II, it was also used extensively by NASA in the USA\u2019s race to the moon. Today, FMEAs are most often seen when designing systems for construction and manufacturing, but the methodology has been incredibly useful for me in both my professional and judging careers. <\/p>\n<h2>What\u2019s a FMEA?<\/h2>\n<p><p><img  style='float:right'  class='lems-mtg-cardimg' src='http:\/\/gatherer.wizards.com\/Handlers\/Image.ashx?size=small&type=card&name=Lens of Clarity'><\/p>FMEA might sound scary and complex, but the basics are very simple.  The best way for us to understand it is to look at all the components.<\/p>\n<p>The first step of FMEA is analyzing each potential problem though the following lenses:<\/p>\n<p><strong>System:<\/strong> What thing are we working with? Registration, scorekeeping, deck checks, etc. This can be listed by task, or by responsibility.<\/p>\n<p><strong>Potential Failure Mode:<\/strong> A specific thing that can go wrong.<\/p>\n<p><strong>Severity:<\/strong> A quantitative value of how much damage this failure will typically do. For magic I\u2019d suggest a scale of 0-5 based on the impact it has on the event and the player experience.<\/p>\n<ul>\n<li><strong>5 &#8211; Critical Failure:<\/strong> This ends the tournament for multiple participants, or does irreparable damage to the event or other lasting damages to an individual. Anything requiring professional medical attention or results in significant destruction\/loss of property automatically falls in this category. Failures of diplomacy that cause players or judges to stop participating in competitive Magic also apply. <\/li>\n<li><strong>4 &#8211; Substantial Impact:<\/strong> A problem that results in significant delays to the event, harms the integrity of the tournament as whole, creates a large impact to tournament integrity for a match, or causes an extremely negative player experience. Having to re-pair a round after seating players, experiences that cause players to drop from the event, and some DQs would fall here.<\/li>\n<li><strong>3 &#8211; Significant Issue:<\/strong> A disruption or delay to an event, or an issue that could impact the integrity of a specific game or match. A bad ruling that substantially impacts a game, a short delay in posting pairings, and any errors in reporting a match fall in this category.<\/li>\n<li><strong>2 &#8211; Minor Issue:<\/strong> A notable problem, but is correctable without substantial impact on the tournament or player experience. These things cause delays of seconds, not minutes, and cause general inefficiencies.\n<\/li>\n<li><strong>1 &#8211; Annoyance:<\/strong> A minor inconvenience that has little or no impact on the event.<\/li>\n<li><strong>0 &#8211; No Impact:<\/strong> Does not impact the event in any measurable way.<\/li>\n<\/ul>\n<p><strong>Current Controls:<\/strong> How do we detect the problem, and do our current systems catch this issue before the potential effects happen?<\/p>\n<p><strong>Occurrence %:<\/strong> Your best guess at how likely an average event will suffer this failure. I prefer to define this as the likelihood of a failure per event, but it could also be calculated by round, so long as your metric is consistent. In larger industries, this would be strictly defined, studied, and measured.<\/p>\n<p><strong>Risk Priority Number (RPN):<\/strong> A simple formula: Severity x Occurrence = RPN. This is a quantitative value that lets us prioritize what problems need to be addressed first. A common problem with a minor impact is often more important to address than an extreme situation that\u2019s incredibly unlikely.<\/p>\n<p><strong>Suggested Actions:<\/strong> What would you like to do about this problem! This section should also include people to follow-up with and questions that need to be answered.<\/p>\n<p>FMEAs work best when tailored for specific situations&#8211;the potential problems at a GP, as well as their impacts on hundreds to thousands of players, will be very different than those at your local PPTQ. Be aware that tweaking your system may lead to a new set of problems, so run your great ideas through the process again! For small systems, it\u2019s possible to go through this process quite quickly. This method is great for polishing up all the little things in your events.<\/p>\n<h2>Now what?!?<\/h2>\n<p>The purpose of a FMEA for judging is to guide a discussion and lead us to good decisions for how we run events. The core metric of FMEA is the RPN, the Risk Priority Number. The higher the number, the more significant an impact a problem is likely to have. These critical areas demand attention and resources for an event to run successfully. Pay attention, this stuff will ruin your day!<\/p>\n<p><p><img  style='float:right'  class='lems-mtg-cardimg' src='http:\/\/gatherer.wizards.com\/Handlers\/Image.ashx?size=small&type=card&name=Monsoon'><\/p>A low RPN is a good indicator that your current systems are taking care of the potential problem adequately. These problems can be caught easily, and don\u2019t cause major issues. It can be correct to let something with a minor impact fail in order to solve a bigger issue. Skipping that extra deck-check when judges are going on break, or delaying getting slips out to cover a flurry of judge calls at the start of the round are both examples of how we can optimize how we fail when resources are limited.<\/p>\n<p>Often a potential problem can be solved with very little or no resources. Some of the best habits of judges like picking up trash and pushing in chairs are fixing problems that won\u2019t derail an event, but take next to nothing to solve. Just be reasonable and skimp whenever needed. These are awesome things to do, but a bad idea if a ruling is waiting on you! <\/p>\n<h2>In the Real World<\/h2>\n<p>Just like the exercise of going through a FMEA is a great way of thinking critically and in detail about potential problems, but it\u2019s a much more powerful tool than that. Let\u2019s take a look at one issue been a big challenge for the program recently: product and promo distribution at large events, and how a FMEA might help our planning. <\/p>\n<p>As events get bigger, and TOs offer more and more custom VIP packages, how we handle all the logistics of the player meeting get increasingly challenging. There are literally thousands of packs, promos, and deck boxes that all need to be distributed as quickly as possible, and many different tiers of product that have to be given to exactly the right players. If we screw this up, we\u2019re costing the TO money, and denying players what they paid for. It\u2019s a particularly sensitive area for players, and each time something fails, it can do substantial damage to the reputation of the Judge Program, the TO, and Organized Play overall.<\/p>\n<p>So what\u2019s our FMEA look like for the player meeting?  Let\u2019s say we\u2019re using the \u2018standard\u2019 approach of judges directly handing out packages to the appropriate players.<\/p>\n<p><a href=\"http:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/1-Distribution-v2-Cut-JPEG.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/1-Distribution-v2-Cut-JPEG.jpg\" alt=\"1-Distribution v2 Cut\" width=\"1469\" height=\"734\" class=\"aligncenter size-full wp-image-552\" srcset=\"https:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/1-Distribution-v2-Cut-JPEG.jpg 1469w, https:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/1-Distribution-v2-Cut-JPEG-300x150.jpg 300w, https:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/1-Distribution-v2-Cut-JPEG-1024x512.jpg 1024w\" sizes=\"auto, (max-width: 1469px) 100vw, 1469px\" \/><\/a><\/p>\n<p>Some quick conclusions off the bat: things go most obviously wrong when product runs out on the floor. Even when product is counted and allocated for specific areas, it\u2019s very common for piles to get mixed up, judges to hand out in the wrong area, or just simple miscommunications to run rampant. While this type of issue normally only leads to a slight delay, how easy it is for this to go wrong in any number of ways cranks its RPN up to the max. This means if you\u2019re planning logistics for an event, it\u2019s worth it to slow down the distribution by holding a few judges in reserve to bring extra product to where it\u2019s needed; you\u2019ll save time in the long run. <\/p>\n<p>Poor communication of the plan is the most common cause of failures in this list: it contributes to almost every failure point. This means it\u2019s well worth it for the logistics team\/Head Judge to communicate as clearly as possible before the event. Judges need to know where they will get the right stuff and where they should hand out the stuff, and they should also know where all of the other stuff is going so they can pivot and help out elsewhere when things go wrong.<\/p>\n<p>Also, the most common \u201cnightmare scenario\u201d of product running out isn\u2019t one we need to worry about&#8211;especially in big events, our TOs are pros. I\u2019ve yet to see a large event run out of packs. Yes, it\u2019s potentially catastrophic, but we can breathe easy knowing our current systems take care of it. (Keep in mind, though, this FMEA is all about the initial player meeting&#8211;having enough preregistered pools for sleep-in specials may be a completely different story!)<\/p>\n<h2>What to do?<\/h2>\n<p>Another great way to apply FMEAs is as a comparative tool for picking the best system to use for an event. To illustrate this, let\u2019s look at two methods of handling EOR for large events, which I\u2019ll define as having flights of 700-1000 players.<\/p>\n<p>The traditional approach is for one judge to have a clipboard of all of the outstanding matches that the scorekeeper provides. I\u2019ve worked under this system during GP Charlotte and multiple SCG Opens.<\/p>\n<p>Another approach is to have multiple judges each coordinating EOR coverage in smaller zones, rather than having everyone go to a single judge. This system often involves these judges creating lists of outstanding tables by hand, rather than getting it from the scorekeeper, and may feature the EOR team lead coordinating coverage between zones. This is the method my flight used on day 1 of GP Vegas 2015.<\/p>\n<p>So, what does the FMEA look like for each?<\/p>\n<p><a href=\"http:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/2-EOR-Clipboard-Cut-v3-JPEG.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/2-EOR-Clipboard-Cut-v3-JPEG.jpg\" alt=\"2-EOR-Clipboard Cut v3\" width=\"2266\" height=\"1418\" class=\"aligncenter size-full wp-image-551\" srcset=\"https:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/2-EOR-Clipboard-Cut-v3-JPEG.jpg 2266w, https:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/2-EOR-Clipboard-Cut-v3-JPEG-300x188.jpg 300w, https:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/2-EOR-Clipboard-Cut-v3-JPEG-1024x641.jpg 1024w\" sizes=\"auto, (max-width: 2266px) 100vw, 2266px\" \/><\/a><\/p>\n<p>Right away, we see that the clipboard method fails quickly whenever the judge with the clipboard is overwhelmed. It\u2019s difficult to track a stream of incoming slips while simultaneously assigning judges to cover specific matches and bring back intel on potential problems. Some judges can do an amazing job, but we simply can\u2019t overlook that this system creates a choke point which quickly leads to delays when overwhelmed.  Tens or possibly hundreds of tables are too much to keep in your head all at once.<\/p>\n<p><a href=\"http:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/3-EOR-Zone-JPEG.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/3-EOR-Zone-JPEG.jpg\" alt=\"3-EOR-Zone JPEG\" width=\"2309\" height=\"1435\" class=\"aligncenter size-full wp-image-550\" srcset=\"https:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/3-EOR-Zone-JPEG.jpg 2309w, https:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/3-EOR-Zone-JPEG-300x186.jpg 300w, https:\/\/blogs.magicjudges.org\/bearz\/files\/2015\/08\/3-EOR-Zone-JPEG-1024x636.jpg 1024w\" sizes=\"auto, (max-width: 2309px) 100vw, 2309px\" \/><\/a><\/p>\n<p>Zone coverage, on the other hand, has the opposite problem: There are more judges involved, and more moving parts, so there are more places where things can fail. But the task is much simpler. Any L2 can use the skills they\u2019ve honed managing their local PPTQs to cover 50-100 tables. It\u2019s easy to just walk and physically check on any question-mark matches, and when things do go wrong, the problem is contained to a small area of the tournament, so it\u2019s a manageable problem to triage. When the zone leads feel like all of their matches are covered, their hand-written list can be compared against the outstanding matches from the scorekeeper. Since this comparison happens later in the round, it\u2019s a much more manageable list.<\/p>\n<p><p><img  style='float:left'  class='lems-mtg-cardimg' src='http:\/\/gatherer.wizards.com\/Handlers\/Image.ashx?size=small&type=card&name=Contested War Zone'><\/p>Both systems can work. For a smaller flight, when the number of tables is more manageable, a clipboard offers an efficient, centralized solution. It allows a more experienced judge to centrally distribute resources throughout the event in an optimized fashion, which is handy with a green staff. Conversely, zone coverage empowers judges to \u201cown\u201d their areas and be proactive about getting things covered in manageable chunks. It works great when you have a team of judges that can solve problems with little instruction, but it may require an additional management layer to make sure judges are spread throughout the entire event.<\/p>\n<p>One of the best aspects of the judge program is that it\u2019s full of very smart people, who\u2019ve come up with many possible ways to run events. Choosing among these options, however, is a major challenge in running large events. What works for one event may not work for others; intellectually, we know the most robust solution will depend on the specifics of the event in question&#8211;the number of players, the number of staff and their experience, and so on.  FMEA allows us to quantify these factors, and tease out the most robust solution for any given event or staff.<\/p>\n<p><H2>Moving Forward<\/h3>\n<p>We\u2019re seeing more and bigger events as the game grows, and it doesn\u2019t look like things will slow down anytime soon. We often talk about how each event is another opportunity to learn; the dark side of that is that each additional event is another opportunity for things to go wrong&#8211;and they will. We\u2019ll use the wrong tool for the job, we\u2019ll forget exactly how some system is supposed to work, or we won\u2019t have enough resources to implement our systems successfully. <\/p>\n<p>Failure will be more and more normal in our events.<\/p>\n<p><p><img  style='float:right'  class='lems-mtg-cardimg' src='http:\/\/gatherer.wizards.com\/Handlers\/Image.ashx?size=small&type=card&name=Favor of the Mighty'><\/p>&#8230;but the show must go on.  The best events are those that succeed in spite of challenges; they\u2019re ones where the players have a fantastic time.  If systems fail more often, that doesn\u2019t mean our events have become horrible.  What it really means is that we need to design and implement our systems to make every event great even when things fail.<\/p>\n<p>FMEA is one of many tools that can help us prepare for failures and make sure they don\u2019t spiral out of control.  With a bit of attention, a dash of preparation, and a lot of hard work, we\u2019ll bring Magic to new heights.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>What will you do when things start going wrong at your next event?  Prepare yourself by reading Chase Culpon&#8217;s guest post on failure mode and effect analysis!<\/p>\n","protected":false},"author":63,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[4,14],"tags":[],"language":[],"class_list":["post-547","post","type-post","status-publish","format-standard","hentry","category-events","category-guest-post"],"_links":{"self":[{"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/posts\/547","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/users\/63"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/comments?post=547"}],"version-history":[{"count":19,"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/posts\/547\/revisions"}],"predecessor-version":[{"id":569,"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/posts\/547\/revisions\/569"}],"wp:attachment":[{"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/media?parent=547"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/categories?post=547"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/tags?post=547"},{"taxonomy":"language","embeddable":true,"href":"https:\/\/blogs.magicjudges.org\/bearz\/wp-json\/wp\/v2\/language?post=547"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}