Procuring Targeted Lethality for Accelerated Readiness
Colonel Justin W. Eggstaff
Director, Marine Corps
Operational Test and Evaluation Activity
From Armor & Mobility, October 2019 Issue
Colonel Justin W. Eggstaff currently serves as the Director of the Marine Corps’ Operational Test and Evaluation Activity (MCOTEA). Colonel Eggstaff was commissioned in January 1996 through the Marine Corps’ Platoon Leader’s Course. After graduating from The Basic School in October 1996, he was designated an Unrestricted Naval Aviator in August 1998. Assigned to fly the AH-1W Attack Helicopter, Colonel Eggstaff served with HMLA-369 at Camp Pendleton, CA deploying multiple times to the Asia-Pacific and Middle Eastern theaters. While at HMLA-369, he served in numerous squadron billets, including the Logistics Department Head, Ground Safety Officer, NATOPS Officer, NATOPS Instructor, Human Affairs Officer, and several maintenance officer billets. In addition, Colonel Eggstaff served as a Forward Air Controller (FAC) with the 3rd Battalion/5th Marine Regiment.
In June 2003, Colonel Eggstaff was selected to the U.S. Naval Test Pilot School Cooperative Program. After graduate school at the Air Force Institute of Technology in Dayton, OH, and Test Pilot School at Naval Air Station Patuxent River, MD Colonel Eggstaff served as the lead H-1 Developmental Test Pilot and Operations Officer for the Navy and Marine Corps’ rotary wing test squadron, HX-21. While at HX-21, Colonel Eggstaff became a member of the Defense Acquisition Corps (DAC) and was selected for the Marine Corps’ Professional Acquisitions Community (PMOS 8059). Following his assignment as a Developmental Test Pilot, Colonel Eggstaff served in numerous billets at Naval Air Station Patuxent River, including: H-1 Class Desk, Chief Engineer, Deputy Program Manager, and Marine Detachment Executive Officer. In July 2013, Colonel Eggstaff was selected to attend Top Level School at the Eisenhower School, National Defense University. At the completion of Top Level School, he was assigned to the Joint Staff as a Requirements Officer in the Capabilities and Acquisition Division of the J-8 Directorate. In July 2017, Colonel Eggstaff was selected to serve as the Military Deputy to the Deputy Assistant Secretary of Defense for Manufacturing and Industrial Base Policy in the Office of the Secretary of Defense.
A&M: Please provide some background into your role and mission in today’s global operational environment.
Col. Eggstaff: The U.S. Marine Corps established the Marine Corps Operational Test and Evaluation Activity (MCOTEA) as a way to provide the Commandant of the Marine Corps with an independent assessment of materiel solutions in order to inform production decisions. At MCOTEA, we take a combat system like the new Joint Light Tactical Vehicle (JLTV), give it to Marines, place them in a realistic operational environment, collect data on how well they accomplish their mission with the new equipment, survey the Marines to hear their thoughts, and report about how effective, suitable, and survivable that system can expect to be if purchased for the Marine Corps.
A&M: With Operational Test Activity (OTA) involvement throughout the acquisition lifecycle, how does your team focus target aspects to shorten OTA and lengthen optimal system life?
Col. Eggstaff: Can I start with an analogy? At some point, we’ve all been given a project from our boss. How frustrating would it be to go back to your boss to show what you’ve done only to have your boss say, “That’s not what I wanted! Try again.” The waste…time, money, opportunity lost on other things you could have been doing. No way! For any project, you incorporate status checks, regular updates, etc. to ensure your work is on track with your boss’s vision so that you don’t waste precious resources. Traditionally (or historically), so-called “Operational Testing” in DoD acquisitions occurs towards the end of a program’s development cycle. By DoD Instruction 5000.02, the capstone Independent Operational Test & Evaluation or IOT&E event occurs after system development and the Milestone C Acquisition Decision (effectively, the decision to enter low rate initial production (LRIP)) and prior to approving Full-Rate Production (FRP) and Fleet introduction. In that view, IOT&E is sometimes referred to as the program’s “Final Exam.” Of course, waiting until the very end of the Engineering, Manufacturing and Development (EMD) portion of the acquisition lifecycle to finally hear how the system performs under realistic conditions is a terrible idea. By that time, it’s often too costly to make meaningful changes or worse, start from scratch. At MCOTEA, we start by “imagining” what a full-blown IOT&E would look like if we were to actually wait until that production representative test article were available after Milestone C, break down the mission sets into measurable components and knowledge points, then look across the development cycle to find where we can gather as much data as soon as possible in order to provide that insight to the decision-maker. That translates into a shorter and more deliberate IOT&E AND the better chance we have to catch an issue before it becomes too late or too costly to fix. The more we “know” about a system before we have to test it, the less we have to “find out” during the Final Exam.
A&M: As the testing morphs to demonstrating a system in an operational environment, talk about some strategies your team is using to maximize true results.
Col. Eggstaff: The difference between “testing” and “demonstrating” a system in an operational environment has a very specific nuance that is incredibly important to us at MCOTEA as it relates to uncertainty. Testing ensures that a product works, while a demonstration assumes that a product works. We’ve all walked through the mall and have been asked to watch a salesman demonstrate a product. Through one demonstration, the salesman’s intent is to convince us that their product has an intended effect. However, the decision to buy this product comes with a high level of uncertainty. What happens when you get it home? Will it perform the same way it did in the mall? Is your home even like the environment in the mall? Did the salesman have certain tricks for the making it work? All of this leads to uncertainty. Of course, the salesman knew that the demonstration would be successful because he’s repeated it enough times to know that it would work. At MCOTEA, our job is to make sure the products the Marine Corps intends to purchase have been repeated enough times and in all of the scenarios and intended environments that we are as certain as the salesman before we decide to buy. We work to develop a testing strategy that uses the power of repeated demonstrations over time to reduce uncertainty. Our accumulation of knowledge over time informs the various decisions throughout the acquisition lifecycle. Ideally, we are involved during requirements generation. Starting at that “good idea” and continuing throughout the development of a system, if we hope to field systems rapidly at the “Speed of Relevance” in accordance with the National Defense Strategy, we must start with the end-state in mind.
A&M: Some challenges in today’s Marine Expeditionary Force (MEF) are dictating greater flexibility of deployment, can you speak to some areas?
Col. Eggstaff: The Marine Corps needs to be adaptable and able to tailor its forces to meet existing and emerging adversaries. In order to maintain flexibility and deploy Marines at the cutting edge of technology, we must be able to rapidly prototype and field capability. In other words, we are now in an environment where decision-makers are willing to take acceptable risk and field systems that are “good enough.” When “speed” is the name of the game, many consider our traditional development processes “broken.” They’re not. They’re just not designed to provide less than “complete” solutions. Our so-called traditional acquisitions processes, whether you call it DoD 5000, a waterfall approach, a systems engineering “V,” or another classic development process, are designed to systematically buy down or eliminate risk. Their intent is to navigate from a user need or capability gap to a robust, stable materiel solution, complete with mature logistics support and supply chain, adequate training, and tactics, techniques and procedures that ensure a seamless introduction and integration to the Fleet. Sure, we can tailor these processes in an effort to go faster; however, eliminating bureaucracy, “trimming the fat” or otherwise skipping so-called unnecessary steps has proven hard to do. Acquisition professionals across DoD are trying to figure out innovative ways to develop and deploy systems faster.
A&M: In terms of rapid prototyping and fielding, elaborate on ways your team is working to shorten the front-end of the acquisition lifecycle.
Col. Eggstaff: So, I stated that we are now in an environment where decision-makers are willing to field systems that are “good enough,” realizing that waiting for that complete solution will take too long. Decision-makers are willing to take acceptable risk. That being said, decision-makers must know the truth, good or bad, about the system they intend to field. So, what does that mean for MCOTEA? It means that operational testing, especially when rapidly prototyping and fielding, must be understood for its true purpose, reducing uncertainty, not reducing risk. We want the decision-maker to understand the risk they are preparing to take. I’ve been talking a lot about how we are working to be involved throughout the acquisition timeline. I’ve argued that operational test shouldn’t be considered a final exam for an acquisition program. That doesn’t change. In fact, it now becomes more important than ever. If we are going to field systems faster, decision-makers need to understand the risks associated with a system in order to decide when they’re at an acceptable level to deploy. In order to facilitate this, we’ve been working to characterize a system’s risk in terms of their suitability and effectiveness in the clearest terms possible. One area where I feel we’ve really made headway is in cyber. Instead of simply listing gaps or vulnerabilities in the system, we’ve begun to explain cyber survivability in terms of a system’s resilience and durability, much more meaningful metrics of a system’s survivability in a cyber contested environment.
A&M: From a cybersecurity testing perspective, speak to the resilience and durability of USMC expeditionary systems.
Col. Eggstaff: Let’s say you’re in your car. Do you need gas? It depends. Some people wait until the low fuel light comes on, while others look for a gas station at ½ tank. We may not all agree on when to fill up, but we can agree on the objective measure that informs our opinion: the gas gauge. The gas gauge is a way to assess risks. If we’re in town near a gas station, the risk may be low, but if we’re on an empty desert road with an empty tank, then the risk is higher. No matter the situation, the gauge shows how much fuel is left, and we can decide how much further we’re willing to drive before getting fuel. So, what if we had a gas gauge for cyber? With a cyber gauge we could determine how risky it is to take on a mission, and how many missions we can expect to complete before the risk gets too high. A gauge like this could help us decide if our system was good enough, or if we needed to stop to “fill up with gas.” MCOTEA’s cyber gauge measures operational resilience, the ability to resist, absorb, and recover from adverse cyber impacts to mission-essential functions. It’s sophisticated enough to consider a multi-function, multi-impact environment. It enables a decision-maker to look at this gauge, and if the risks are too high, then they can decide to spend more time and money fixing vulnerabilities. The cyber gauge has one other feature that’s useful, it changes with use. It helps us see the durability of our resilience over time and helps us understand how much current and future risk we face. With this information decision-makers can consider how much risk they are willing to accept. If they want to reduce risk, they can put more “gas in the tank” (i.e., fix vulnerabilities, update anti-virus, etc.) and buy back resilience.