Need for Robust Systems Engineering in a Time of Budget Austerity
Need for Robust Systems Engineering in a Time of Budget Austerity
+1 (703) 653 5952
Third Millennium Systems LLC
+1 (703) 757 7644
William F. Neuendorf
+1 (703) 625-1811
Today’s larger, more complex systems are riskier than before, with dire consequences if they fail. This paper makes the case that systems engineering is needed now more than in the past, and it would be a mistake to reduce funding for systems engineering at the same time that national security programs in general are being reduced.
Systems engineering helps improve decision making under conditions of uncertainty and complexity, using a combination of proven and new tools. Systems engineering orchestrates integration among disciplines and subsystems using collaborative tools such as integrative review. Systems engineering supports new acquisition models such as incremental commitment by implementing enterprise architecture roadmaps and recommending new technology to improve capability. Model-based systems engineering leads to lower cost, improved capability, lower risks, and improved defenses against attack.
Systems engineers are learning techniques from complexity science such as hierarchical and modular design, prevention of failures from unintended consequences, and analysis of power law characteristics in complex systems. The paper discusses these concepts to show the importance of maintaining systems engineering in the face of budget austerity.
Introduction: The Systems Engineering Need
Twenty-first century acquisition is fundamentally more complex than previous acquisition approaches, in that today’s systems not only need to be more highly capable than before, they need to be built more quickly, and at less cost. They also need to interoperate with other systems, including legacy systems, systems being built today, and perhaps systems that are not even designed yet. The problem of engineering the big picture to accommodate these requirements is the responsibility of systems engineering.
Prominently visible in the aerospace, defense, and energy sectors are systems characterized by the common trait that they “must work”; failures become societal events and national tragedies. Three-Mile Island, Challenger, Columbia, the power blackouts of 1965 and 2003 in the northeast United States, and the recent Gulf of Mexico oil spill are unfortunate examples. These sectors are likewise characterized by the necessity to analyze, understand, and predict the “in the large” consequences of decisions, actions, and designs having limited scope, but which actually are of profound import, too often seen only in retrospect. (Griffin 2010)
The purpose of this paper is to make the case that systems engineering is needed now more than in the past, and it would be a mistake to reduce funding for systems engineering at the same time that national security programs in general are being reduced. Systems engineering in fact is what will protect the mission and the nation, and minimize the effect of budget cuts.
The objective of systems engineering is to see that the system is designed, built, and operated so that it accomplishes its purpose in the most cost-effective way possible, which considers performance, cost, schedule, and risk. The value of systems engineering depends on how well the system performs and the cost of the programs to build it and to run it. Up to a point (about 15% of a program), “better/more systems engineering correlates to shorter schedules...[and] lower development costs”. (Honour 2004) “Inclusion of greater [systems engineering] effort can improve software productivity by factors from 18% (small software projects) to 92% (very large software projects).” (Boehm, Valerdi and Honour 2008)
To provide this value, systems engineering juggles two different focuses. The first is to perform system design and build tasks that are specific to the life cycle phase. These start with identifying what the system must do (mission purpose and operational concept through requirements), and designing the system (architecting and allocating to subsystems); continue in mid-program with ensuring that subsystem decomposition is true to the system concept and that parts bought or made are acceptable, and integrating and testing the system; wrapping up with finishing system validation and helping transition to operations.
The second focus is to continuously perform overarching tasks that help ensure system performance and program cost effectiveness by maintaining the alignment of system functionality with strategic goals. Understanding and describing the problem, maintaining technical integrity of the system architecture (Sheard 1996), continuously ensuring that the system being built is the right system to meet the strategic purposes of the customer are some of these. Others are performing multidisciplinary analyses for performance prediction or tradeoff support, (Boehm, Valerdi and Honour 2008) providing metrics and other information for program management, and helping disciplines communicate details and a big picture view.
At this moment, the government is challenged with an uncertain picture of current program funding and with the likelihood of further changes in funding. Cost cutting is highly likely in the immediate future. Systems engineering has historically been vulnerable to budget reductions. Because the very nature of systems engineering is to make decisions in a cost effective manner, the need for systems engineering is particularly acute for more cost-constrained programs. (Lloyd 2007) Systems engineering helps in three ways. First, systems engineering prior to a cost cut ensures that the system (or SoS) design is as modular as possible in order to be robust to programmatic changes. Second, during a cost cut systems engineering helps identify which cuts will have the minimum effect. Third, after a cut systems engineering helps establish the appropriate tradeoffs to ensure the remaining system is optimized for performance given the remaining funds.
By applying the correct level of systems engineering techniques, talent, and tools, to the system acquisition problem, the government can maintain mission success while operating under reduced budgets. This is particularly necessary to cope with changes in the world today.
How the World of System Development Is Changing
Twenty-first century life has increased in complexity significantly. The pace of change has increased as well. Today’s systems are more costly, uncertain, complex and risky than before, with dire consequences of failure. Our economic, political, and infrastructure systems now depend heavily on information that is globally interconnected. The systems are under threats that are remote, continuous, and changing.
For companies and even for whole economies, to remain economically successful requires continuous introduction of new technology. New usages of legacy systems, ubiquitous software with undocumented behavior, and attempts to apply new technologies such as cloud computing can increase the risk in any domain. Yet failure in these large critical systems is not acceptable. (Griffin 2010)
Twenty-first century warfare must deal with technologically-skilled, asymmetrical adversaries. Not only are they hard to identify, locate, and quash in an enduring manner, many also rapidly obtain new technology, to our detriment. This makes leapfrogging their capabilities critical, and it reduces the time to specify, compete, and build new systems.
Acquisition complexity has markedly increased, from building systems such as a tank or aircraft, to developing ongoing, evolving systems that include tanks and aircraft, but also platoons, squadrons, space systems, ground systems, sea systems, computer systems, and people systems. Within such a complex space, even identifying a scenario (the first step in determining an operational concept for such a system-of-systems) is difficult. In today’s “global interactive brown-field,” no longer can systems be designed from scratch, but rather capabilities must be created by adapting and adding to existing components. Systems of systems must be adapted, reused, improved, and changed out.
An evolving version of systems engineering is critically required to be able to describe and adapt the ongoing system concept to the needs of today, in a technologically possible yet cost effective manner. Simplistic processes are no longer enough to guarantee that the engineering of the system is assured. A lack of good systems engineering can lead to situations that are not even manageable. Yesterday’s systems engineering is no longer adequate.
How Systems Engineering Is Adapting to Meet the Changing Needs of Customers
Systems engineering is adapting to changes in the world and in systems by recognizing and focusing on complexity, by using the power of ubiquitous computers to model the systems and the situations they are operating in, and by coordinating with enterprise architecture.
Systems engineering uses tools to deal with complexity and uncertainty. Systems engineering uses tools to deal with complexity and uncertainty.
Systems engineering improves decision-making under uncertain conditions. Systems engineers employ statistical analysis methods in economic analysis, operations research, and management science. (Griffin 2010) Systems engineers identify the uncertainty that comes from the statistical properties of the cost estimating relationship and the uncertainty from detailed technical and schedule or programmatic risks. Systems engineers use Monte Carlo simulations to develop cost S-curves, assigning probability distributions to the continuous inputs in the estimating relationship.
Systems engineers improve integration with collaboration tools. Systems engineers use collaboration tools to ensure multiple disciplines and multiple organizations can obtain the information they need to work effectively apart and together, so that individual systems will support the capability required and the customer’s need. Process enactment tools ensure that whatever is known about the task ahead can be performed as specified, saving human decision making for uncertain situations, anomalies and contingencies.
Systems engineers orchestrate integrative review. Today’s systems are too complex and too well-connected for any individual to be able to understand a whole system. As a result programs break large tasks into smaller tasks, and allocate the smaller tasks to different engineers. Even if the engineers are successful at comprehending and managing their assigned tasks, the interactions are now compromised. To remedy this, systems engineers obtain integrative review by experts in other areas, such as use of the system, details of interfacing subsystems, testing, systems and software security. (Boehm and Lane 2007b)
Systems engineering supports incremental commitment acquisition. To deal with uncertainties in competition and technology evolution, and with changes in organizations and mission priorities, agencies have been seeking a type of acquisition that allows more gradual financial commitment. The difference has been described as the difference between Roulette and poker. In Roulette, a full set of resources (bet) is committed to a fixed-specification contract (number) at one time, and then the acquirer waits to see whether the bet pays off. Incremental commitment is more like poker, which involves smaller bets placed incrementally on an evolving game, with opportunities to ask for new cards and modify the bet as the game matures and more information becomes available. (Boehm and Lane 2007a) Longer development cycles make it more likely that several of these uncertainties or changes will occur and make the originally-defined system obsolete. Systems engineers plan system development using short increments, to help ensure that early, high priority capabilities can be developed and fielded and changes can be more easily accommodated in future increments.
Systems engineers implement enterprise architecture roadmaps. The “to-be” enterprise architecture presents capability roadmaps which may call for developing or modifying individual systems to serve as components of a new capability. Systems engineers, who are responsible for the strategic capability throughout the system life cycle, define and maintain the critical dependencies among system components. These dependencies are logically addressed by organizing capability-related components into a portfolio. A given component may reside in multiple portfolios, supporting multiple capabilities.
Systems engineers recommend where new technology can improve capability. Systems engineers identify emerging technologies such as web services, grid computing, virtualization, autonomic computing, and migration to on-demand adaptive environments. They assess each technology’s potential benefits and costs, both variable (a function of consumption) and fixed (independent of consumption).
Systems engineers create useful and enduring models: Model Based Systems Engineering (MBSE). As systems engineering tools become more software based, and as computing capability becomes ever more powerful and interoperative, models are becoming more capable and more important. Models can be used for reducing cost or risk, and for improving both understanding and capability. Models, however, are not intelligent; they must be applied by smart people.
Because all models simplify reality, all models are wrong; however, some are useful. (Box 1987) To make models useful requires smart development, discovery, use and reuse. Smart means the systems engineer understands how and why models can be wrong, and what makes them useful, as well as how to represent systems and their aspects by a model that preserves the features that are important to preserve while simplifying out clutter. Cost of model-building and maintenance must be balanced against utility of the model to perform necessary systems engineering tasks.
Model-based systems engineering has many beneficial effects:
Model Based Systems Simulation is a kind of discrete modeling showing stochastic results and variations called “what ifs”. The results can be used to identify easiest, cheapest, and most capable builds from variations of existing equipment. As part of all program trade-off studies, systems engineers quantify and manage total ownership costs.
A life-cycle cost management program identifies a common set of ground rules and assumptions for life-cycle cost estimation; ensures that best-practice methods, tools, and models are used for life-cycle cost analysis; track the estimated life-cycle cost throughout the project life cycle; and most important, integrate life-cycle cost considerations into the design and development process. Program Management’s Cost as an Independent Variable (CAIV) methodology creates a CAIV model that is utilized throughout the entire life cycle of the acquisition process. Via the CAIV model, systems engineers provide tradeoff study analyses that are input into decision making.
Improving Capability. Models allow testing in situations where physical tests would not give good results. For example, it is nearly impossible to produce true zero-gravity test conditions on earth, so it is usually better to simulate. Models can simulate physical quantities such as the interference of radio frequencies between a active synthetic aperture radar and its host satellite subsystems, a test that might severely degrade system performance if performed on a real system. Models also allow injection of specific faults into specific places at specific times, where it would be difficult to get the exact desired fault to happen at the appropriate time. In this manner, simulations can help trace anomalies back to causes.
Defending against attacks. Models are important in a variety of program protection methodologies, ranging from software and system assurance, through cyber security. Systems engineering models range from models of attackers to models of defense layers and of vulnerabilities. Models serve well when an attack strategy should not be attempted in real life, but there needs to be some way to show what would happen if they were encountered.
Reducing Risk. Technical risk is reduced by models that improve understanding of a system and how it may break. (Petty 2010) Stochastic simulations can highlight which systems, elements, or components are most likely to break. Continuous simulations can identify where problems may cascade. Together the risk assessment would feed mitigation planning that reduces technical risk.
Why Concepts from Complexity Science Will Drive Systems Engineering Solutions
The tools described in the last section are evolving to better address the increasing complexity of today’s and tomorrow’s systems. Insights from the science of complexity, which has its roots in the 1970s in the study of chaos and fractals, are now being applied to the engineering of large-scale complex systems. This is adding approaches, methods, and some tools to the systems engineering arsenal. Some of the newer concepts include hierarchy, modular design, unintended consequences, and power law distributions.
Design using hierarchies and modules. Complex systems throughout nature have evolved hierarchical principles and modular designs. For example, animals from fruit flies to humans share gene sequences for muscle function. (Wang 2007) Once a function is shown to work, it is easier and more reliable to keep using it rather than tweak it and possibly experience unknown problems. Similarly, systems engineers have learned the principle of modular design from hardware in the Henry Ford era and from software more recently. By identifying and reusing or adapting technologies that have performed a specific function well in the past, systems engineers minimize program risk.
The hierarchies that are created in complex systems derive from the principle of process reuse. It is very easy to grow a hierarchical complex system because the rules are simple. A tree has the rules “grow a bit, then divide” which starts on the stem of a tiny plant and continues out through the twigs, creating a fractal structure. Fractals are forms that are self-similar at many scales. “Building blocks” of systems engineering standards (ANSI/EIA 1999) demonstrate fractal concepts that produce hierarchical structures, as are teams at multiple levels and organizational charts. Systems engineers apply complex systems concepts such as hierarchy and modularity to make robust systems.
Predict and prevent failures from unintended consequences. Higher capability systems have inherently more complexity systems than simple systems. Ashby showed that a control system must have a complexity at least equal to the complexity of the situation being controlled. (Ashby 1956) Because of this, complex systems may experience unintended consequences. (Petty 2010) A particularly troublesome type of unintended consequence is cascading failures, which includes cracks propagating within mechanical materials and even large-area power outages. Systems engineers identify and prevent such failures by studying failures that have happened in system elements or in similar systems, and performing risk analysis across interfaces within highly interconnected complex systems.
For example, the traditional Concept of Operations of a system and the influence of environment on a complex system are treated as models of the interaction of a system with its environment. The difference in the models is a traditional model assumes the system does not adapt to the environment and does not change the environment except in a predetermined manner. The complex model assumes that when a system enters an environment, both the system and environment change each other in varying degrees. (Hyberston 2011)
Identify and use power law situations. Power law distributions describe systems with a few big components, a medium number of medium-sized components, and very many small components. Pareto charts are an example of power laws on real programs, showing a few large action items on the left and many small ones trailing off to the right. Another example familiar to systems engineers is the “80/20 rule”: 80% of a program’s problems are caused by 20% of the elements, for example, or 20% of the problems will require 80% of the resources to fix.
Power laws are ubiquitous within complex systems theory. Fractal structures exhibit power laws. Systems engineering processes, which are performed in similar ways at all scales, have a fractal structure. One consequence is that 80% of what is done in any systems engineering enterprise will use a common 20% of all possible processes. Of all possible tasks that ever could be performed by a systems engineer, 80% are used rarely, if ever. Documenting the 20% most important and commonly used processes covers 80% of needed processes. Documenting the other 80% would at best provide decreasing returns; at worst the effort is unending.
When we document the 20% of most-needed processes, we implicitly allocate knowledge of the other 80% of the processes to humans. Normal behavior can be documented in processes; atypical or anomalous issues require human experience and knowledge. When an engineer gets to a problem that is unique to a domain or unprecedented or not addressed specifically in processes, the engineer has to use judgment to determine what to do next. This means that systems engineering is critically dependent on the experience, ability, and judgment of the systems engineer.
Evolving Considerations Will Require High-End Analytical Systems Engineering Solutions
System complexity will increase. In the future, systems will become bigger and even more critical, not allowed to fail even momentarily without unacceptable consequences. To achieve these goals, systems engineering will need well-controlled and high-assurance processes that provide system synchronization, balance, assurance, and agility. (Hybertson 2011) Multi-owner, multi-
Multi-owner, multi-mission systems of systems will dominate, possibly using a more integrated supply chain, to supply systems that may serve multiple systems of systems. Since no one-size-fits-all solution will satisfy all stakeholders, prioritizing will be continuous and critical.
Good systems engineering will identify needed and unanticipated emergent behavior resulting from the complexity. Requirements will evolve and will be recognized as not pre-specifiable. Budgets and schedules also will evolve. Growing systems will continue to use systems and systems of systems that work, and will evolve them to create new capabilities. Both uncertainty and risk will be managed proactively.
The rapid pace of change will continue. The rapid pace of change experienced today will continue and probably accelerate. Competitions will focus on mission priorities, adaptation of technology, modular use of Commercial Off-the-Shelf (COTS) technology, and understanding of the system environment. Incremental development will be employed to speed deployment, reduce cost risk, and avoid technology obsolescence. (Boehm and Lane 2007a) Concurrent processes will provide speed advantages over sequential processes, managing the inherent risks explicitly. Model-based engineering will provide both prescience and rapid adaptability.
Integration will be more challenging. The potential level of integration risk is often substantial because of a tendency to underestimate integration difficulty, and simultaneously overestimate the maturity of items that require integration. (Conrow 2005) Although understanding of the characteristics of massive amounts of interacting software will be important, it will also be important to acknowledge the role of humans within complex systems, whether they are users, adversaries, customers, or other roles. It will be as important as ever to manage legacy elements, in providing continuity of legacy services, in accommodating the attendant constraints, and in removing obsolete elements from service in a smooth manner. (Boehm and Lane 2007b)
Systems engineering takes on new roles. Systems engineers will take on new roles to deal with these challenges. They will be integrating an ever-increasing number of disciplines, so they will have to learn about each new discipline. They will be learning more about various models and tools so that they can identify appropriate highly capable tools and provide people skilled in using them. (Hybertson 2011) They will be helping project management by owning the mathematics needed to deal with complexity, from stochastic statistics to chaos and complexity as well as combinatorial computation, in addition to more historical engineering math such as Fourier and Laplace transforms.
Conclusion. Good high-end, analytical systems engineering is needed now more than ever. If Government cost cutting will require reductions in new systems, we will need a robust systems engineering capability to minimize the impact to the overall mission utility, while meeting tightened cost constraints. For these reasons, it is important that the national security structure within the government maintain support to the high-end systems engineering base to maintain our qualitative edge in our military and intelligence systems.
American National Standards Institute/Electronic Industries Alliance. Processes for Engineering a System (ANSI/EIA 632), January 1999.
Ashby, Ross. 1956. “Cybernetics and Requisite Variety”, An Introduction to Cybernetics, Accessed 21 October 2011. http://www.panarchy.org/ashby/variety.1956.html
Boehm, Barry, Ricardo Valerdi, and Eric Honour. 2008. The ROI of Systems Engineering: Some Quantitative Results for Software-Intensive Systems. Systems Engineering 11(3).
Boehm, Barry and JoAnn Lane. October 2007. “Using the incremental commitment model to integrate systems acquisition, systems engineering, and software engineering.” CrossTalk – The Journal of Defense Software Engineering 20(10): pp. 4-9. (Boehm and Lane, 2007a). 2007. “System of Systems Lead System Integrators: Where Do They Spend Their Time and What Makes Them More or Less Efficient?” Wiley Interscience. (Boehm and Lane, 2007b)
Box, George E. P. 1987. Empirical Model-Building and Response Surfaces, co-authored with Norman R. Draper, p. 424, ISBN 0471810339.
Conrow, Edmund. February 2005. “Risk Management for Systems of Systems.” CrossTalk – The Journal of Defense Software Engineering.
Griffin, Michael D. “How do we fix systems engineering?” 61st International Astronautical Congress, Prague, Czech Republic, 27 September–1 October 2010.
Honour, Eric C. 2004. “Understanding the Value of Systems Engineering.” Proceedings of the Fourteenth Annual International Symposium of the International Council on Systems Engineering, Toulouse, France.
Hybertson, Duane “Next Generation Systems Engineering: Expansion, Foundation, Unification” The MITRE Corporation McLean, VA INCOSE 2011.
Lloyd, Jim. October 2007. “While You Were Sleeping: The Loss of the Lewis Spacecraft” NASA Leadership ViTS.
Petty, Mikel “Modeling and Validation Challenges for Complex Systems” Huntsville, AL, AMSC Complex Systems M&S Workshop, 3 February 2010.
Sheard, Sarah A. “Twelve Systems Engineering Roles” Proceedings of the Sixth Annual International Symposium of the International Council on Systems Engineering, Boston Massachusetts, July 7-11, 1996.
Wang, Zhenyu. Dec. 2007. “The Emergence of Modularity in Biological Systems,” Accessed 21 October 2011. http://guava.physics.uiuc.edu/~nigel/courses/569/Essays_Fall2007/files/Wang.pdf
Rich Rosenthal is the Chief Technology Officer at TASC, where he sets TASC’s direction for technology investments, establishes thought leadership for the technology base, manages intellectual property and supports technical personnel development. He has over 28 years of experience in technology development, engineering, management and investment analysis. His previous roles include manager of the Advanced Concepts department at TASC, and engineering control and navigation systems, and selecting IRAD projects, at TRW. He also has worked as a technology analyst and portfolio manager at a buy-side investment firm, focusing on technology industries. Mr. Rosenthal is Project Management Institute-certified Project Management Professional, a senior member of American Institute of Aeronautics and Astronautics, and a member of the Institute of Electrical and Electronics Engineers and INCOSE. He currently serves on the AFCEA Technology Committee. He earned a bachelor’s degree in aero/astro engineering (avionics option) from the Massachusetts Institute of Technology (MIT), and a master’s of science electrical engineering degree in signal processing, communication systems and optics from the University of Southern California. He has two patents in spacecraft design, and has presented at several conferences on systems engineering.
Sarah Sheard is the Principal at Third Millennium Systems, where she consults with private companies and government agencies regarding systems engineering processes, systems engineering curriculum, and interfaces to software and project management. Ms. Sheard also teaches courses ranging from basic systems engineering to advances for the future, including application of the sciences of complex systems to systems engineering. She has 30 years of experience in systems engineering, process improvement, and curriculum development and implementation. An INCOSE Fellow and winner of the INCOSE Founder’s award, in 2012 Ms. Sheard earned a doctorate at Stevens Institute of Technology with a focus on correlation of complexity to program outcomes.
William F. Neuendorf is a Senior Systems Engineer at TASC, where he develops systems and software engineering processes, methods and tools. He has 35 years of experience designing and developing engineering, scientific and manufacturing systems, and process improvement. Mr. Neuendorf is a FEAC Certified Enterprise Architect, and a member of INCOSE. He holds a Bachelor of Science in Mathematics and a Master’s of Science in Engineering Management.