By Grace LewisTechnical Lead Edge-Enabled Tactical Systems Research In 2009, a popular blogger published a post entitled "SOA is Dead," which generated extensive commentary among those who work in the field of service-oriented architecture (SOA). Many practitioners in this field completely misinterpreted the post; some read the title and just assumed that the content referenced the demise of SOA. Quite the opposite, the post was inviting people to stop thinking about SOA as a set of technologies and start embracing SOA as an approach for designing, developing, and managing distributed systems that goes beyond just the technology. Unfortunately, even though SOA is still alive and widely adopted, a belief still persists that SOA can be purchased off the shelf. This post highlights recent research aimed at clarifying this misperception for architects, as well as identifying the elements that constitute a service-oriented system and the relationships between these elements. Service Orientation in a Nutshell Service orientation is a way of designing, developing, and deploying software systems. The architecture of a service-oriented system can typically be characterized by three high-level components: services that represent a reusable business functionality exposed via standard service interfaces service consumers that use the functionality provided by these services as part of the processes they implement an SOA infrastructure that connects service consumers to services Rather than being viewed as an off-the-shelf product, SOA should be viewed as an architectural style or paradigm for designing systems. From this paradigm, an infinite number of configurations can be derived that either benefit or harm a system. Even organizations that acquire products to build a service-oriented system from a vendor must contend with numerous architectural and design decisions. These decisions include deciding what services to expose and whether the functionality exposed comes from legacy systems, is built from scratch, or is acquired from a third party.  Other critical considerations include deciding how information about the system will be accessed and shared, as well as how to promote quality attributes that are important for a particular organization. For example, service-oriented systems typically promote interoperability and modifiability at the expense of optimal performance and stringent security. Implementing a Service-Oriented System Many architectural and design decisions must be made when implementing a service-oriented system. When an organization acquires an SOA infrastructure product, such as an enterprise service bus (ESB), from a vendor it is buying only one piece of the service-oriented system. The architecting of a service-oriented system, however, begins when the system stakeholders define the quality attributes that must be met by the system, such as easy and flexible integration with legacy systems (interoperability) streamlined business processes (maintainability) reduced costs (modifiability) agility to handle rapidly changing business processes (extensibility) Common components of an SOA infrastructure that can help meet some of these quality attributes include a service registry and repository, which can be custom-built or, more commonly, provided by a product in the SOA infrastructure. Typical functionality includes dependency management, discovery, versioning, event notification, access control, policy management, and federation capabilities. a messaging system, also known as message-oriented middleware, which is often a part of execution environment of enterprise applications. A messaging system allows distributed components to exchange asynchronous messages. Today messaging systems are commonly used in service-oriented systems for transactions that involve background processing because they provide high levels of scalability and reliability. a business process engine, which is a software component responsible for processing incoming requests by performing the steps—defined according to rules—of business processes that correspond to that request. A key element of the business process engine is a business process modeling (BPM) tool that enables the description—and often visualization—of the business processes. The BPM tool then generates a business process that is deployed to and executed by the business process engine when processing requests. monitoring and management tools, which enable organizations to detect, diagnose, and react to potential problems in service-oriented systems. These tools are used for runtime monitoring to provide information that can be used to maintain system quality of service and to inform tactics and patterns, such as dynamic load balancing, failover and recovery procedures, and service removal and replacement.  SOA Architecture Principles Service-oriented systems, like any system, have business mission goals and a corresponding set of desired quality attributes that drive the architecture of the system. SOA, as an architectural style, has a set of service-oriented principles that influence the architecture of a system and impact a system’s quality attributes. When these two sets of quality attributes collide, conflicts arise, and architects must apply each principle in the context of the business goals of the system to make necessary tradeoffs and to make architecture decisions to meet the system’s business goals. Some of the key principles are described belowLoose Coupling Loose coupling can play a great role in impacting quality attributes (either positively or negatively) in SOA adoption. Architects can enable loose coupling by making several key decisions, including ensuring that service providers and service consumers can make independent decisions about technology creating services that are independent, self-contained capabilities that can be used in isolation from those capabilities provided by other services enabling the actual binding between a service consumer and a service at runtime  Reusability Reusability mostly refers to service reusability, which includes services that are reusable because they represent self-contained functionality that can be reapplied in multiple business processes. If service reusability is a goal, architects should identify services that perform utility operations, such as logging and data manipulation, and design them so they are independent of the business processes that use them design (when possible) stateless services that do not maintain a conversational state provide an infrastructure that abstracts or mediates differences between service consumers and service interfaces, such as different versions of standards (e.g., SOAP 1.1 vs. SOAP 1.2) or data formats (e.g., XML vs. CSV). Composability Composability enables organizations to rapidly change elements of a business process in response to changes in the business environment, without impacting consumers of the composite service that implements the business process.  Service composability relies on many of the same characteristics as service reusability including self-contained functionality standardized interfaces availability in a service registry Composability introduces architectural risks that require strategies for mitigation, including compositions with services that span multiple organizational boundaries or transactions that contain services that belong to different systems. Discoverability Discoverability in a service-oriented environment refers to the creation and publication of services in a location (such as a service registry, web page, or directory) that is accessible to and can be queried by service consumers. Metadata associated with a service includes its interface specification or contract in addition to any other data that can help a developer make decisions about whether or not to use that service.StandardizationStandardization is perhaps the service-oriented principle that has contributed the most to widespread SOA adoption. Especially for Web Services, standardization has driven the creation of a large amount of tools and third-party components that can be leveraged by developers and can effectively lead to shorter development and deployment times. In addition, standardization enables the leverage of legacy systems because these can remain in their native platforms while exposing their functionality via standard interfaces. A caveat when using Web Services is that while the basic WS-* stack (HTTP, XML, WSDL, SOAP) has remained fairly stable over the years, other standards such as, WS-Security, WS-Transaction, or WS-Addressing continue evolving and contain areas that are subject to multiple interpretations. Aware of this shortcoming, the WS-I organization was created to promote interoperability by providing clarifications, refinements, interpretations, and amplifications in areas of specific standards that are indeed subject to multiple interpretations. Applying Our Research on Service-Oriented SystemsI authored, with fellow researchers, a technical report that described the effect that service-oriented principles have on system quality attributes. As an architectural style, SOA may be an appropriate solution in some situations, but there will be other situations where it is not appropriate, or it has to be used in conjunction with other technologies to achieve the desired system qualities. SOA architects are often at a conflict because on one hand there are business and mission goals that dictate the quality attributes that are important for the system. On the other hand, SOA architects using service-orientation want to adhere to service-orientation principles and leverage those to the advantage of the system. Our intent in documenting our research is to help architects of service-oriented systems design better systems and understand the tradeoffs. Later this month, at the SEI Architecture Technology User Network (SATURN) Conference (SATURN 2013), my SEI colleague, Stephany Bellomo, will be teaching a two-day course, Advanced Topics in Service-Oriented Architecture, that expands on the concepts presented in this posting. Additional Resources To learn more about the course, Advanced Topics in Service-Oriented Architecture, which will be offered April 29 and 30 at SATURN 2013, please visit http://www.sei.cmu.edu/saturn/2013/courses/index.cfm#advsoaTo read the SEI technical report, Architecting Service-Oriented Systems, please visithttp://www.sei.cmu.edu/library/abstracts/reports/11tn008.cfmTo view the first installment of a virtual tutorial that Grace presented on Architecture and Design of Service-Oriented Systems, please visit http://www.sei.cmu.edu/library/abstracts/webinars/Architecture-and-Design-of-Service-Oriented-System.cfm

By Julien DelangeSenior Member of the Technical Staff     Research, Technology, & System Solutions Software and systems architects face many challenges when designing life- and safety-critical systems, such as the altitude and control systems of a satellite, the auto pilot system of a car, or the injection system of a medical infusion pump. Architects in software and systems answer to an expanding group of stakeholders and often must balance the need to design a stable system with time-to-market constraints. Moreover, no matter what programming language architects choose, they cannot design a complete system without an appropriate tool environment that targets user requirements. A promising tool environment is the Architecture Analysis and Design Language (AADL), which is a modeling notation that employs both textual and graphical representations. This post, the second in a series on AADL, provides an overview of existing AADL tools and highlights the experience of researchers and practitioners who are developing and applying AADL tools to production projects. Improving Software Development with Tools According to the IEEE Software article, The Impact of Tools on Software Productivity, "Tools have long been recognized as an effective way to improve software development variables such as productivity and product quality." System engineers use different notations and tools that are hard to integrate due to different representation methods and semantics gaps between notations. For that reason, tools must interoperate to help engineers bridge the gap between different specifications.  A number of AADL tools have been developed by organizations, such as Rockwell Collins, the European Space Agency, Honeywell, and Ellidiss Software. These tools provide modeling concepts that describe the runtime architecture of application systems in terms of concurrent tasks, their interactions, and their mapping onto an execution platform.  To support AADL for the design of life-/safety-critical systems, SEI researchers—in conjunction with members of the AADL ecosystem—have developed various tools that help define specifications with modeling capabilities, implementations with automatic code generation, and validation and/or verification with analysis plug-ins that check architecture issues from corresponding models. Examples of AADL Tools Popular AADL tools include the Open-Source AADL Toolset Environment (OSATE), which supports AADL within the Eclipse environment, and Ocarina, which is a command-line tool for expert AADL users.OSATE. The SEI developed OSATE to support the AADL language within Eclipse, which is popular in the software development community and offers built-in support for many languages including Java, C, and C++. Adding support for AADL thus also facilitates connections with existing tools and avoids dedicated training on Eclipse. OSATE supports the core AADL language, as well as several annex languages (such as the error or behavior annex language for designing specific aspects of the system) and several plug-ins for validating system requirements (such as latency or dependability).OSATE is released under the open-source Eclipse Public License (EPL) and available for most platforms (including Windows, Mac OS X, and Linux). Two versions of OSATE are available: The stable version is updated every two to three months after each AADL committee meeting. This version has a set of supported functionalities that are considered "stable," but it does not include all the latest functionality. The testing version is evolving constantly and rebuilt every night with the latest patches and bug fixes. This version is considered "unstable," however, because it is continually under development. The OSATE source code is available on github, which is a social platform for code-sharing that allows users to contribute, potentially learn, and reuse OSATE-based functions for their own projects. As with most active, open-source projects, a dedicated bugtracker lists existing bugs and planned improvements. OSATE extensions allow users to develop their own AADL tool on top of OSATE without having to re-design everything from scratch. Many users created OSATE extensions for research projects, such as the System Architecture Virtual Integration (SAVI) program, which was undertaken by the Aerospace Vehicle Systems Institute, or the Refinement of AADL Models for Synthesis of Embedded Systems (RAMSES), a code generation framework that targets avionics and automotive systems. The most widely-used plug-ins for OSATE include The Requirements Definition and Analysis Language Tool Environment (RDALTE), which connects AADL models with requirements documents. The Instance Model Viewer (IMV), which provides a graphical visualization of AADL models. RC Meta, an Eclipse-based toolset built by Rockwell Collins, which assists with the design and validation of cyber-physical systems. RC Meta includes an AADL /SysML translator and helps engineers track the consistency of models when using both notations. The LUTE constraint language also developed by Rockwell Collins, which queries and validates a model by associating constraints (such as properties to be declared, required composition, etc.) with model components. OCARINA. Jerome Hugues from the Institut Superieur de l’Aeronautique et de l’Espace (ISAE) developed Ocarina to provide the following capabilities for command-line driven use of AADL: Model parsing and analysis, which allows users to check that AADL models are syntactically and semantically correct. Model validation and verification, which allows users to check model properties and consistency. For example, Ocarina can check whether a model contains a given amount of memory component or that the processor associated with some software components has enough processing capacity to process data on time. Code generation, which targets embedded platforms, including the generation of code that implements the system described by the model. The code-generation features of Ocarina target embedded platforms and operating systems, such as Real Time Operating Systems (RTEMS), which are common in the aerospace industry; ARINC653-compliant systems, which are common in avionics; and other mainstream platforms, such as Linux and its real-time variants. Ocarina is now part of the TASTE framework, which is supported by the European Space Agency to automate the design and implementation of their life-/safety-critical projects.  For example, the European IST-ASSERT project enabled the automated production of system implementation from models—a major achievement. In particular, it generates and manages the configuration code of embedded systems, which is challenging because life-/safety-critical systems have specific configuration and deployment requirements. For example, they often need to set the polling period for obtaining sensor data and the need to ensure a task’s completion at a specific pace. Leveraging the AADL Ecosystem to Extend Its Power and Reach One of the strengths of AADL is the expertise and passion of technologists in its ecosystem. To glean insights from some of them, we interviewed the following AADL researchers, developers, and users at the January 2013 AADL Committee Meeting: Steven Miller and Darren Cofer work at the Rockwell Collins Advanced Technology Center, which develops innovative solutions for both its commercial and military business units.  They work in the Trusted Systems group where they develop and apply advanced analysis tools, including formal methods that can be used by the center’s business units to build and verify software components in high-integrity systems, such as avionics flight and mission computers.  Jerome Hugues is an associate professor at ISAE, the reference engineering school in Aeronautics and Space Engineering. He is also co-head of the Advanced Master on Embedded Systems (MS EMS) program and the last-year computer science curriculum. Hugues specializes in the teaching of software architecture. His research is focused on embedded systems architecture with a strong focus on the AADL, as evidenced by his work on the Ocarina command-line-driven AADL tool described earlier. He has been part of the AADL standardization committee since 2006, and a member of its steering committee since 2011. Miller and Cofer have been applying AADL and OSATE for modeling distributed real-time and embedded systems that are of interest to their company. As they described it We began serious investigation of AADL as part of our DARPA-funded META project in 2010.  We used AADL to model portions of an avionics system architecture.  Our objective was to use AADL to support compositional reasoning about the system-level behavior of an architectural model in which the behaviors of individual components are specified by means of assume-guarantee contracts. Under the META project, Rockwell Collins implemented a new constraint language, Lute, on top of OSATE to verify the model characteristics according to design requirements and/or contracts.  For example, Lute can be used to check that a collection of tasks, such as those computing statistics from a sensor, are all executing at the required rate. Engineers use these tools to check integration consistency. Hugues began investigating the use of AADL in 2006 while working on the IST ASSERT Project, which was led by the European Space Agency to study the design of on-board software that is integrated in launchers or satellites. In our discussion, Hugues described his development of Ocarina: Relying on complex, object-oriented frameworks was a blocking factor, as those cannot be easily qualified. Switching to AADL was appealing: I could separate the modeling of the configuration of my system from the actual logic, and have a tool that generates automatically for me the application-specific middleware, instead of using object-orientation. This was one of the first contributions I made to AADL through the Ocarina toolset. The AADL ecosystem integrates well with and offers many connections to other tools used for designing life-/safety-critical systems, as Hugues reported: During my last projects, and as an academic, I tested (or reviewed) most AADL tools: OSATE, STOOD, but also gateways to other analysis tools (formal methods such as timed automata, petri nets; scheduling analysis; model transformation). It is fascinating to see the diversity of analysis supported from one single description language. I also contribute to the implementation of some of them through the Ocarina toolset. In that context, AADL is a good candidate to connect different notations. Likewise, Miller and Cofer highlight how AADL’s well-defined semantics and description of both hardware and software concerns help bridge the gap between different notations and avoid usual issues such as semantics inconsistency and notational differences: Some of our engineers are using SysML to model system architectures and requirements and we expect SysML to be the dominant modeling language in our product divisions.  However, the SysML language does not provide sufficiently formal semantics to support the kinds of analysis that we want to do.  We have tried to resolve this problem by developing translators from SysML to/from AADL. Having used AADL as an underlying language to connect different representations, Miller and Cofer reported promising feedback: There are a number of tools and efforts aimed at system-level modeling of complex embedded systems.  We would like to see AADL used as a common underlying language for these systems that could be used for both synthesis and analysis. This feedback is also shared by Hugues, who described using AADL with other languages on his projects: In parallel, I investigated the use of other formalisms (such as Simulink, SCADE or SysML) in conjunction with the AADL. My main motivation to keep using AADL for my future projects and research is that AADL acts truly as a "backbone" for your system: it nicely federates all the blocks of your systems, but also lets you use external models when needed. Such adaptability makes AADL appealing: I can focus on the architecture using one formalism, and bind it to another one for requirements engineering, control/command, etc. The Future of AADL Tools SEI researchers have been developing new tools and functions in OSATE for many years. We are now developing features to support the specification of faults in the system architecture and provide analysis tools so that users may document their impact. For example, in the medical domain, a fault with a pump infusion system on the patient device should not propagate to the injection subsystem nor alter the quantity of drugs being injected. On the other hand, new functions motivate new needs, as Miller and Cofer highlighted: The lack of a stable graphical environment has been a limitation both in communication (demonstrating results to others) and in quickly/accurately modeling new systems.  It is often difficult to figure out how to use some of the more advanced features of AADL when we need them. The correct syntax is difficult to deduce from the reference manual and the examples available often do not illustrate what we want to do. To address some of these needs, the Instance Model Viewer (IMV), a graphical viewer, has been added to OSATE. The graphical edition is more convenient than the textual version and will make OSATE more accessible to the user community. Convenience is also one of the reasons that engineers use other modeling frameworks that support a user-friendly graphical notation, even if they do not support analysis or validation capabilities. As Hugues reported: The main challenge I’ve seen is human-related: convincing people to invest on a new language instead of relying on the (presumably) known UML derivatives. When people caught on to AADL, they understood how much they can benefit from it. The existing IMV modeling framework motivates the need for new research projects and opens new possibilities. Users want to experiment with new features and additions in the latest version of AADL, including the new components and language annexes (such as ARINC653 or the Error-Model). Miller and Cofer described a similar experience with respect to the DARPA META program: We are continuing the work on compositional verification that we started in META in DARPA’s HACMS project, as well as in projects funded by AFRL and NASA.  In the HACMS project, we are using AADL to model a UAV system and developing new tools to verify that it is robust against cyber attacks. This blog post presents an overview of existing AADL tools, highlighting two key tools developed and used by the community. Many other tools exist, however, that are either open-source or commercial. Users who are interested in learning more about these tools can visit the AADL community wiki, which provides an extensive list of the tools that support AADL. The next post in this series will highlight the use of AADL in the medical domain, including reports from researchers that use AADL for specifying and validating medical devices/domains. Additional Resources OSATE https://wiki.sei.cmu.edu/aadl/index.php/Osate_2 OSATE2 download: https://wiki.sei.cmu.edu/aadl/index.php/OSATE_2_download_page OSATE2 source code: https://github.com/osate/ IST-ASSERT: http://www.assert-project.net/ META project: http://cps-vo.org/node/2675 AADL latency analysis http://www.sei.cmu.edu/library/abstracts/reports/07tn010.cfm AADL dependability http://www.sei.cmu.edu/library/abstracts/reports/07tn043.cfm AADL wiki: https://wiki.sei.cmu.edu/aadl/index.php/Main_Page

By Will DormannSenior Member of the Technical StaffCERT Occasionally this blog will highlight different posts from the SEI blogosphere. Today’s post by Will Dormann, a senior member of the technical staff in the SEI’s CERT Program, is from the CERT/CC (Coordination Center) blog. This post explores Dormann’s investigation into the state of signed Java applet security.

By Joseph ElmSenior Member of the Technical Staff Building a complex weapon system in today’s environment may involve many subsystems—propulsion, hydraulics, power, controls, radar, structures, navigation, computers, and communications.  Design of these systems requires the expertise of engineers in particular disciplines, including mechanical engineering, electrical engineering, software engineering, metallurgical engineering, and many others. But some activities of system development are interdisciplinary, including requirements development, trade studies, and architecture design, to name a few.  These tasks do not fit neatly into the traditional engineering disciplines, and require the attention of engineering staff with broader skills and backgrounds.  This need for breadth and experience is often met by systems engineers. Unfortunately, system engineering is often not valued among all stakeholders in the Department of Defense (DoD), and is often the first group of activities to be eliminated when a program is faced with budget constraints.  This blog post highlights recent research aimed at demonstrating the value of systems engineering to program managers in the DoD and elsewhere. In 2004, the Director for Systems Engineering in the Office of the Undersecretary for Defense for Acquisition, Technology and Logistics (OUSD(AT&L)) came to the National Defense Industrial Association (NDIA) and voiced concerns that DoD acquisition programs were not capitalizing on the value of systems engineering (SE). He knew the value of SE and knew that it could help DoD programs, but he also knew that not all DoD program managers shared his convictions. Consequently program managers were taking shortcuts and eliminating SE capabilities from their programs.  He came to NDIA seeking quantitative evidence of the value of SE. Subsequently, others have recognized this same problem.  A recent Government Accountability Office (GAO) report indicates that acquisition program costs are typically 26 percent over budget and development costs are typically 40 percent more than initial estimates. These programs routinely fail to deliver the capabilities when promised, experiencing, on average, a 21 month delay.  The report finds that "optimistic assumptions about system requirements, technology, and design maturity play a large part in these failures, and that these optimistic assumptions are largely the result of a lack of disciplined SE analysis early in the program." Despite findings such as this, many programs still fail to deploy good SE.  Why?  It may be because there is relatively little quantitative evidence of the impact and value of SE.  Everyone can see the SE costs, such as the labor applied and the time allocated in the schedule.  The benefits of SE, however, may be less identifiable.  They often manifest themselves as risks that didn’t materialize rework that didn’t need to be done customer complaints that didn’t occur, and product deficiencies that are circumvented Because these benefits are hard to quantify, however, the return from investment in SE is often unrecognized.  To get a true picture of the value of SE, we need to quantitatively measure its impact on acquisition program performance. The remainder of this blog posting describes a research effort that the SEI undertook in partnership with the NDIA and the IEEE Aerospace and Electronic Systems Society (IEEE AESS).  This effort provided quantitative evidence of the value of SE in terms of its impact on program cost, program schedule, and program technical performance - impacts that are crucially important to program managers and executives. Building on Previous Research in Systems Engineering While "software engineering" is etched in stone at the SEI’s Pittsburgh headquarters, it is sometimes hard to draw a clear line between software and the systems supported by software.  For that reason, SEI staff have often conducted research in the SE realm. For example, the Capability Maturity Model Integration Framework addresses processes that apply equally to software development and system development. Architecture development and evaluation methods developed for software are routinely adapted and applied to systems. Through the SEI’s affiliation with NDIA, my fellow researcher, Dennis Goldenson and I became involved in developing their response to the 2004 inquiry from OUSD (AT&L) mentioned earlier.  In 2005, I suggested conducting a survey of acquisition programs to gather information about their activities related to SE, and how those programs performed.  We could then identify relationships between these factors.  We conducted the survey in 2006 and published our results in 2007. Our initial research demonstrated that those programs that deployed more systems engineering performed better against measures of cost, schedule, and technical performance.  In 2010, the DoD approached NDIA and the SEI about strengthening the business case for SE by expanding the survey population to include not just the NDIA but other professional organizations including the IEEE-AESS, and the International Council on Systems Engineering (INCOSE). For this latest study, we surveyed individual programs in participating organizations to obtain answers to the following questions: What systems engineering activities do you perform on your program?  How well does your program perform? We surveyed 148 different programs.  Although most programs were supplying systems for the U.S. defense sector, we also received a few responses from organizations serving other market sectors and operating in different countries. An Even Stronger Link Between SE and Performance Our latest results, published in the SEI technical report, The Business Case for Systems Engineering Study:  Results of the Systems Engineering Effectiveness Study, identified strong links between the performance of systems engineering tasks and overall program performance. These results provide a convincing case for the value of systems engineering. This latest study collected information from participating programs along three dimensions: systems engineering deployment. We assessed SE deployment by examining both the presence and the quality of work products resulting from SE activities.  These work products were selected from those listed in the CMMI Framework by a panel of SE experts.  Based on this assessment, SE deployment for each program was categorized as either low, medium, or high. program performance. We assessed program performance as a combination of cost performance (satisfaction of budget), schedule performance, and technical performance (satisfaction of requirements).  Again, based on this assessment, program performance for each program was categorized as either low, medium, or high. program challenge.  Some programs are inherently more challenging than others due to factors such as size, duration, technology maturity, interoperability requirements, etc.  Based on the combination of these factors, program challenge was categorized as either low, medium, or high. We then looked for relationships between these metrics. We found a very strong relationship between SE deployment and program performance.  In particular, as programs deployed more SE, they delivered better performance.  For example, among those programs deploying the least SE, only 15 percent delivered the highest level of program performance.  Among those deploying the most SE, however, 56 percent delivered the highest level of program performance. As one would expect, our research showed an inverse relationship between program challenge and program performance.  But, we also learned that SE practices became even more valuable when used with these challenging programs. We already noted that the number of programs delivering high program performance increased from 15 percent to 56 percent as SE deployment increased.  For the most challenging programs, however, the number of programs delivering high program performance increased from 8 percent to 62 percent with increased SE deployment.  This result clearly shows the increasing need for SE as programs become more challenging. As mentioned above, we measured SE deployment by assessing SE-related work products for each program.  Now, we could group these artifacts into process areas such as requirements development and management program planning product architecture trade studies product integration verification validation program monitoring and control risk management configuration management integrated product teams Grouping artifacts into process areas enabled us to probe more deeply into the relationships between SE and program performance, identifying not only the overall benefit of SE but also the benefit of specific SE processes. For each program, we assessed SE deployment in each of the 11 process areas above and analyzed the relationship to program performance.  Here again, we found strong supporting relationships for all SE process areas - increased SE deployment in any of these areas contributed to better program performance. Relationships to program performance, however, were stronger in some than in others. Particularly strong relationships to program performance were found for program planning. The number of programs delivering highest performance increased from 13 percent to 50 percent as SE activities related to program planning increased. requirements development and management. The number of programs delivering highest performance increased from 21 percent to 58 percent as SE activities related to requirements development and management increased. verification. The number of programs delivering highest performance increased from 16 percent to 54 percent as SE activities related to verification increased. product architecture. The number of programs delivering highest performance increased from 16 percent to 49 percent as SE activities related to product architecture increased. As strong as these relationships were, we found that they grew even stronger for the more challenging programs. Transitioning to the Public The results of our research can be used in a number of ways by system developers, system acquirers, and academia. For example, our findings constitute a baseline of SE deployment across the industries surveyed.  System developers can use our methods to assess their own SE capabilities, compare them to this baseline, and identify their strengths and weaknesses.  They can then develop process improvement plans to improve their weaknesses and strategies to leverage their strengths.  We continue to work with defense contractors who are applying this process to improve their SE capabilities. System acquirers can also benefit from these findings.  A Program Management Office (PMO) acquiring a system needs to deploy good SE practices in planning the program, defining system requirements, developing system architectures, etc.  The PMO also needs to ensure that the system supplier deploys good SE practices.  For example, the PMO must first include in the solicitation a definition of SE activities that they expect from the supplier.  They should evaluate the supplier’s response to these expectations as a factor in the selection of the supplier.  They should also ensure that the SE expectations are included in the contract.  They should monitor the supplier’s performance throughout execution to ensure that SE expectations are being met. The academic community can also use the results from our study.  For example, several universities offering systems engineering programs at the master’s level are using this information in their curriculum and their courses to show their students the value of systems engineering and to direct some of their courses to capitalize on the knowledge that we’ve gathered here.  INCOSE is also incorporating the results of the survey into its systems engineering handbook. Additional Resources The technical report The Business Case for Systems Engineering may be downloaded at http://www.sei.cmu.edu/library/abstracts/reports/12sr009.cfm Organizations interested in the continuing effort of showing the value of systems engineering may join INCOSE’s System Engineering Effectiveness Working Group athttp://www.incose.org/practice/techactivities/wg/seewg/ Organizations interested in joining NDIA’s Software Engineering Effectiveness Committee athttp://www.ndia.org/Divisions/Divisions/SystemsEngineering/Pages/Systems_Engineering_Effectiveness_Committee.aspx

By William Anderson Senior ResearcherSoftware Solutions Division The ubiquity of mobile devices provides new opportunities to warn people of emergencies and imminent threats using location-aware technologies. The Wireless Emergency Alerts (WEA) system, formerly known as the Commercial Mobile Alert Service (CMAS), is the newest addition to the Federal Emergency Management Agency (FEMA) Integrated Public Alert and Warning System (IPAWS), which allows authorities to broadcast emergency alerts to cell phone customers with WEA-enabled devices in an area affected by a disaster or a major emergency. This blog posting describes how the Software Engineering Institute's (SEI) work on architecture, integration, network security, and project management is assisting in implementing the WEA system, so it can handle a large number of alert originators and provide an effective nationwide wireless emergency warning system. Why We Need the WEA System In the summer of 2012, the western United States experienced unprecedented wildfires costing several lives and hundreds of millions of dollars of property damage. The Lower North Fork fire in Colorado caused three fatalities and could have been worse due to massive failures in the landline-based telephone alerting systems. A local television station reported that incorrect records in a geographic database were responsible for many of the failures. The Denver Post report on the 2012 Waldo Canyon fire near Colorado Springs (the most destructive fire in Colorado history) revealed similar problems in the emergency alert system. An additional complication to the alert process lies in the fact that more than 35 percent of U.S. homes do not have a land-based phone line. Given the mobile nature of today’s society, a better solution is needed to warn people of imminent threats using geographic data. Pursuant to the Warning, Alert, and Response Network (WARN) Act, FEMA's IPAWS, and the Department of Homeland Security’s Science & Technology (DHS S&T) Directorate have partnered with the Federal Communications Commission and commercial mobile service providers to develop the WEA system to bring emergency alert messages to mobile devices configured to allow wireless emergency alerts. This effort is the latest in a series of alert and warning systems that date back to 1951 when President Harry Truman established the Control of Electromagnetic Radiation (CONELRAD) program. The WEA system will enable alert originators—including federal, state, tribal, and territorial governments as well as local emergency response authorities that provide fire protection, transit, and public utilities—with the potential to reach the more than 325 million wireless subscribers. The four largest cellular carriers—AT&T, Sprint Nextel, T-Mobile, and Verizon, which connect with 90 percent of all cell phones in the United States—have agreed to participate along with another 72 smaller carriers (see FCC Master CMAS (prior designation for WEA) Registry for reported numbers). The SEI has two major tasks in this effort: developing an integration strategy to help assimilate alert originators into the system and identifying an initial set of best practices that apply to WEA use. The SEI is just one of several organizations involved in the WEA system project. The John Hopkins Applied Physics Laboratory is constructing a large-scale computer model of the system on which to run simulation studies and an environment to test the interfaces and system. The RAND Corporation developed a mobile-penetration strategy to determine the availability and acceptability of mobile alerting services across the country. The MITRE Corporation is working on test plan development. The WEA system went online nationally in April 2012. This activation meant that the backbone infrastructure is up and running, providing alerting authorities with the appropriate alert originating software to send messages to WEA system-enabled handsets. The hard task of expanding the number of participating originators and mobile carriers can now commence.   When an emergency responder or alerting agency, such as the National Weather Service (which went online June 28, 2012), or the National Center for Missing and Exploited Children (NCMEC) (which issued its first AMBER alert through WEA on December 18, 2012) becomes aware of an event that the public should know about, it can use the WEA system to initiate a geographically targeted alert message and identify smart phone users in the geographic area who should receive the message. The WEA system portion of the Common Alerting Protocol (CAP) specifies a text message of up to 90 characters detailing the topic of the alert, the area affected, the alert expiration, the prescribed action, and the alert sender. Most modern phones (e.g., 3G, 4G, or LTE) are already equipped with the technology to receive WEA messages. Cell phone users are automatically subscribed to the service; they do not have to opt in to receive alerts. They may opt out of all but the presidential alerts. The WEA system sends out three types of messages presidential alerts AMBER (America’s Missing: Broadcast Emergency Response) alerts imminent threats to safety or life The WEA system also uses Federal Information Processing Standard (FIPS) codes, which are alphanumeric codes to target cell phones in geographic areas at the county level. The WEA system operates using cell broadcast, a one-to-many technology rather than point-to-point (phone number addressed) text messages. The WEA system messages are transmitted on a separate channel from the voice and data communication links. As a result, authorities sending out emergency alerts can do so even when voice and data channels are saturated. Leveraging Our Architecture Analysis Expertise Along with Lawrence Jones, project lead for the SEI’s work on the WEA effort, and our colleagues in the Software Solutions and CERT divisions at the SEI, we have been leveraging our architecture and system-of-systems analysis techniques to help ensure the smooth national deployment and operation of the WEA system. Specifically, we are using the Mission Thread Workshop (MTW) approach that has been used by the Department of Defense in large systems-of-systems analysis to describe the step-by-step sequences required to execute a given mission. We are applying the MTW approach to the WEA system to develop mission thread descriptions of typical integration challenges. To analyze the WEA system as a socio-technical system-of-systems, we developed mission threads to describe operational scenarios that the emergency response community might encounter that would benefit from the WEA system alerting capabilities. Table top walk-throughs of the mission threads with emergency management experts reveal underlying quality attributes (QAs) that the system must have to perform properly. These underlying QAs can include normal system QAs, such as resilience, reliability, modifiability, securability, and sustainability, as well as QAs that are particularly relevant to the WEA system, such as accuracy, urgency, timeliness, and relevance. Addressing Security Concerns A key quality attribute of interest for the WEA system is security. Like other cyber-enabled services, the WEA system is subject to cyber threats that may prevent its use or damage the credibility of the service it provides. Attackers may attempt to delay, destroy, or modify alerts, or even insert false alerts, actions that may pose a significant risk to the public. Accordingly, a team of experts from the SEI's CERT division is examining the security aspects of the WEA system using the MTW approach. By reviewing the step-by-step sequence documented in mission threads, security experts are identifying threats and potential vulnerabilities that may make the WEA system susceptible to cyber-attacks. To address these concerns, the SEI is developing a cybersecurity risk management strategy for the WEA system that alert originators can use to identify, prioritize, develop, and implement cybersecurity risk mitigation plans.   Best Practices Work Although WEA adoption is still in the early stages, the SEI is leading a best practices task that supports the DHS effort. This work includes a lessons learned report on the New York City WEA demonstration conducted in December 2011 creation and analysis of a WEA trust model that incorporates inputs from the emergency management organization community and a sample of the public. This study identifies key factors involved in trust in WEA messages to increase understanding of the effectiveness of these alerts selected example best practices Addressing Challenges of Scale A key challenge to deploying and operating the WEA system effectively is dealing with issues of scale. It’s estimated there are between 10,000 and 40,000 alert originators who may initiate the WEA system messages including designated federal, state, territorial, tribal, and local authorities (the WEA system alert authorization is a state-driven organizing model that is expected to mimic the hierarchical structure of emergency and disaster response networks). The SEI is developing integration strategies to help those originators to bring the WEA system capability to their constituents. Many of these originators will rely on vendors to provide solutions. The vendors build the systems that help emergency response operators prepare alerts for dissemination. The human-machine interfaces vary but generally support common features, such as a text-based interface, checklist prompts, predefined templates for message support, and map-based interfaces for geographic targeting assistance. Vendors also ensure that the operators have the needed message control fields, such as What is the severity? What is the urgency? When should the message be taken down? Originators face many scale-related challenges as they attempt to implement the WEA system: Many alert originators employ different types of software and have acquired that software from multiple vendors or, even, custom built it themselves. The state of readiness to integrate with the WEA system varies from originator to originator. The use of protocols varies from one originator to another. The WEA system standard is the Common Alerting Protocol which, among other things, has a 90-character message length limit (for the WEA system), which is even shorter than a Tweet! Originators feel that this limit impedes their ability to aptly describe a situation to users, especially for AMBER alerts. The WEA system is often integrated or supplemented with other existing emergency alert system channels, including television, radio, and social media. Related DHS Science and Technology Directorate research projects involve improving the granularity of the geo-location capability (some carriers are already committed to a finer granularity than is required) and addressing the challenges associated with citizen reaction/non-reaction to the alerts. In many cases alerting authorities want to narrow their message to a geographic area smaller than the county level. They are also concerned that those who receive messages react promptly and correctly. What’s Ahead The SEI team working on the WEA system project is consolidating the results of the data collection phase of our project interviewing originators and the vendor community that supports the originators. Questions include What kind of systems do the originators use? How do the originators adopt new systems? Do they have system documentation? What problems have they encountered using the systems? The results of our work will be delivered to DHS this summer with public release anticipated sometime in the fall. We would welcome your feedback on our work. Please share your insights in the comments section below. Additional Resources For more information about the Mission Thread Workshop, please visithttp://www.sei.cmu.edu/architecture/tools/establish/missionthread.cfm For more information about the the Wireless Emergency Alerts (WEA) Research, Development, Testing, and Evaluation (RDT&E) program, formerly known as the Commercial Mobile Alert Service (CMAS) RDT&E program, please visit  http://www.fema.gov/wireless-emergency-alerts

By Peter Feiler Senior Member of the Technical StaffSoftware Solutions Division Aircraft and other safety-critical systems increasingly rely on software to provide their functionality. The exponential growth of software in safety-critical systems has pushed the cost for building aircraft to the limit of affordability. Given this increase, the current practice of build-then-test is no longer feasible. This blog posting describes recent work at the SEI to improve the quality of software-reliant systems through an approach known as the Reliability Validation and Improvement Framework that will lead to early defect discovery and incremental end-to-end validation. A Fresh Look at Engineering Reliable Systems Studies by the National Institute of Standards and Technology and the National Aeronautics and Space Administration show that 70 percent of software defects are introduced during the requirements and architecture design phases. Moreover, 80 percent of those defects are not discovered until system integration test or later in the development cycle. In their paper "Software Reduction Top 10 List" researchers Barry Boehm and Victor Basili wrote that "finding and fixing a software problem is 100 times more expensive than finding and fixing it during the requirements and design phase." Aircraft industry and International Council on Systems Engineering data shows a rework cost multiplier of 300 to 1,000. Much of this defect leakage and the resulting cost penalty are due to incomplete and ambiguous requirements and mismatched assumptions in the interaction between the components of embedded software system architectures.  To address these problems, I have developed—together with fellow SEI researchers John Goodenough, Arie Gurfinkel, Charles Weinstock, and Lutz Wrage—the Reliability Validation and Improvement Framework, which takes a fresh look at engineering reliable systems. Reliability engineering has its roots in engineering physical systems. The focus of reliability engineering has been on using historical data to predict mechanical failures in physical parts, assuming that design errors have little impact due to slowly evolving designs. Defects in software systems, however, are design errors for which reliability predictions based on historical data have been a challenge. At the same time, embedded software has become the key to system integration and thus the driver for system reliability. We needed a new perspective on reliability because it is unrealistic to assume that software has zero defects. Our work on developing a framework began when we were approached by the U.S. Army Aviation and Mission Research Development and Engineering Center (AMRDEC) Aviation Engineering Directorate (AED), the agency responsible for signing off on the flight worthiness of Army rotorcraft. AMRDEC wanted a more reliable approach than "testing until time and budgets are exhausted" to qualify increasingly software-reliant, safety-critical systems. Four Pillars for Improving the Quality of Safety-Critical Software-Reliant Systems We needed a new approach for making software-reliant systems more reliable, one that allows us to discover problems as early as possible in the development life cycle. For safety-critical systems these are not only defects in functional design but also problems meeting operational quality attributes, such as performance, timing, safety, reliability, and security. Our approach needed to identify not only defects before a system is built, but also issues that are hard to test for. In addition, the approach needed to ensure that unavoidable failures are addressed through fault management and that unhandled faults are identified, providing resilience to counter unplanned usage and unexpected conditions. The Reliability Validation and Improvement Framework that we developed incorporates four engineering technologies to address the challenges outlined above: formalization of mission and safety-critical requirements at the system and software level. Requirements on a system—the first pillar of our framework—are typically determined by business needs and operational use scenarios. They are often developed by system engineers and may evolve over time.  As Boehm points out in his 2006 paper, "Some Future Trends and Implications for Systems and Software Engineering Processes," there is a gap in translating, especially non-functional system requirements, into requirements for embedded software. Individual software units are then developed without this contextual knowledge, making assumptions about the physical system, computer hardware platform, and the interaction with other software tasks, each of which can affect functional and non-functional system properties. Our framework focuses on capturing the "shalls" of a system by specifying mission capabilities under normal conditions (mission requirements such as functionality, behavior, and performance), as well as the "shall nots," by specifying how the system is expected to perform when things go wrong (dependability requirements, such as safety, reliability, and security). Requirements are associated with the system in the context of assumptions about the operational environment and decomposed into requirements of different components of the system. This approach allows the validation and verification of requirements and assumptions in the correct context. We have identified an excellent resource, the Requirements Engineering Management Handbook developed by a formal methods group at Rockwell Collins for the Federal Aviation Administration (FAA). The handbook defines an 11-step process for capturing requirements in a more systematic and formal way using an explicit architecture model as context.  This approach allows for completeness and consistency checking of requirement specifications.  architecture-centric, model-based engineering. The aircraft industry has recognized that software-reliant system development must take an architecture-centric, model-based, analytical approach to address the limitations of conventional build-then-test practices. The industry has embraced virtual system integration to achieve validation through static analysis of integrated architecture and detailed design models. This approach leads to early discovery of software-induced problems, such as timing related loss of data, control instability due to workload sensitive latency jitter, and loss of physical redundancy due to load balancing of partitions in a networked systems. This second pillar in our framework uses analyzable architecture models combined with detailed design models and implementations to evolve and validate a system incrementally. The OMG SysML architecture modeling notation is gaining popularity in the system engineering community. For embedded software systems, SysML is complemented by SAE International’s  Architectural Analysis and Design Language (AADL), a notation I authored that provides a set of concepts with well-defined semantics for representing embedded software system architectures, which include the software task and communication architecture, its deployment on a networked computer platform architecture, and its interaction with the physical system. These semantics lead to precise specification of execution and interaction behavior and timing. A standardized error model extension to AADL supports identification of safety hazards, fault impact analysis, and specification of fault management strategies to help meet reliability and availability requirements. In this architecture-centric virtual integration approach, the annotated architecture model drives analysis of different functional and non-functional properties by generating analytical models and thereby consistently propagating changes into each analysis, as well as to implementations generated from validated models. The virtual integration approach uses a multi-notation model repository that utilizes standardized model interchange formats and maintains consistency across models, while allowing suppliers and system integrators to utilize their own tool chains. static analysis of functional and non-functional system properties. Static analysis is any technique that formally proves system properties from system models prior to deployment and execution. Static analysis complements simulation and traditional testing to validate and verify a system. This third pillar of our framework focuses on incrementally validating and verifying the virtually integrated system before the actual production software is written. Formal analysis frameworks, such as model checking for verifying functional behavior, and rate monotonic schedulability analysis for assuring timing requirements, are scalable solutions that have been successfully applied to avionics and space systems. Properties defined in the SAE AADL standard and annotations using the standardized Behavior, Error Model, and ARINC653 Partitioned Architecture extensions to AADL support different functional and non-functional analyses from the same model. system and software assurance. Safety cases using a goal-structured notation have been used extensively outside the United States to assure safety in nuclear reactors, railroad signaling systems, avionics systems, and other critical systems.  A best practice of this fourth pillar of our framework involves the development of evidence in parallel with the system design throughout the development life cycle. This evidence ranges from requirements and design review results and predictive analysis results of virtually integrated systems, to test results to provide justified confidence. This approach records claims about the system, assumptions made in the process, and evidence required to satisfy the claims. The SEI has been involved in evolving this approach into assurance cases for different system concerns. Assurance cases provide a systematic way of establishing confidence in the qualification of a system and its software by recording and tracking the evidence and arguments (as well as context and assumptions) that the claims of meeting system requirements are satisfied by the system design and implementation, and making the argument that the evidence is sufficient to provide justified confidence in the qualification. Recent developments indicate that assurance cases are being adopted within the United States. For example, the U.S. Food and Drug Administration is exploring the inclusion of assurance case reports into regulatory submissions for infusion pumps. An International Standards Organization for assurance cases is currently under development. Overview of the Reliability Validation and Improvement Framework In the figure above, the four components combine to transform the traditional software development model of a heavily document-driven practice into an analytical practice based on architecture-centric models. As we write in our SEI technical report, the centerpiece of this framework is an architecture-centric model repository supporting multiple modeling notations and a combination of architecture and detailed design models together with source code implementations. The ability to virtually integrate and analyze the models is key to improving reliability by discovering problems early in the life cycle and reducing defect rework at a higher cost in later phases. Since incomplete, ambiguous, and inconsistent requirements contribute 35 percent of system-level defects, it is valuable to formalize requirements to a level that can be validated and verified by static analysis tools. Formalization of requirements establishes a level of confidence by assuring consistency of the specifications and their decomposition into subsystem requirements. The requirements are decomposed in the context of an architecture specification and refined into concrete, formalized specifications for which evidence can be provided through verification and validation activities. This method is reflected in the Requirements Engineering Management Handbook process and supported by the draft Requirement Definition and Analysis Standard extension that is part of the AADL standard suite and applicable to other architecture notations. The aerospace industry, which already has several well-entrenched safety practices, has embraced System Architecture Virtual Integration (SAVI), which a multi-year initiative conducted under the auspices of the Aerospace Vehicle Systems Institute (AVSI). SAVI aims to improve current practice and overcome the software cost explosion in aircraft, which currently makes up 65 percent to 80 percent of the total system cost with rework accounting for more than half of that. SAVI has chosen SAE AADL as a key technology and performed several proof-of-concept demonstrations of the virtual integration concept to achieve early discovery of problems. SAVI’s current work focuses on demonstrating model-based, end-to-end validation and verification of safety and reliability, as well as defining model repository requirements to facilitate commercial tool vendor adoption.  SAVI also has an ongoing effort to assess return on investment and necessary changes to acquisition and qualification processes. The application of static analysis to requirements, architecture specifications, detailed designs, and implementations leads to an end-to-end validation and verification approach. The research community in the United States and Europe has embraced AADL models as a platform for integrating formal analysis frameworks and transitioning them quickly to industrial settings. An assurance case interchange format is being standardized and evidence tracking in the context of formalized requirements is supported through the Requirement Definition and Analysis extension to AADL. As illustrated in the figure above, our revised software development model is represented by two parallel activities: The first activity reflects the development process: build the system. This development process focuses on the creation of design artifacts ranging from requirement specification and architecture design, detailed design, and code development through integration, target, and deployment build. The second activity reflects the qualification process: build the assurance case for the system. This qualification process comprises the traditional unit test, integration test, system test, and acceptance test phases.  It extends into the early phases of development through the concept of an architecture-centric virtual system integration lab to support validation and verification throughout the life cycle. Conclusion At the SEI we continue to provide technical leadership to the SAE AADL standards committee, participate in the SAVI initiative, codify elements of the four pillars described in this post into various methods (such as the Virtual Upgrade Validation and assurance cases). We also work with organizations such as the U.S. Army, NASA, FDA, and various industrial partners to apply different aspects of these four pillars in actual programs and projects. We welcome your feedback on our work and are looking for organizations interested in applying this approach. If you are interested, please leave a comment below or send an email to info@sei.cmu.edu. Additional Resources To read the full SEI technical report, The Reliability Validation and Improvement Framework, please visit http://www.sei.cmu.edu/library/abstracts/reports/12sr013.cfm To read the white paper describing this work, Four Pillars for Improving the Quality of Safety-Critical Software-Reliant Systems, please visithttp://www.sei.cmu.edu/library/abstracts/whitepapers/FourPillarsSWReliability.cfm

By Soumya Simanta Senior Member of the Technical Staff Software Solutions Division Warfighters in a tactical environment face many constraints on computational resources, such as the computing power, memory, bandwidth, and battery power. They often have to make rapid decisions in hostile environments. Many warfighters can access situational awareness data feeds on their smartphones to make critical decisions. To access these feeds, however, warfighters must contend with an overwhelming amount of information from multiple, fragmented data sources that cannot be easily combined on a small smartphone screen. The same resource constraints apply to emergency responders involved in search-and-rescue missions, who often must coordinate their efforts with multiple responders. This posting describes our efforts to create the Edge Mission-Oriented Tactical App Generator (eMontage), a software prototype that allows warfighters and first responders to rapidly integrate geotagged situational awareness data from multiple remote data sources. An Abundance of Siloed Data While the internet has made data more accessible than ever, the information is often siloed in different systems. To complicate matters, it’s not easy for warfighters to share and tailor data in the field. Typically, data filters are predefined at development time and are often not flexible enough to meet the range of unancticipated requiremnts that warfighters and first responders may have in the field. The modifications required to support newer capabilities is often done offsite and, therefore, takes longer to reach the hands of users.  These delays are problematic for warfighters, who need the ability to configure data sources as quickly as possible. Currently, warfighters need multiple systems to view different types of data including historical information, such as the occupants of a certain address real-time information including geo-location data that reports the location of team members (often referred to as "blue force tracking") alerts including information (such as severe weather alerts) from federal agencies Using a system that we are prototyping here at the SEI, a warfighter would be able to mash up data from these sources, as well as public data sources (such as Google Places, Twitter, or FourSquare) and/or private Department of Defense (DoD) data sources. More Control to the End User Our research at the SEI is focused on giving more control to the end users, in this case warfighters and emergency responders. In particular, we provide them the ability to define data filters at runtime without writing code and mash up data from different sources, so that they can access the information that they need, when they need it, on their device. To enable this capability, I am working with a group of SEI researchers, including Gene Cahill, on the development of eMontage, a Java-based client/server application that allows users to create data filters on Android smartphones. We built eMontage using commercial off-the-shelf components, as well as public and private data sources. eMontage allows users to mash geotagged data from private and public information sources and combine relevant data on one application and screen. The eMontage architecture allows for the addition of both public and private data sources in a relatively short amount of time.  In this context, "short" translates to a few hours to a week, depending on the data format and application programming interface (API) support provided by the remote data source or web service. We are collaborating with Dr. Brad Myers and his student, Kerry Chang of Carnegie Mellon University’s Human Computer Interaction Institute. Dr. Myers has collaborated with us on previous initiatives in this arena.  They are working to help us define an appropriate interface for soldiers to use the handheld software in the challenging situations they face.  We are also collaborating on a parallel effort that deals with an interface for rapid entry of unstructured information into a smartphone. In the implemented prototype, known as Listpad, the user enters unstructured text as he or she would on a notepad, but the information is saved in a structured format. Through contextual clues, Listpad autocompletes data entry and also allows for automatic data-type detection. Testing Our Prototypes and Approach In 2012, we tested our prototypes at Camp Roberts in the United States Special Operation Command Naval Postgraduate School (NPS) Field Experimentation Cooperative Tactical Network Testbed. Our effort was aimed at determining whether warfighters could successfully use our applications determining the learning curve involved soliciting feedback on the user interface gathering additional use cases and features identifying additional data sources and formats Warfighters testing the prototype provided us with positive feedback, as well as a list of suggested features for future prototypes. In February 2013, we demonstrated the capabilities of eMontage at the Camp Roberts’ NPS's Joint Interagency Field Exploration (JIFX). This event explored the potential of emerging capabilities to address challenges faced by warfighters and first responders. Our demonstration pulled geotagged data for Hurricane Sandy from the Wireless Emergency Alert System, Twitter, Flickr, and Foursquare. We created mashups of the data that blended eyewitness information, photographs, and other information about affected areas in realtime during the hurricane. At that event, we received vital feedback from first responders, including the Federal Emergency Management Agency and the Department of Homeland Security. Future Work Our initial aim with eMontage was to develop a capability that would allow warfighters and emergency responders on the tactical edge to easily and quickly integrate tactical information sources on their smartphones.  We are working on improving the performance and security of eMontage by adding a caching mechanism and standard security mechanisms. We are also leveraging the eMontage architecture to design a prototype that will analyze streaming information (such as a Twitter stream or user location stream) at the edge in a project called Edge Analytics. Dr. Myers and Ms. Chang are also working to partially automate the "adding a new data source" to the eMontage workflow. We also plan to explore the integration of these research projects by running the eMontage server on a cloudlet and in conjunction with group context-aware mobile applications to guide the data filtering process. We welcome your feedback on our work. Please leave feedback in the comments section below. Additional Resources For more information about the SEI’s work in pervasive mobile computing, please visithttp://www.sei.cmu.edu/mobilecomputing/ To download a pdf describing other work by SEI researchers in advanced mobile systems, please visithttp://www.sei.cmu.edu/library/assets/brochures/Infosheet_RTSS_AMS_Jan13.pdf To view the presentation, eMontage: An Architecture for Rapid Integration of Situational Awareness Data at the Edge, which I presented with Ed Morris and Gene Cahill at SATURN 2013, please visithttp://www.sei.cmu.edu/library/assets/presentations/simanta-saturn2013.pdf? To view the presentation, Listpad: Creating Customized Structured Data on Mobile Devices,  by Kerry S. Chang, Brad A. Myers, Gene M. Cahill, Soumya Simanta, Edwin Morris, and Grace Lewis, please visithttp://www.cs.cmu.edu/~shihpinc/pdf/Listpad_workshop.pdf

By Kevin FallChief Technology Officer I recently joined the Carnegie Mellon Software Engineering Institute (SEI) as deputy director and chief technology officer (CTO). My goal in this new role is to help the SEI advance computer science, software engineering, cybersecurity, and related disciplines to help ensure that the acquisition, development, and operation of software-dependent systems have lower cost, higher quality, and better security. I have spent the past two decades conducting a range of research and development activities, and I have served on various Department of Defense (DoD) advisory boards. In this blog posting, I’d like to talk a little bit about my background and outline the priorities I’m pursuing at the SEI. In subsequent blog postings, I’ll describe the SEI technical strategy in more detail. My Introduction to Software I became interested in programming and reverse engineering (analyzing copy protection schemes) while in high school in southern California, where I learned to program in 6502 assembly language and BASIC on an Apple ][. This sparked my interest in software, especially operating systems. My first summer job was writing software, primarily a code de-obfuscation system, for a small training company in Manhattan Beach, California. During my undergraduate studies in computer science at the University of California, Berkeley (UC Berkeley), I spent two summers as an intern at TRW (now Northrop Grumman) in Redondo Beach where I was introduced to Berkeley UNIX (the Berkeley Software Distribution or BSD) and the C programming language. Ultimately, I worked on BSD UNIX itself, where I developed TCP/IP software targeted for deployment on the Cray X/MP and integrated Massachusetts Institute of Technology’s (MIT) Kerberos Authentication system (now the basis for authentication in Windows environments) into the BSD authentication framework. I also authored the BSD version of the cat program, perhaps the most widely used basic program in the system. While I was working at Berkeley integrating Kerberos into BSD, Robert Morris’ worm hit the Internet. Using some of the reverse-engineering skills I had acquired earlier, I helped in de-obfuscating the binary code comprising the worm. Coincidentally, this incident would be the primary motivator for the Defense Advanced Research Projects Agency (DARPA) to sponsor the creation of the SEI’s CERT Division. During the summer, after I left UC Berkeley, I joined MIT’s Project Athena doing further work on Kerberos prior to joining the PhD program in computer science and engineering at UC San Diego (UCSD). My research at UCSD focused on operating system support for multimedia. While in graduate school at UCSD, I consulted for the San Diego Supercomputer Center (then run by General Atomics) and for the Center for Communications Research (one of the Institute for Defense Analysis (IDA's) federally funded research and development centers (FFRDCs). Early Professional Career After graduating with a PhD from UCSD, I was hired to work with the Network Research Group at Lawrence Berkeley National Laboratory, which produced the tcpdump and pcap tools, numerous other TCP/IP improvements, and the NS2 network simulator, for which I served as chief architect. In this role, I added a novel emulation capability, enabling the intermixing of real-world network traffic with simulated traffic. In addition, I worked with Sally Floyd to produce a number of detailed analyses of TCP’s congestion control algorithms. My involvement with internet technologies in the late 1990’s afforded me the opportunity to work in a startup company, NetBoost, that built hardware and software for arbitrary manipulation of network traffic. The primary use of NetBoost’s technology was for building a hardware-accelerated intrusion detection system. NetBoost was acquired by Intel Corporation in 1999. After this acquisition, I worked in a product group at Intel in the area of network processors (CPUs designed to manipulate networks traffic), then moved on to a research division. Intel had established a set of independent labs ("lablets") at the University of Washington, UC Berkeley, Carnegie Mellon University (CMU), and the University of Cambridge (England). I joined the UC Berkeley lab in 2001 and worked on various aspects of networking, including communications in especially challenging environments, for which I became an IEEE Fellow in 2009. In 2011, Intel changed its university-engagement-strategy, and these labs were re-structured as smaller "Intel Science and Technology Centers" (ISTCs), which are still present at various locations, including CMU. I left Intel shortly thereafter, completing the textbook TCP/IP Illustrated Volume 1 (second edition), which was published in November 2011. I joined Qualcomm in 2011, where I worked mostly on adaptive video streaming until the end of 2012. My Service On DoD Advisory Boards While I was working at Intel and Qualcomm, I also spent time on a number of DoD advisory boards. I was a member of the Defense Science Study Group (DSSG), a DARPA-sponsored program that connects academics and academically-leaning industry members with the DoD and other government agencies. I later served as a consultant for the Defense Science Board, where I represented the potential effects of cyber attack and defense for the purpose of understanding its capabilities relative to other types of attacks and defenses, including nuclear, directed energy, chemical, conventional explosives. In 2008, I began my four-year term as a member of the Air Force Scientific Advisory Board (AF SAB), and, consequently, a special government employee. While there, I met Professor Doug Schmidt, then deputy director and CTO of the SEI. We worked together on a recently-released study that assessed the technologies and policies associated with assuring cyber situational awareness for the U.S. Air Force. Doug and my immediate predecessor, Professor Bill Scherlis, have provided me with a great deal of advice and context about the research and transition activities at the SEI, for which I am immensely grateful. Progress Thus Far at the SEI Since joining the SEI in January of this year, I have been busy with a range of projects, including reviews of proposals and subsequent selection of our FY 2014 research projects. The selected research projects include a method for quantifying uncertainty in early lifecycle cost analysis (QUELCE), techniques for automated discovery of vulnerabilities in x86 binaries, and a multi-faceted project involving several parts of CMU’s School of Computer Science that focus on technologies needed by soldiers and first responders operating at the tactical edge (EETS). You’ll be able to read more about many of these projects in upcoming postings on the SEI Blog and forthcoming web sites. We recently presented the strategic direction and results of our proposal process to the SEI’s Technical Advisory Group (TAG) and Joint Advisory Committee Executive Group (JAC-EG) that report to our DoD government sponsors at the Assistant Secretary of Defense for Research and Engineering (ASD(R&E)). Likewise, we also recently presented this material to the SEI’s Board of Visitors (BoV), which reports to the provost at CMU. Thanks to hard work by many members of the SEI, these reviews have gone exceptionally well. Indeed, members of the TAG, JAC-EG and BoV have offered many constructive suggestions and connected the SEI with fruitful collaboration opportunities. Strategic Direction My background and experience as a product developer, researcher, and an adviser to the DoD have taught me several things. First, the best research often comes from working on real problems with real users. Second, working with high-quality colleagues breeds high-quality work, so growing our connections with the best researchers and practitioners will help bring out the best in our technical staff. Third, rigid stovepipes lead to duplication, inefficiency, and sometimes conflict. Based on these observations, I have assembled the following goals for myself, the office of the CTO, and other groups at the SEI over the coming months. We intend to Develop an even higher quality and cohesive research program. Within the scope of visibility and responsibility for oversight that the CTO’s office has at the SEI, I want to ensure research projects are of high quality. In particular, each funded project at the SEI must have a clearly stated motivation and intellectual merit, along with a grounded understanding of the parties that can benefit from the work and the estimated timeframe involved. As deputy director for research, I also want to ensure the SEI’s research program is cohesive, and that the SEI’s portfolio of projects explores and defines important areas that best leverage our talents and capabilities, while also creating useful techniques, methods, and tools for our government, industry, and academic stakeholders. Increase collaboration with CMU and other academic researchers. The SEI is one of only two, DoD-sponsored FFRDCs operated by a university. As such, we have a special opportunity and obligation to participate with academia to ensure that our sponsors have access to the most innovative and game-changing technology and ideas available. We are positioned to advance this goal in a way unavailable in the past now that our DoD ASD(R&E) sponsor has enabled us to designate work as "fundamental research," which enables it to be published in a timely manner. I am committed to effectively leveraging the expertise and collaboration opportunities available to the SEI as part of CMU. We are also seeking to expand our engagement with other academic and service lab researchers. Enhance accessibility to SEI’s work. From an operational perspective, I also want to make it easier for sponsors, partners, and software professionals to interact and collaborate with the SEI. In addition to the fundamental research designation mentioned above, we are reorganizing to make us more structurally efficient, to facilitate collaboration across the SEI’s divisions, and to foster feedback loops among various groups of researchers and practitioners. This reorganization includes an internal and external "knowledge management" effort to ensure members of the SEI community can easily determine what type of work we are doing and have done in the past. In addition, we are mining our own expertise and best-known methods derived from research insights and from experience gained through working with our customers. My goal in writing this blog posting was to introduce SEI blog readers to my background and provide some insights into what I see as the shape of things to come at the SEI. I am excited about the opportunities in front of us and the structural changes we are undertaking. I believe we will continue to build upon our legacy of success in software research and transition with help from you. I’ve been working with members of the SEI’s Tech Council to craft a strategic research plan that I will describe in later blog postings. Please don’t hesitate to contact the office with ideas, thoughts, constructive suggestions, or other comments. You may also send an email to info@sei.cmu.edu, or leave feedback in the comments section below. Thanks! Additional Resources For more information on SEI research activities, please visithttp://www.sei.cmu.edu/research. To view my staff page and the office of the SEI’s chief technology officer, please visithttp://www.sei.cmu.edu/about/people/profile.cfm?id=fall_16588

By James Cebula Senior Member of the Technical StaffCERT Division Risk inherent in any military, government, or industry network system cannot be completely eliminated, but it can be reduced by implementing certain network controls. These controls include administrative, management, technical, or legal methods.  Decisions about what controls to implement often rely on computed-risk models that mathematically calculate the amount of risk inherent in a given network configuration. These computed-risk models, however, may not calculate risk levels that human decision makers actually perceive. These decision makers include the network team (e.g., those people who work to implement the controls to ensure the network is secure), the information assurance (IA) officer, and authorizing officials. For example, these models may be missing immeasurable risk factors (e.g., adversarial sophistication or the value to an adversary of data stored on the network) that decision makers perceive as high risk.  This blog post describes the problem of how network security professionals perceive risk in more depth, as well as the SEI’s research efforts to study what risk factors influence perceptions of network risk and how risk perceptions are formulated.  Within the Department of Defense (DoD) and .gov civilian government space, organizations must go through a rigorous process of control implementation, control testing and validation, and documentation of results before bringing an information system or network into production. In this process, controls selected for a particular network are configured and tested by an organization’s network team, led by an information assurance (IA) officer, to reduce the amount of inherent network risk. Guidelines such as the International Organization for Standardization’s (ISO) 27000 series and the National Institute of Standards and Technology’s (NIST) 800 series exist for this rigorous process; however, these guidelines are generic and not tailored to an organization.  After control configuration and testing is complete, a so-called authorizing official signs-off on the network on behalf of the organization. This official indicates that prior to bringing the network into production, the level of residual risk is acceptable. Risk cannot be eliminated, but it needs to be managed at a level that allows the organization to carry out its mission.  The implementation of certain controls comes with tradeoffs (e.g., network performance decrements, network usability constraints, etc.). The ratio of risks to benefits accrued from a particular implementation of controls must be tolerable to the organization’s users, management, IA officer, and authorizing official. While the adverse consequences of a particular control configuration may be known during the process of assessing risks and benefits, the risk factors that the IA officer and authorizing official perceive are not known. Risk factors are pieces of information that are considered when formulating a perception of risk.  For example, an IA officer may believe the network risk is minimal based on risk factors such as minimal adversarial activity a fully staffed network team a network architecture without wireless connections Complications can arise, however, when the IA officer and authorizing official form different network risk perceptions. Circumstances can be equally dangerous when they agree. For example, an IA officer and authorizing official may agree that a particular network risk level is minimal, but the two parties may in fact be considering different risk factors. Consequently, communications between both parties about risk-mitigating strategies can be hampered. Risk perception and its underlying risk factors must be studied to improve network security.  One challenge to the study of risk perception is determining who has the ground truth on network risk levels.  For example, the network team and the IA officer may consider different risk factors than the authorizing official because they all often have different experiences and expertise. These risk factors may be contingent upon a person’s job role (e.g., the firewall expert may perceive risk factors related to protecting a network from intrusions that other team members may not perceive) or the context of the organization (e.g., the organization has no firewall expert).  Conversely, the authorizing official may consider additional risk factors that can be applied to any organization.  This potential for inconsistent perceptions of network risk led us to consider the following two questions: Who has the ground truth on the network risk level at any given moment? What risk factors are most important to perceptions of network risk? Our research is not intended to identify the ground truth, but rather to document the variance of risk factors considered when formulating a risk perception. The SEI research team includes Jennifer Cowley, Sam Merrell, Andrew Boyd, David Mundie, Dewanne Phillips, Harry Levinson, James Smith, Jonathan Spring, Kevin Partridge, and myself.  Through a mixed methods research approach, we aim to assess the factors that individuals use when formulating a perception of network risk whether different perceptions of network risk occur whether those differences are due to different network contexts, personal experiences, and different job-related expertise This research seeks to extend the field of risk perception research into cybersecurity.  There have been several studies examining computed risk in cybersecurity, but little research exists on perceived risk in cybersecurity-related disciplines.  The SEI has studied computed-risk modeling, which associates risk with the likelihood that an outcome will occur and the impact it will have on those affected. Computed risk modeling is frequently recommended as a means to conduct network risk assessments because it is based on measurable factors such as the degree of implementation of controls. In contrast, perceived risk is what individuals identify, understand, and view as the current network risk, independent of computed risk. Risk perceptions may be based on a variety of factors much broader than the factors used in computed risk models. These factors may include prior success or failure experiences with controls currently implemented, knowledge of the adversarial sophistication level, and organizational constraints. Computed risk levels may be one risk-perceptive factor among many considered when formulating network risk perceptions.  Our Research Approach Computer scientists and practitioners have recently developed an interest in studying the human element of network security. Our project is one of the first human-subjects studies undertaken at the SEI.  Prior to the initiation of studies, the SEI conducted a moderated focus group of subject matter experts (SMEs, e.g., former auditing officials and individuals with prior industry experience with the network accrediting process) to identify all possible factors that may impact network risk perceptions.  Examples of these factors include the presence or absence of a known adversary on the network the system of network controls that were or were not implemented workforce shortages current adversarial activity that is known by the operator Note: Internal factors, such as personality and mood, may be equally important  when researching network risk perceptions. To scope the work appropriately, however, we did not include them in this research.   Our mixed methods research approach involves four human-subject studies, each with a unique mission and research method. Table 1 summarizes our study purposes and methodology. The completed results will be available later this year. The four small studies sampled a wide variety of network professionals such as individuals who implement and test network controls, IA officers, and authorizing officials. Job-related demographics were also collected. Table 1.  Overview of the parameters of all four studies The factors generated in the SME focus group were tested in all four studies to assess the importance of each factor to perception of network risk. Because there is no unifying definition of risk, we asked participants to define risk in their own words, so we can analyze the data with respect to their definitions. In addition, we asked participants in studies 2-4 to use the NIST guidelines (NIST Special Publication 800-30) to indicate the level of network risk as low, medium, or high. In this research effort, context is operationally defined as description of an organization’s network at a particular point in time. We hope to identify factors that are consistently important for low-, medium-, and high-risk perceptions. Future Work Earlier studies have found that risk perceptions help predict behavior.  It is therefore essential that the IA officer, network team, and authorizing official have similar network risk perceptions to motivate effective risk mitigating behavior.  Conversations about network risk factors provide a more comprehensive picture of network risk than the NIST rating of low, medium, or high mentioned above. Our long-term goal is to develop a metric of perceived network risk, so that we can study the relationship between perceived risk and the performance of humans and teams. After we better understand how people perceive risk, we can enhance existing training curricula with strategies to mitigate the effects of perceived network risk on human and team performance. Additional Resources For more information about research in the SEI’s CERT Division, please visitwww.cert.org. For more information about the SEI’s research in risk, please visitwww.sei.cmu.edu/risk.

By Grace LewisTechnical Lead, Edge-Enabled Tactical Systems ResearchSEI Software Solutions Division Soldiers and emergency workers who carry smartphones in the battlefield, or into  disaster recovery sites (such as Boston following the marathon bombing earlier this year) often encounter environments characterized by high mobility, rapidly-changing mission requirements, limited computing resources, high levels of stress, and limited network connectivity. At the SEI, we refer to these situations as "edge environments." Along with my colleagues in the SEI’s Advanced Mobile Systems Initiative, my research aims to increase the computing power of mobile devices in edge environments where resources are scarce. One area of my work has focused on leveraging cloud computing so users can extend the capabilities of their mobile devices by offloading expensive computations to more powerful computing resources in a cloud. Some drawbacks to offloading computation to the cloud in resource-constrained environments remain, however, including latency (which can be exacerbated by the distance between mobile devices and clouds) and limited internet access (which makes traditional cloud computing unfeasible). This blog post is the latest in a series that describes research aimed at exploring the applicability of application virtualization as a strategy for cyber-foraging in resource-constrained environments. Cloudlet-Based Cyber Foraging Cyber-foraging is a technique for dynamically augmenting the computing resources of resource-limited mobile devices by opportunistically exploiting a fixed computing infrastructure nearby. This technique allows mobile users to offload computationally-expensive processing (such as face recognition, language translation, speech and image recognition) from a mobile device onto more powerful servers, thereby preserving device battery power and enabling more powerful cloud-based computing. These capabilities are valuable for soldiers or emergency workers who often operate in situations where these resource-intensive applications must be deployed reliably and quickly. Our research focuses on cloudlet-based cyber foraging. Cloudlets, a concept created by Mahadev Satyanarayanan "Satya" of Carnegie Mellon University’s School of Computer Science, are discoverable, generic, stateless servers located in single-hop proximity of mobile devices. Cloudlets can operate in disconnected mode, which means that communication with the central core is only needed for provisioning. They are also virtual-machine (VM) based, which means that they promote flexibility and mobility, a perfect match for edge environments. My previous work in this field focused on cloudlets provisioned using VM synthesis, which was described in the SEI technical note Cloud Computing at the Tactical Edge. When we first began using cloudlets for edge applications, we realized we’d need to make changes to the provisioning process them so they’d functional efficiently and effectively in resource-constrained environments. Satya first proposed using virtual machine (VM) synthesis to provision cloudlets. During VM synthesis, a file called an "application overlay" is transferred during runtime from the mobile device to the cloudlet. This application overlay is built offline and corresponds to the binary difference between a base VM and that VM after the server portion of an application is installed. After the VM overlay has been transferred, it is applied to the base VM. The result is a complete VM that is running the server portion of the application that is executed from a client running on a mobile device. As you can imagine, VM synthesis involves large application overlay files, which can be costly to transfer in terms of battery and network bandwidth consumption in mobile and edge environments. As an alternative, we started looking at application virtualization as a possible solution to this problem. Application Virtualization Application virtualization uses a similar approach to operating system (OS) virtualization, which tricks the software into interacting with a virtual rather than the actual environment. While OS virtualization emulates the hardware for a guest OS, application virtualization emulates OS functionality for an application. To accomplish this emulation, a runtime component intercepts all system calls from an application and redirects these to resources inside the virtualized application. As we described in our technical report, Application Virtualization as a Strategy for Cyber Foraging in Resource-Constrained Environments, the combination of cloudlets and application virtualization allowed us to support the following design goals: Simplicity. Cloudlets should be set up quickly and conveniently. Preparing an application for deployment on a cloudlet should not involve extensive manual overhead, and cloudlet discovery should not require action from the user. Similarly, offloading the application to the cloudlet must be intuitive and simple.  Generality. Packaged applications should be loosely coupled to the operating system so that they can run on multiple cloudlets. This loose coupling enables regular updates and upgrades to the operating system that runs virtual applications without breaking functionality. Applications that are not too deeply integrated into the operating system or too specific to special hardware should be eligible for offloading to the cloudlet. Speed. The time from when the user selects an application to when it is ready for execution—which includes code offload time—should be reasonably small. The user must be able to track the deployment progress via messages from the cloudlet. Our approach to application virtualization with cloudlets accomplishes these goals because it does not require any code modifications and provides a high degree of application portability. Application virtualization tools generate smaller files since they package only those dependencies that are necessary for portability. Our experiments (reported in our technical note) showed that a virtualized application is approximately five times smaller than its application overlay equivalent. Smaller file sizes also lead to smaller application-ready times, defined as the time between the start of the transfer of the virtualized application to the cloudlet and the time that the application is ready for execution on the cloudlet Implementation The two main components of our implementation are the mobile device and the cloudlet host, as shown in the figure below.  The cloudlet host serves as a machine that lends its resources to the mobile device. In our implementation, all devices are connected to the same subnet within a wireless network. The cloudlet host runs a hypervisor to host multiple VMs, which provide a selection of various operating systems and versions. The two main architectural components of our implementation are The mobile device. We developed on Android 4.1, which supports multicast and is required for the discovery mechanism. All the elements of the cyber-foraging-ready application are stored on the mobile device including the application client, the application metadata, and the application package that contains the application server. The cloudlet host. This component comprises a multicast-supporting machine that runs the VM hypervisor. At the SEI, we used two tools to create and execute the virtualized applications for both Linux and Windows systems. Code, Data, and Environment (CDE) is an application virtualization tool for Linux developed by Philip J. Guo and Dawson Engler. CDE allows for virtualizing applications by monitoring their execution. Through the ptrace system call, the supervising CDE program finds files that have been accessed during execution and packages them. The resulting package also contains the environment settings and the CDE runtime environment, which executes the application. Cameyo, an application virtualizer for Windows, packages the application and its dependencies into one single, executable file. Unlike CDE, which monitors execution, Cameyo monitors the installation process and offers two mechanisms for accomplishing the virtualization.  The first mechanism involves taking a snapshot of the system, installing the application, taking another snapshot, and then computing the dependencies and modified registry keys in the difference between the two snapshots. Instead of snapshots, the second mechanism simulates the installation process, tracking all of the installer’s actions. This simulated installation does not have any permanent effect on the system. Benefits and Drawbacks of Our Approach Application virtualization has an advantage over VM synthesis in terms of its performance while deployed. We relied on two metrics to evaluate our approach: deployment time and energy consumption. These two metrics are related because the greater the amount of data transmitted, the greater amounts of battery power required. One benefit of application virtualization is significantly smaller file sizes, the result of packaging just the application and not calculating a binary difference between virtual machine images. Another benefit of application virtualization is the loose coupling between an application and its required cloudlet environment. We did realize some drawbacks of our approach. Application virtualization is not suitable for every type of environment. For example, device drivers interact with the hardware directly and cannot be virtualized. It is also hard to virtualize software that interacts with OS internals, such as antivirus programs. Application virtualization also requires careful dependency management to ensure that an application operates correctly on a cloudlet. If all dependencies are not included in the virtualized application then it will not work. Moreover, missing dependencies must be added manually, which is tedious and error-prone.  Future Work in Advanced Mobile Systems Our Advanced Mobile Systems Initiative is building a portfolio of capabilities and options for use by soldiers and emergency workers who operate in various environments with different quality attribute requirements. Cyber-foraging is one of three areas that we are investigating and our body of work on support for resource-constrained environments is growing. The other two areas of work in the SEI’s Advanced Mobile Systems Initiative are group-context-aware mobile applications in which groups of mobile devices share contextual information so that teams and their mobile devices can make better decisions regarding resource consumption These types of applications also give users a better understanding of environmental factors and the activities being performed by the group based on information gathered by sensors on the mobile device or connected to the mobile device. user-configured situational awareness mashups that allow soldiers and emergency workers to filter and combine geo-located data such as DoD situational awareness SA feeds and public data sources into a single map-based user interface. In developing this cyber-foraging strategy, I collaborated with Dominik Messenger, a student from the Karlsruhe Institute of Technology (KIT) in Germany. He came to CMU last year via the InterACT exchange program. His advisor at KIT contacted me and said that he was very interested in our research. This research and the resulting technical report were part of his diploma thesis. We look forward to your feedback on our work in the comments section below. Additional Resources To view the SEI technical note, Application Virtualiation as a Strategy for Cyber Foraging in Resource-Constrained Environments, please visithttp://www.sei.cmu.edu/library/abstracts/reports/13tn007.cfm. To read the SEI technical note, Cloud Computing at the Tactical Edge, please visithttp://www.sei.cmu.edu/library/abstracts/reports/12tn015.cfm. For more information on the SEI’s research in the field of pervasive mobile computing, please visit http://www.sei.cmu.edu/mobilecomputing/research/index.cfm To read the paper, The Case for VM-Based Cloudlets in Mobile Computing, please visit http://www.cs.cmu.edu/~satya/docdir/satya-ieeepvc-cloudlets-2009.pdf.