"Perfect", "Can’t be improved", "No further improvement needed" - reading through customer feedback for the Carers Allowance Digital Service it strikes me how many of our users think that there’s nothing more to do. We can pack up and go home.  Au revoir. Sayonara. Farewell! Far be it from me to suggest these particular users are wrong, it’s just that they’re not quite right on this one. Excellent, yes. Perfect, no. And it’s not just our users. We achieved Live Accreditation Status in November 2014 and for many people this translated as ‘job done’. Erm…no! We haven’t ‘gone live’ in the old fashioned sense. The interpretation of go-live that applies to traditional projects is ‘go-dead’ in the context of agile. We ‘go live’ every fortnight. When we went Live there was a backlog of changes and features still waiting to be prioritised. We didn’t get to live and stop, we got to live and started doing more. Since November we've made over 20 releases covering around 300 user stories. We haven’t done that for the sake of it, it’s in response to the needs of our users. These needs drive everything we deliver, they don’t stop, they change and evolve. We’re here to ensure that the service meets those needs. As the service matures we've put even more focus into our analytics to identify barriers within the user journey; why are we losing so many people at the disclaimer? Why are so many people calling to find out whether we have received their claim? Why do people spend so long at certain questions? This analysis feeds the research, the research feeds the backlog and it all contributes to continuously improving the service for users. Here’s just a few of the many improvements delivered since live, we've: moved the Disclaimer to the front of the service so users know what they are signing up to before they start - completion rates have rocketed from 60% to over 80%. Simpler for users. introduced email notifications to let users know that we've received their claim resulting in reduced calls to our contact centres. Clearer for users. reviewed and clarified all the questions - completion times have tumbled below 25 minutes. Faster for users. It doesn't stop with research. We’re trialling new technologies, pushing new infrastructure and security features that will enable smoother delivery of the Departments other digital initiatives. This means more and more services will be delivered this way. Services that will work better for users. With a satisfaction score consistently at 90% and being told we’re perfect, we’re in a happy place but this journey is far from complete. We’ll continue to do more until the service no longer needs to meet the need of its users - and that doesn't feel like any time soon.

By Greg ShannonChief ScientistCERT Division For two consecutive years, organizations reported that insider crimes caused comparable damage (34 percent) to external attacks (31 percent), according to a recent cybercrime report co-sponsored by the CERT Division at the Carnegie Mellon University Software Engineering Institute. Despite this near parity, media reports of attacks often focus on external attacks and their aftermath, yet an attack can be equally or even more devastating when carried out from within an organization. Insider threats are influenced by a combination of technical, behavioral, and organizational issues and must be addressed by policies, procedures, and technologies. Researchers at the CERT Insider Threat Center define insider threat as actions by an individual who meets the following criteria: a current or former employee, contractor, or business partner who has or has had authorized access to an organization’s network, system, or data and intentionally exceeded or intentionally used that access in a manner that negatively affected the confidentiality, integrity, or availability of the organization’s information or information systems. Insider threats are influenced by a combination of technical, behavioral, and organizational issues that organizations must address through policies, procedures, and technologies. Insider threats are influenced by a combination of technical, behavioral, and organizational issues and must be addressed by policies, procedures, and technologies. Researchers at the The CERT Insider Threat Center provides analysis and solutions to organizations through partnerships with the U.S. Department of Defense, the U.S. Department of Homeland Security, the U.S. Secret Service, other federal agencies, the intelligence community, private industry, academia, and the vendor community. This blog post, the second in a series, introduces the CERT Insider Threat Center blog, which highlights the latest research and security solutions to help organizations protect against insider threat. Before we take a deeper dive into the most visited insider threat posts of the last six months, let's take a look at the top 10 posts (as measured by number of visits) on both CERT blog (CERT/CC and Insider Threat): The most visited posts on the CERT/CC blog center around a critical area of research: SSL Certificates as a core foundation of trust transmissions on the Internet along with certificates. These posts explore weaknesses in those trust relationships as implemented in mobile platforms and also highlight tools that have been created at CERT to explore those vulnerabilities. Before we take a deeper dive into SSL Certificates, let's take a look at the top 10 posts (as measured by number of visits) on the CERT blogs: Although some of these posts are several years old, their continued popularity demonstrates the ongoing relevance of work by researchers at the CERT Insider Threat Center. Insider Threat Statistics During presentations, assessments, or while instructing courses, our researchers are often asked about the state of insider threat. "Just how bad is it?" is a question often heard.  Capturing accurate data on insider threat proves difficult, however, as organizations are often loathe to report incidents and risk negative press or damage to standing. These repeated requests became the catalyst for the post Interesting Insider Threat Statistics which has been the most popular post on the CERT Insider threat blog in the six months ending in March 2015. This blog post presents statistics as well as the cost that organizations encounter as a result of an insider threat incident. Here is an excerpt from the post: According to the 2010 CyberSecurity Watch Survey, sponsored by CSO Magazine, the United States Secret Service (USSS), CERT, and Deloitte, the mean monetary value of losses due to cybercrime was $394,700 among the organizations that experienced a security event. Note that this figure accounts for all types of security incidents, including both insiders and outsiders. What is especially concerning is that 67 percent of respondents stated that insider breaches are more costly than outsider breaches. This dollar figure does not fully account for the damages caused by insiders, though. For instance, activities such as website defacement and exposure of private email correspondence may not involve expensive remediation, but they would still cause a great deal of harm to the victim organization. How valuable is your reputation? How much does your website represent you? If you are an e-commerce company that assures its customers that they will have secure transactions, imagine the damage to your business if your website gets compromised. Another common question we often receive is, "How many insider attacks take place annually?" This is a much more difficult question to answer. Consider that in the same survey, among 523 respondents, 51% of those who experienced a security incident also experienced an insider attack. The problem with approximating a total number of insider attacks is that, in our experience, a large number of these attacks go unreported. In fact, according to the survey, "the public may not be aware of the number of incidents because almost three-quarters (72%), on average, of the insider incidents are handled internally without legal action or the involvement of law enforcement." There are a variety of reasons why companies choose not to report insider cases; in particular, lack of evidence to prosecute, damage levels that were insufficient to warrant prosecution, inability to identify the perpetrator, and fear of public embarrassment. However, even this does not tell the full story. Based on our research and collaboration with other industry leaders, we believe that most insider crimes go unreported not because they are handled internally, but because they are never discovered in the first place. The complete post Interesting Insider Threat Statistics can be read here.  Insider Threat and Physical Security of Organizations In our database of incidents involving malicious insider activity—these include crimes of IT sabotage, theft of intellectual property, and fraud—about 8 percent involve physical security issues. Physical access to an organization's secure areas, equipment, or materials containing sensitive data may make it easier for a malicious insider to commit a crime. Therefore, an organization’s physical security controls are often just as important as its technical security controls. The post Insider Threat and Physical Security of Organizations provides some case studies of physical security issues as well as some physical security controls. Here is an excerpt from the post: In our case repository of incidents of malicious insider activity, including crimes of IT sabotage, theft of intellectual property, and fraud, about 8 percent involve physical security issues of concern. The case summaries below outline a few of these cases that we’ve analyzed. For more than a year, a contract janitor stole customer account and personally identifiable information from hard-copy documents at a major U.S. bank. The janitor and two co-conspirators used this information to steal the identities of more than 250 people. They were able to open credit cards and then submit online change-of-address requests so the victims would not receive bank statements or other notifications of fraudulent activity. The insiders drained customers’ accounts, and the loss to the organization exceeded $200,000. A contract programmer tricked a janitor into unlocking another employee’s office after hours. He switched the door’s name plate and requested that the janitor let him into "his" office. The programmer, who had already obtained employment with a competitor, was able to download sensitive source code onto removable media. A hospital security guard accessed and stole personally identifiable information regarding the organization’s patients. The guard and three co-conspirators opened fraudulent cell phone plans and credit card accounts. As part of the scheme, they changed the account addresses of the victims so the bills would never reach the account owners. After being caught, the insider was ordered to pay $18,000 for the crime. A communications director showed an expired ID badge to a security guard to gain unauthorized access to a data backup facility. Once inside, the director unplugged security cameras and stole backup tapes containing records for up to 80,000 employees. A contract security guard used a key to obtain physical access to a hospital’s heating, ventilating, and air conditioning (HVAC) computer and another workstation. The guard used password-cracking software to obtain access and install malicious software on the machines. The incident could have affected temperature-sensitive patients, drugs, and supplies. An insider stole an organization’s trade-secret drawings that were marked for destruction and sold them to a competing organization. The victim organization estimated its losses at $100 million. The competing organization that received the stolen documents was forced to declare bankruptcy after a lawsuit. We have also observed the following physical security issues in the case data: Infiltration/exfiltration of physical property: activities such as bringing removable media in and out of a facility Improper termination of an employee’s physical access or access badge Unauthorized access to facility: employees entering facilities during unusual hours or unauthorized employees walking through an open door behind an authorized employee (known as "piggybacking") Generally poor physical security: general issues such as insufficient guard oversight or insufficient separation of duties for physical access controls Employee used an unauthorized workstation: employees who are able to physically enter another employee’s office/workspace and access their workstation Breaking and entering/physical destruction: employees breaking into secure spaces or stealing physical equipment Janitorial staff issues: janitorial staff who steal sensitive information or are socially engineered into violating physical security Improper disposal or destruction of organization information The complete post Insider Threat and Physical Security of Organizations can be read here. Theft of Intellectual Property and Tips for Prevention One of the most damaging ways an insider can compromise an organization is by stealing its intellectual property (IP). An organization cannot underestimate the value of its secrets, product plans, and customer lists. In the publication An Analysis of Technical Observations in Insider Theft of Intellectual Property Cases, CERT Insider Threat researchers took a critical look at the technical aspects of cases in which insiders stole IP from their organization. Insiders commit these crimes for various reasons, such as to benefit another entity, to gain a competitive business advantage, to start a competing organization or firm, or to gain personal financial benefit. By understanding the specific technical methods that insiders use to steal information, organizations can consider gaps in their network implementation and can identify ways to improve controls that protect their IP. Technical discussions of IP theft are helpful for operational staff to understand how insiders can compromise their organization. Additionally, organizations should always attempt to better understand the human behavioral elements of insider crimes. The report A Preliminary Model of Insider Theft of Intellectual Property details two preliminary models of behavior associated with insider theft of IP. The third most visited post on the CERT Insider Threat blog Theft of Intellectual Property and Tips for Prevention presents highlights of the research in both reports. Here is an excerpt from the post: Our study indicated that the most common method of physical exfiltration of data was removable media. Prior to 2005, the most common removable medium was writable CD. However, recent incidents indicate that removable USB mass storage devices like thumb drives and external hard disks are now more popular. USB devices have a much greater storage capacity than CDs, which makes it easier for insider to move their entire desired data set at once. What can organizations do about these problems? First, they can always consider the role of best practices and established standards in defending against insider attacks. Insider attacks frequently exploit policies or controls that are covered in accepted best practices for IT system security. Second, organizations should always consider more than just the technical aspects of the crime. In a recent report Deriving Candidate Technical Controls and Indicators of Insider Attack from Socio-Technical Models and Data, we examined the importance of creating technical indicators for behavioral actions so that we can gain a more complete understanding of how to defend against insider crimes.Organizations should pay specific attention to these technical vulnerabilities while they attempt to understand what controls are practical to put in place for removable media in the organization. If removable media is necessary to keep operations moving, an organization may want to establish technical measures to limit which machines allow use of removable media, take an inventory of authorized media, and implement some measure of physical security to prevent removal or introduction of new uninventoried devices from the facility. When considering network security, organizations should attempt to identify suspicious email communications (particularly with attachments) to direct competitors, foreign governments, or other illegitimate recipients of corporate mail. Organizations should consider using a log aggregation and indexing tool to look for patterns in behavior that might warrant further investigation. This is especially true during major organizational events that may cause stress among employees, such as mergers, downsizing, acquisitions, or reorganizations. These events could possibly influence employee behavior in a negative way, and a heightened awareness of security might be necessary. The complete post Theft of Intellectual Property and Tips for Prevention can be read here.  Theft of Intellectual Property by InsidersThe CERT insider threat database was started in 2001 and contains insider threat cases that can be categorized into one of four groupings: fraud sabotage theft of intellectual property miscellaneous The post Theft of Intellectual Property by Insiders presents cases in our database that involve the theft of IP. As of the date of this post (December 18, 2013), 103 insider threat cases in the database included the theft of IP. (All statistics are reported as a percentage of the cases that had relevant information available.)Here is an excerpt from the post: Insider theft of IP occurred most frequently in the information technology (35 percent of cases), banking and finance (13 percent), and chemical (12 percent) industry sectors. (The industry sector was known in 101 of the 103 cases.) The majority of insider IP theft incidents occurred onsite.  (The attack location was known in 78 of the 103 cases.)   Trusted business partners accounted for over 17 percent of attackers and former employees accounted for 21 percent. (Employment status was known in 100 of the 103 cases.) Over 30 percent of insider theft of IP cases were detected by non-technical means, while fewer than 6% cases were detected by a software solution. The financial impact of these attacks is substantial. The impact was over $1,000,000 USD in 48 percent of cases and over $100,000 in 71 percent of insider theft of IP cases.  (Financial impact was known in 35 of the 103 cases.) For additional information and more in-depth analysis of the insider threat cases involving the theft of IP with foreign beneficiaries, please see our report Spotlight On: Insider Theft of Intellectual Property Inside the United States Involving Foreign Governments or Organizations. In addition to the theft of intellectual property, the CERT Insider Threat Center has conducted studies of other insider threat cases, including insider fraud in the U.S. financial services sector and potential patterns of insider threat cases involving sabotage. The complete post Theft of Intellectual Property by Insiders can be read here. Looking Ahead: Helping Organizations Establish an Insider Threat Program This has been an important year for the insider threat blog in terms of keeping our stakeholders informed and helping them protect themselves against ever-present cyber threats. Now that we have looked at the top posts, I would like to make you aware of a new series that was recently launched on the Insider Threat blog. Earlier this year, researchers at the CERT Insider Threat Center launched a series of blog posts aimed at helping organizations establish an insider threat program. This series is intended to help organizations affected by Executive Order 13587, Structural Reforms to Improve the Security of Classified Networks and the Responsible Sharing and Safeguarding of Classified Information, to establish a program for deterring, detecting, and mitigating insider threats. This executive order affects organizations that work within the U.S. federal government and that operate or access classified computer networks. The first post by Randy Trzeciak, technical manager of the Insider Threat Center, was published early in March 2015 and outlines planned posts for the series. Here is an excerpt from that post: Because of a number of high-profile incidents that have significantly impacted organizations recently (e.g., sabotage, theft of information, fraud, national-security espionage), many organizations across government, industry, and academia have recognized the need to build an insider threat program (InTP) to protect their critical assets. Over the course of the next few months, we will be discussing the following topics as part of our blog series:    Introduction to the CERT Insider Threat Center    Components of an Insider Threat Program    Requirements for a Formal Program    Organization-Wide Participation    Oversight of Program Compliance and Effectiveness    Integration with Enterprise Risk Management    Prevention, Detection, and Response Infrastructure    Insider Threat Training and Awareness    Confidential Reporting Procedures and Mechanisms    Insider Threat Practices Related to Trusted Business Partners    Data Collection and Analysis Tools, Techniques, and Practices    Insider Incident Response Plan    Communication of Insider Threat Events    Policies, Procedures, and Practices to Support the Insider Threat Program    Protection of Employee Civil Liberties and Privacy Rights    Defining the Insider Threat Framework    Developing an Implementation Plan    Conclusion and Resources In this series we will describe the key elements of an effective insider threat program. We will begin by examining the need to build a program. The complete post, InTP Series: Establishing an Insider Threat Program (Part 1 of 18), can be read here. As always, we welcome your ideas for future posts and your feedback on those already published. Please leave feedback in the comments section below. Additional Resources: For more information about the CERT Insider Threat Center, please visithttp://www.cert.org/insider-threat/. To view the CERT Insider Threat Blog, please visithttp://www.cert.org/blogs/insider-threat/.

By Chris TaschnerProject Lead CERT Cyber Security Solutions Directive This post is the latest installment in a series aimed at helping organizations adopt DevOps. Tools used in DevOps environments such as continuous integration and continuous deployment speed up the process of pushing code to production. Often this means continuous deployment cycles that could result in multiple deployments per day. Traditional security testing, which often requires manually running multiple tests in different tools, does not keep pace with this rapid schedule. This blog post introduces a tool called Gauntlt, which attempts to remedy this issue. The idea behind Gauntlt is to make writing rugged code easier by allowing developers to turn security testing into code. This approach, in turn, simplifies the integration of security testing into the deployment and testing processes. Gauntlt provides adapters for curl, nmap, sslyze, and garmr and also features a generic command line adapter to run any command line tool. One of the greatest changes to software engineering that DevOps presents is getting developers and operations together collaborating during the entire systems development lifecycle (SDLC). Adding security testing into the SDLC will help to reduce some problems with security testing. Often security testing is done after the fact and then solutions are bolted on. Security testers, developers, and operations staff often work in silos, which reduces communication. This isolated work environment also means that when security flaws are found in software or infrastructures, disputes will rise. Because security testing is typically done so late in the process, it can become costly to add required security solutions. In addition, flaws might be missed when testing is done on a large code base. Gauntlt allows development and operations teams to collaborate with security testers and work side-by-side throughout the SDLC by enabling the performance of security testing via test scripts, which can be incorporated into the continuous integration process. This approach helps move security testing left, from the end of the process to the start. Pushing security testing left improves security of the application The idea of having a single, readable test script lets developers and operations staff contribute and inspect the tests being performed. This collaboration helps prevent possible miscommunication about what should and shouldn’t be allowed. Using this methodology, security testing engineers won't design tests with assumptions that developers aren’t aware of, and developers will learn security practices and become better equipped to build in security up front. In addition, Gauntlt enhances the reproducibility of security testing. Individual security testers can run any of the tools mentioned above and interpret the output and be effective. But, if the security testers are having an off day, or forget to run a particular test, they can easily miss important security issues. Gauntlt turns security testing into code and makes security testing easily repeatable by performing all security testing at the push of a button. Launching Gauntlt for the first time requires a quick three-step process that’s described at gauntlt.org. Once Gauntlt is installed, attacks can be written utilizing the tools provided. "Attack" is the terminology that Gauntlt uses to mean security test. Gauntlt provides some example attacks written for each of the adapters it handles. The attack files are written using the Gherkin syntax as defined by Cucumber. Here is a sample attack that attempts to detect robots.txt files on a webserver. The @slow tag tells Gauntlt to allow for a 30-second timeout, @veryslow allows for a five-minute timeout. These timeouts prevent tools from holding up testing while allowing for flexibility for tests that might take longer to complete normally. @slow Feature: nmap robots attackBackground:  Given "nmap" is installed  And the following profile:    | name     | value      |    | hostname | www.cert.org |Scenario: Detects robots.txt files on this host.  When I launch an "nmap" attack with:    """    nmap --script http-robots.txt <hostname>    """  Then the output should contain:    """    | http-robots.txt:    """ Overall, Gauntlt provides a powerful framework that can be used as a foundation for performing directed tests on a web project. With this tool users can begin to further automate their testing environment and simplify the interpretation of testing results. For more information on the syntax and how to utilize this tool see Hands On Gauntlt by James Wickett. The site includes a free sample that will help users get started. Every two weeks, the SEI will publish a new blog post offering guidelines and practical advice for organizations seeking to adopt DevOps in practice. We welcome your feedback on this series, as well as suggestions for future content. Please leave feedback in the comments section below. Additional Resources To view the webinar Culture Shock: Unlocking DevOps with Collaboration and Communication Aaron Volkmann and Todd Waits please click here. To view the webinar What DevOps is Not! with Hasan Yasar and C. Aaron Cois, please click here. To listen to the podcast DevOps—Transform Development and Operations for Fast, Secure Deployments featuring Gene Kim and Julia Allen, please click here. To read all of the blog posts in our DevOps series, please click here.

I am one of six people who were recently selected for the first ever DWP Junior WebOps training program. The program was specifically created for DWP by external IT training provider QA Training with input from the DWP WebOps team. It runs for 12 weeks and covers a wide range of topics and skill areas that are essential to creating and maintaining user-centred digital services at DWP. The six of us are a mix of ages and backgrounds. Everyone on the program has come from a different part of DWP but we all have a common interest in WebOps and all share a desire to learn. What is WebOps? WebOps was initially described to me by colleagues as "the dark arts" that keep the wheels turning at DWP Technology. A few weeks into the training program, I’d sum up WebOps as the deployment, operation, maintenance, tuning and repair of web-based services. This means that once a service has been developed and delivered into the live environment by our product teams, it’s up to WebOps to keep it functioning as designed. The Junior WebOps Program As we develop new digital products and services at DWP, so the WebOps team needs to grow to support them. The Junior WebOps training program is one way of meeting that need by developing skilled staff in house (we’re also recruiting for experienced WebOps professionals right now). The program is divided into three blocks - I’ve just completed the first block. The learning experience The first day was like attending a new school. Getting up at 5.30am to get to Leeds for 9am wasn’t pleasant, but I was eager (and a bit nervous) for the massive journey that lay ahead. I’d had a telephone call with the others on the program the week before starting, but it was still exciting getting to know each other on that first day, and embarking on a 12-week journey together into the unknown. The first week focused on agile and lean. These are methods of working and thinking that are now well embedded at DWP when it comes to building digital services. Being from an ITIL & PRINCE2 background, it was extremely interesting and new to me, but I definitely saw the value in it and we started applying some of the agile principles immediately (more on that later). By the end of Week 1 we had formed a great team (Lego and an impromptu evening out with the WebOps team and apprentices might have played a part!) and were set for the rest of Block 1. Core WebOps Technologies The next few weeks focused on the core technologies used in WebOps. First up was Linux in Week 2 - a whole new operating system for most of us. The course was excellent, although even with a technical background I found it intense and challenging at times. We were told before we started that this program wasn’t going to be easy and that it would require serious commitment - Week 2 really brought this home. Week 3 began with a couple of revision days - some time to gather thoughts and collate what had been learnt to date - and finished with some structured networking with experienced WebOps colleagues and staff from other teams. This turned out to be the calm before the storm. Week 4 ramped up the intensity even further as we took on the LAMP stack (Linux, Apache, MySQL and PHP) - the backbone of the WebOps world. We had got the basics of working with Linux down in Week 2 so it was good to break those skills out again, but it was still an incredibly challenging few days learning two new software packages and a programming language. Incorporating agile One of the best things about the program is that we weren’t just learning abstract concepts - we started using the things we were learning immediately, particularly when it came to agile. Once we’d got the basic agile learning down, we began each day with a stand-up meeting where we could discuss any issues we’d had the day before and agree what we would do that day. Not only did the stand-ups help us get to know each other better, they were also a chance to help each other out with anything we were struggling with and ensure everyone was progressing evenly - really useful given our different backgrounds and knowledge levels. Each week also finished with a retrospective where we advised both the WebOps team and the external training provider on our experiences of the course - what went well, what didn’t and any recommendations for improvements. This is important to the program as we are the first iteration and our feedback will help improve and shape the next version of the course. My personal retrospective on Block 1 is that it has been tough but enjoyable, if you like to learn! I can’t wait to get stuck into the next Block.

In about a month, individuals will be heading to Las Vegas to attend the 2015 SHRM Annual Conference.  This will be my 15th SHRM Annual Conference, and, based on my years of experience, here are the things you do NOT want to do while attending. 1.  Do NOT avoid drinking water Its the desert, people.  Every day will likely be 100 degrees and it will be a dry heat, so you won't even feel like you're sweating.  But, given the significant amount of walking you're likely to do as well as the...

By Carol Woody Technical Manager of the Cybersecurity Engineering Team CERT Division This post was co-authored by Bill Nichols. Mitre’s Top 25 Most Dangerous Software Errors is a list that details quality problems, as well as security problems. This list aims to help software developers "prevent the kinds of vulnerabilities that plague the software industry, by identifying and avoiding all-too-common mistakes that occur before software is even shipped." These vulnerabilities often result in software that does not function as intended, presenting an opportunity for attackers to compromise a system. This blog post highlights our research in examining techniques used for addressing software defects in general and how those can be applied to improve security detection and management. The errors on Mitre’s Top 25 list are both quality problems and potential security problems. Software security generally shares many of the same challenges as software quality and reliability. Consider Heartbleed, a vulnerability in the open source implementation of the secure socket layer (SSL) protocol. At the time Heartbleed was discovered, available software assurance tools were not set up to detect this vulnerability. As we discuss later in this post, however, Heartbleed, could have been found through thorough code inspection. Assurance and Finding Defects We define software assurance as demonstrating that software functions as intended and only as intended. Vulnerabilities permit unintended use of software that violates security and are therefore defects.  Unfortunately, all software begins with defects, and we have no means to prove that any software is totally free of defects or known vulnerabilities. This ambiguity means that there will always be a risk that software will not function as intended at later times. Likewise, this risk led us to explore whether techniques used for addressing defects in general can be used to improve security defect detection and management. We began by reviewing detailed size, defect, and process data for more than 100 software development projects amassed through the SEI’s Team Software Process (TSP) work. The projects include a wide range of application domains, including medical devices, banking systems, and U.S. federal legacy system replacement. This data supports potential benchmarks for ranges of quality performance metrics (e.g., defect injection rates, removal rates, and test yields) that establish a context for determining `very high quality products. We have the following types of information available for each project: summary data that includes project duration, development team size, cost (effort) variance, schedule variance, and defects found; detailed data that includes (planned and actual) for each project component: size (added and modified lines of code [LOC]), effort by development phase, defects injected and removed in each development phase, and date of lifecycle phase completion. We analyzed this data to identify baselines for expected numbers of defects of various types and the effectiveness of defect removal techniques. Before we discuss our findings, it is important to note that injecting defects is really an inevitable byproduct of software development. Developers make defects as a natural byproduct of building and evolving software. Removing these defects, however, proves more difficult. It is hard to eliminate all defects from software. The data we analyzed indicated that defects similar to many known security vulnerabilities were injected during the implementation phase. A few of these software products (five) not only had low levels of defects when released, but also were used in domains that required safety or security. The additional testing or analysis performed on these projects gave us a measured sample of escaped safety or security defects. We found there is no silver bullet for addressing defects or security vulnerabilities. Among the cases we examined that proved most effective, however, was the application of standard quality techniques, such as documented designs, review of the designs against requirements, code inspections (all performed prior to testing). The projects most effective at defect removal did not rely solely upon testing or static analysis to discover defects. Testing and tools were used as a verification of completeness. In cases where early removal techniques had not been effectively applied, developers were often overwhelmed with responses from their static tools. Conversely, the projects that were applying strong early quality assurance techniques received substantially fewer warnings when using tools, making the follow up easier to manage. The five projects selected from the SEI’s TSP database demonstrated that producing products with few safety or security operational issues requires an integration of quality reviews for defect removal with security or safety-critical reviews. Examinations were done for both code quality and security/safety considerations at each review point in the lifecycle beginning with early design. As detailed in our technical note, Predicting Software Assurance Using Quality and Reliability Measures, which I co-authored along with Robert Ellison and Bill Nichols, assuring that a software component has few defects also depends on assuring our capability to find those defects. Positive results from security testing and static code analysis are often provided as evidence that security vulnerabilities have been reduced. Recent "goto fail" and Heartbleed vulnerabilities demonstrate, however, that it is a mistake to rely on these approaches as the primary means for identifying defects. As we learned by examining these two vulnerabilities, the omission of quality practices, such as inspections, can lead to defects that can exceed the capabilities of existing code analysis tools. More information on each of these cases is included below. Case Study: "goto fail" Vulnerability In 2014, Apple fixed a critical security vulnerability that was likely caused by the careless use of "cut and paste" during editing. The programmer embedded a duplicate line of code that caused the software to bypass a block of code that verifies the authenticity of access credentials. Researchers discovered this security flaw in iPhones and iPads; Apple confirmed that it also appeared in notebook and desktop machines using the Mac OS X operating system. The vulnerability is described in the National Vulnerability Database as follows: Impact: An attacker with a privileged network position may capture or modify data in sessions protected by SSL/TLS. Description: Secure Transport failed to validate the authenticity of the connection. This issue was addressed by restoring missing validation steps. This vulnerability allowed attackers to use invalid credentials to gain access to any information on the targeted device such as email, financial data, and access credentials to other systems and devices. A variety of standard quality techniques, such as a personal review by the developer or a more formal peer review, should have identified the defect for removal. A number of structured development techniques, if applied consistently, could also have identified and possibly prevented implementation of the security coding defect that led to the vulnerability, including designing code to minimize branching and make it more predictable and testable architecting the design specification to become the basis for verifying code, including but not limited to, requirements analysis, and test While these techniques are excellent strategic recommendations to improve quality in general, they cannot prevent careless mistakes. The same caveat would apply to recommendations such as (1) provide better training for the coders, (2) use line-of-code coverage test cases, or (3) use path-coverage test cases. Using static analysis to identify dead code could have flagged this defect, but not all such mistakes result in truly dead code. It is better to find and remove these problems during a personal review, an informal peer code review, or a formal peer code inspection. Case Study: Heartbleed Vulnerability The Heartbleed vulnerability occurred in the OpenSSL "assert" function, which is the initiator of a heartbeat protocol to verify that the OpenSSL server is live. OpenSSL is an open-source implementation of the secure socket layer (SSL) and transport layer security (TLS) protocols used for securing web communications. The assert function sends a request with two parameters, a content string (payload) and an integer value that represents the length of the payload it is sending. If the OpenSSL connection is available the expected response is a return of the content string for the length specified. The protocol assumes that the requested length of the payload returned is less than 65,535 and less than or equal to the payload length, but those assumptions are never verified by the responding function. A consequence of a violation of either of these limitations is that the request can trigger a data leak. Rather than a buffer overflow, this leak is what is called an over-read. The security risk is that the additional data retrieved from the server’s memory could contain passwords, user identification information, and other confidential information. The defect appears to have been accidentally introduced by an update in December 2011. OpenSSL is a widely used and free tool. At the disclosure of Heartbleed, approximately 500,000 of the internet’s secure web servers certified by trusted authorities were believed to be vulnerable to the attack. The new OpenSSL version repaired this vulnerability by including a bounds check to ensure that the payload length specified by a developer is not longer than the data that is actually sent. Unfortunately, that check is only the start of an implemented correction because elimination of the vulnerability requires the 500,000 users of this software to upgrade to the new version. In addition, because this problem is related to security certificates, protecting systems from attacks that exploit the Heartbleed vulnerability requires that companies revoke old SSL certificates, generate new keys, and issue new certificates.IEEE’s Security and Privacy article Heartbleed 101 provides an excellent summary of why this vulnerability was not found sooner, even with the use of static analysis tools. The designer of each static analysis tool has to make trade-offs among the time required for the analysis, the expert help required to support the tool’s analysis, and the completeness of the analysis. Most static analysis tools use heuristics to identify likely vulnerabilities and to allow completion of their analysis within useful times. The article goes on to explain that while static analysis tools can be effective for finding some types of vulnerabilities, the complexity of OpenSSL (including multiple levels of indirection and other issues) exceeded the capabilities of existing tools to find this type of vulnerability. While the OpenSSL program is complex, the cause of the vulnerability is simple. The software never verified the design assumption that the length of the content to be returned to the caller was less than or equal to the length of the payload sent. Verifying that the input data meets its specifications is a standard activity performed for quality, not just for security. The associated software errors that led to the "goto fail" and Heartbleed vulnerabilities should have been identified during development. These two examples highlight the fact that quality practices, such as inspections and reviews of engineering decisions, are essential for security. For example, testing and code analyzers, must be augmented by disciplined quality approaches. Improve Security with Quality Our research suggests that implementing systems with effective operational security requires incorporating both quality and security considerations throughout the lifecycle. Predicting effective operational security requires quality evidence and security expert analysis at each step in the lifecycle. If defects are measured, it follows that from 1 percent to 5 percent of them should be considered vulnerabilities. Likewise, when security vulnerabilities are measured, then code quality can be estimated by considering them to be 1 percent to 5 percent of expected defects. Further evaluation of systems is needed to see if the patterns suggested by our analysis continue to hold. We explored several options to expand our sample size, but found limited data about defects and vulnerabilities assembled in a form that could be readily analyzed. This analysis must be done for each unique version of a software product. At this time, evaluation of each software product requires careful review of each software change and reported vulnerability. The review not only matches up defects with source code versions, but also reviews each vulnerability reported against the product suite to identify defects specific to the selected product version by parsing available description information and identifying operational results for the same source code. Collection of data about each product in a form that supports automation of this analysis would greatly speed confirmation. We welcome your feedback on our research. Please leave comments below. Additional Resources To read our technical report, Predicting Software Assurance Using Quality and Reliability Measures, please click here.

Tom Simcox - Content Designer Last week I was lucky enough to attend Leeds GovJam - part of the Global GovJam taking place in over 30 cities around the world. It brought together people interested in service design from across the public sector. It was my first GovJam, and unlike anything I’ve experienced before. Doing not talking As a content designer and civil servant, I think we can spend too long talking, justifying or defending our position and ideas. It can get in the way of delivery. At GovJam, things are different. Each GovJam event around the world creates products based on the same secret theme. From the moment it was unveiled, jammers were thinking about problems, quickly building solutions and testing with real people within hours. Building teams and exploring the theme The theme was surprisingly vague. But the simple image of what appeared to be a lock allowed us to explore ideas and be creative, without constraints. We voted on the ideas that interested us and formed teams. My team wanted to explore ways of helping people break out of, or ’unlock’ poverty. Getting out of the building - discovering the problem We hit the streets of Leeds. We talked to people about their attitudes towards food, the choices they made, and about barriers to eating healthily. We’d thought that access to cheap, fresh, nutritious ingredients was holding people back from making better choices. But our research gave us a better understanding. We discovered that people needed to be shown an alternative. Not just told to make better choices. As a result, the idea of the ‘Cook Truck’ was born. The service would take food education into the community. It would show and teach people an alternative. Removing the fear of failing - build, learn, repeat We built a prototype. We didn’t sit down to debate. We didn't talk ourselves out of anything, or question whether it would work. We weren’t afraid of failing. We worked fearlessly and creatively. We made use of whatever we could lay our hands on (Lego, plasticine, lots of cardboard) and just built it. And when we’d built it we didn’t stop there. We showed it. We watched others experience it (we acted out each other’s personas) and we learned from it. And then we improved it. Designing in the open - sharing our prototypes Teams shared ideas with other teams and this collaboration was really important. We even joined the Athens GovJam on Skype to see what they’d been doing. At the end of the jam we published our prototypes on the Global GovJam website. Anyone can look at these and build on the work we’ve started. At the final show and tell, we received a surprise visit from Tom Riordan, CEO of Leeds City Council. Tom spoke about the need to open up service design in the public sector. Budget cuts mean councils need to work smarter, and they need our help. Before I left the GovJam I was thrilled to learn that the council had been looking at an idea similar to the Cook Truck. They’d heard about what we’d designed and were keen to talk to us! What I’m taking back to DWP I was amazed at how much we created in just 48 hours. GovJam is all about trying new ways of designing and collaborating with others. We did it while having lots of fun! But we did more than that. We discovered unmet needs of the citizens of Leeds. We started designing public services to meet them. We did it by being fearless, by going out and talking to people to find out how we could make their lives better. I’m new to my design role, so for me, GovJam was the perfect opportunity to learn by doing. I experienced agile design at its best, and saw what can be done by people passionate about change. If we put users at the heart of what we’re building, aren’t afraid to fail, and share what we’re doing, we can make a real difference. (Thanks to Lisa Jeffrey for use of some of her photos - visit Flickr to see more of Lisa’s images from Leeds GovJam)

By William WoodSenior Member of the Technical StaffSoftware Solutions Division Legacy systems represent a massive operations and maintenance (O&M) expense. According to a recent study, 75 percent of North American and European enterprise information technology (IT) budgets are expended on ongoing O&M, leaving a mere 25 percent for new investments. Another study found nearly three quarters of the U.S. federal IT budget is spent supporting legacy systems. For decades, the Department of Defense (DoD) has been attempting to modernize about 2,200 business systems, which are supported by billions of dollars in annual expenditures that are intended to support business functions and operations. Many of these legacy systems were built decades ago using technologies available at the time and have been operating successfully for many years. Unfortunately, these systems were built with components that are becoming obsolete and have accompanying high-licensing costs for commercial off-the-shelf (COTS) components, awkward user interfaces, and business processes that evolved based on expediency rather than optimality. In addition, new software engineers familiar with current technology are unfamiliar with the domain, and documentation is scarce and outdated. Other problematic factors include business rules that are embedded in code written in obsolete languages using obsolete data structures and the fact that the cadre of aging domain experts maintaining legacy systems are unfamiliar with newer technologies. This blog post provides a case study of a modernization effort conducted for a federal agency by SEI researchers on such a large-scale, legacy IT system.  The Horseshoe Model A general approach to modernization is based on the horseshoe model shown in the figure below.  The basic principle of the horseshoe model is that the transformations are more costly at the higher levels. Merely changing technologies (e.g., moving from one commercial data base management system (DBMS) to another) is done at the technical level, and there are many commercial tools to assist in this transformation. If there are significant changes to business processes, roles, and user interfaces, the modernization will take a greater effort, as measured in terms of cost, performance, and risk. Recently a number of SEI engineers were tasked to plan a proposed modernization of a large-scale, legacy IT system (main frame, hierarchical database, COBOL, job control language [JCL], green screens, spaghetti code) to a modern architectural framework (commodity hardware, Linux OS, JAVA, relational database, well defined framework for distributed processing, 4-layer applications). The modern architectural framework was defined by a technical reference architecture (TRA) and a common platform infrastructure (CPI) satisfying the TRA. The TRA included many constraints on the development, operation, and support for both the data structures and the applications. The CPI included the computing device, software language and associated tools, operating system, relational database and associated tools and services, web services, and other services. The CPI would also be used by other modernization projects within the federal agency. We defined a plan with four phases but were involved directly with only the first two phases. The first phase consisted of only SEI engineers. An SEI engineer led the second phase, which included five customer representatives. The phases are described in detail below: First Phase. The team conducted an analysis of a number (14) of responses to a request for information (RFI) by commercial IT contractors to migrate the legacy software to the modern framework. This migration involved making as few changes as possible to get it to the new TRA/CPI; move it to the CPI in an infrastructure minimally satisfying the TRA; use the CPI, including a specific COTS relational database with associated tools; leave the COBOL and green screens; and make as few changes as possible to the spaghetti code, legacy applications, and data structures’ architecture. This approach was designated as a "lift and shift" (L&S). The RFI mentioned six issues to be addressed.  a. migration from hierarchical database to a relational database b. migration of application code c. integration and testing of the modernized system d. maintenance of the application code e. acquisition and contracting constraints f. past performance These responses form the top six rows in the table shown below.  The team also considered whether the proposals included sufficient technical detail to resolve these six issues and these considerations formed the next six rows of the table. We included two more issues that form the bottom two rows.  Each proposal also included an expected cost and schedule. The table below shows how the proposals (A through N) were judged to have addressed the issues. The results for each response are shown in the adjacent table. The green color indicates that an issue was addressed in a satisfactory manner, yellow that it was addressed somewhat, and red that it was not addressed.  The cost and schedule numbers were not included in the table, but the costs varied by a factor of two. The schedules were all about the same. The proposers A, B, and C are almost completely "green" indicating that they both understood the issues and knew how to tackle them technically. The proposers F, G, H, and I clearly understood the issues, but did not describe the technology they would use to resolve them in sufficient detail. The proposals L, M, and N more or less indicated "we are good at this sort of thing" but failed to address the issues and gave no technical detail.  The proposers D, E, J, and K were rather vague on the issues and technology. The analysis convinced the team that most responders considered a lift and shift to be feasible, so we proceeded on that basis. The team recommended that an effort be started immediately to better understand the structure of the legacy software and data structures and how the different classes of users interacted with these structures. We called this process discovery and analysis (D&A) and it consisted of performing the following activities: review the documentation (it was sparse and dated); analyze the code and data structures for connectivity relationships using commercial tools, such as Lattix; determine how business processes and user roles accessed the code and data structures; and collect the real-time traffic patterns already being measured on the system. In addition, the programmers knew that there were existing issues with the system, such as dead code, dead tables, dead attributes, redundant copies of attributes, and cloned and owned procedures. These issues can also be identified with the same tools. Analysis of the relationships between software elements (data files, programs) can then be used to form clusters of data files, programs, and users with minimal interactions. These clusters would then form the basis for a phased lift and shift effort. Other important considerations that require management involvement included the following  Can the authoritative data be split between the legacy system and the modern system? Is there a need to failback to the legacy system when a failure occurs in the modern system?  How much of the transformation can be accomplished using tools, and how much manual intervention is required? It was recommended that a small task to be initiated to transform one of the clusters. In addition, some constraints must be placed on any solution including ensuring that the most critical users are not inconvenienced by the requirement that they log in to and switch between multiple systems to accomplish their work avoiding transactions that will require atomicity across the legacy and modern systems However, a D&A could not be initiated without performing a return on investment (ROI) analysis for the whole migration. This ROI analysis was done by a customer expert together with SEI input. Second Phase. Based on the RFI responses (but not the D&A results, which had not yet been conducted) the team proposed six alternative approaches ranging from doing nothing (as a baseline) to completely re-engineering the system. The team viewed the lift and shift as a transformation at the technical architectural level, a tool-based transformation of the applications and replacement of green screens as occurring at an applications architectural level, and a re-engineering effort as being at the business architecture level.  The team developed a set of 23 factors to compare between alternatives. These factors can be found in the Additional Resources sectionat the end of this blog. The factors were initially based on the team’s architectural knowledge and experience and the knowledge of the software engineers on the team tasked with sustaining the legacy system. The factors were then reviewed against those from the OMB Exhibit 300, which the Office of Management and Budget (OMB) uses (along with other factors) to analyze proposed and ongoing programs. This comparison led to the introduction of a few more factors. The factors were grouped by cost, performance, and risk. We developed a simple set of measures (1, 3, 5 and 8), which were applied to each factor with 1 being worst and 8 being best. Because the factors were unevenly split among the groups, we also normalized the measures over the groups. The ranking of the alternatives is shown in the graphic below. As the chart above indicates, higher numbers were the best options The decision makers decided on a hybrid approach i. Conduct a D&Aii. Lift and shiftiii. Transformiv. Re-engineer In addition, the team needed to meet with users to discover which business processes were cumbersome and time consuming and should be re-engineered. Third Phase. Build an end-state architecture on top of the modern architectural framework, defining applications as services, and data structures and mapping them to legacy applications and data files transformation method to be used legacy COTS tools and the CPI COTS tools (developmental, test, and operational). The external interfaces In addition, the end-state architecture must account for use case sequences and business case sequences. The end-state architecture must be given a thorough review by stakeholders, and a registry of technical risks must be established based on the evaluation and subsequent upgrades and decisions. Fourth Phase. Build a multi-phase architectural roadmap that allows deviations from the framework constraints at the intermediate phases. This roadmap should start with some pilot projects to better understand the difficulties in all approaches: lift & shift, transform and re-engineer. The phasing should take into account the following: the progress in the development of the CPI and its impact on the migration choosing low-hanging fruit early organizational boundaries that will impact the development splitting of authoritative data process to be followed: move data first, move code first, move a grouping of code/data efficacy of tool support Each phase of the target architecture must have views showing the following:  which legacy elements are to be replaced and a mapping of where they are to be introduced in this phase what end-state elements are to be introduced cross coupling between systems additions and removals of interfaces between the legacy system and the PTA how each class of user will interact with the mixed system As progress is made on a phase, especially the first or pilot phase, more and more information will become available to modify the roadmap, both within the current phase being implemented and the next phase. For example if a new CPI capability is unstable, and this impacts the move of a particular application, then re-schedule the move until the CPI stabilizes.  Wrapping Up and Looking Ahead A basis for modernizing the system was determined, and criteria were created for choosing between alternative approaches. A way of clustering the software elements was described, and the basis for a target architecture was defined. The factors to be used in a modernization effort will depend somewhat on the attributes of the legacy system the attributes of the TRA and CPI business drivers of the program office funding the migration.  The factors developed for this migration effort can serve as a basis for developing factors for another migration effort. Building a roadmap, however, relies on choosing the order of moving the clusters, and the milestones when some class of users will be cutover from the legacy system to the modernized one. Building a roadmap is a complex optimization problem, in which many issues must be resolved and decisions made at different levels:  For example, whether or not to have authoritative data split between the two systems is a high-level decision made at the program management level.  If the split is allowed, then at specific milestones some users can start using the modern system, while others still use the legacy system, demonstrating progress against a schedule. If the split is not allowed, then progress can only be measured by progress against tests conducted at milestones.  The efficacy of tool support can only be determined by using the tools to migrate a portion of a cluster and gaining tool familiarity; then writing a lessons-learned report to provide guidelines to later engineers on how best to use the tools. The decisions made to order the migration of clusters is an optimization problem with a large search space. First, there is the problem of defining the utility function to minimize, expressing the problem mathematically and defining the feasible solution space where all hard constraints are satisfied. A good example of a hard constraint might be that critical users conducting critical capabilities must always use either the legacy system or the modernized system. Some of these issues are being addressed in a new research effort underway at the SEI.  We welcome your feedback on this research. Please leave comments below.  Additional Resources To view the presentation Architectural Insights into Planning a Legacy Systems Migration, which I co-presented with Michael Gagliardi and Philip Bianco at the 2014 TSP Symposium, please click here. 

It’s always interesting to hear how the people management profession differs worldwide. Workplace laws, business culture and social mores vary country by country. What works in one nation, industry or even company, may not work in another because when it comes to managing people—the most complex but critical aspect of business—there is no one-size-fits all approach. Still, some keys to HR’s success are universal. This was our thinking at SHRM when we developed the SHRM Competency Model. Through extensive global research, we set out to codify...

The Secure Communications team recently passed their GDS Alpha review to move into Beta. Product Owner Rachel Woods reflects on the review and offers some insider knowledge on the experience. Rachel Woods - Product Owner There's a phrase often used in the legal world: "if you come to equity, come with clean hands". As a law student I used to imagine standing in court, the high Victorian dark wood judicial bench afore me with a stony-faced judge peering over his half-moon glasses and me all squeaky-voiced and squirming with red-raw, much washed hands, explaining, "They're sparkling, m'lud." Fast forward more than ten years later and this was the first image that sprang to mind when someone uttered the words ‘Digital by Default Alpha Review’. As is usually the case, the fear is of the unknown. I want to share some of my experience in the hope I can prevent you from having similar traumatic flashbacks! It's not scary So firstly, it's not something to fear. Not if you know your project. The panel want to talk to someone who has responsibility and accountability for and within the project - they don't want a briefed figurehead. I've been on the Secure Comms project from the beginning, living and breathing our user needs with the rest of the team. I have the pleasure (and the burden) of understanding the history of the project, what decisions I've made and how they’ve moulded and shaped the project into what it is today. I know exactly why things are the way they are and I can explain that. Don't underestimate the importance of that. It's a conversation One of the things that worked really well at out our review was our focus on showing the thing. I spent about ten minutes giving an overview of the project, just enough to explain what our overarching user need was and how our service would meet it. It really was quick - for the actual description of the service I used our elevator pitch. Why tell them about it when they are about to see it in action? Then, in the best double act since Torville and Dean, our Lead Developer, Kevin Adams, and User Researcher, Chris Beardsell, walked the panel through the prototype. Liz Whitefield (Delivery Manager), Rachel Woods (Product Owner), Chris Beardsell (User Researcher) and Kevin Adams (Lead Developer) pictured straight after their GDS Alpha review As they did they talked about what users had said, what changes had been made and what work we had done with partners to get the solution right. By the time we had finished the demo we had actually covered most of the points within the review criteria. The criteria are really helpful in ensuring you've thought about all the aspects of your project. I still came out thinking, "I forgot to mention..." But the criteria really help you to have a framework for ensuring you cover all the good stuff you and your team have done. It helps you as much as it helps the panel. It's an opportunity Lastly you can't underestimate the value of the independent critical eye. We’re still learning on our team and the review is a great opportunity to get some insight on how you’ve been working as well as what you’ve been working on and get some recommendations. I'm so proud of what the Secure Communications team has achieved in passing the review and moving into Beta. It was exciting to show our ideas and work to a group of people who really got where we were coming from. It’s a good feeling, so my last - and possibly most important - piece of advice is ‘enjoy it’!