There are many good employee engagement surveys out there (SHRM has one, for example). Most of them ask about pay, benefits, workplace culture and other external conditions that help people become engaged in their work. In my new book, Triggers: Becoming the Person You Want to BeTriggers: Becoming the Person You Want to Be, (with Mark Reiter, Crown, 2015), I suggest that we also ask another kind of question—about what employees can do to engage themselves. "Active questions," as I call them, are...

Fiona Speirs - Head of User Research - DWP I’m Fiona Speirs, the Head of User Research in DWP. My aim is to embed the need to design all our digital services around our users. To do this, I’m building a User Research team in DWP. The User Research team generates insight to help enable the Department to make better decisions which take into account the needs and behaviours of our service users, while delivering the desired policy and service outcomes. The role of the user researcher Much has been said about designing digital services around user needs. To do this, we need to really understand our users and to build a rich picture of their attitudes and needs - backed by sound analysis, and quantitative and qualitative evidence. I’m looking for user researchers who can plan and design research programmes, generate new user evidence in creative and innovative ways, and weigh up evidence from different (often conflicting) sources. We’re involved in over a dozen live projects. You’ll work with the digital project teams to generate the feedback and insight that will help to build a clear picture of their users, and deliver solutions that they need. In essence, user researchers get people to focus on creating online services that meet real users’ needs and which are simple and intuitive to use. As a user researcher, you’ll be a natural collaborator, working with really talented designers, developers and analysts in agile teams. You’ll be excellent at managing senior stakeholders, engaging the right people in research findings, watching live research sessions, and increasing understanding of user needs. The user research team We’ve already got some excellent user researchers in DWP. To meet the challenge of designing better digital services in government, we want to grow our skills and the user research team. People increasingly expect to access services digitally at a time that suits them. To meet this challenge, we are transforming the way we operate to design automated, efficient services in an agile way that puts the users’ needs first. User research is at the heart of this. User research vacancies We have vacancies for 5 User Researchers and 2 Senior User Researchers.

By Todd Waits, Project Lead Cyber Security Solutions Directorate  This post is the latest installment in a series aimed at helping organizations adopt DevOps. In the post What is DevOps?, we define one of the benefits of DevOps as "collaboration between project team roles." Conversations between team members and the platform on which communication occurs can have a profound impact on that collaboration. Poor or unused communication tools lead to miscommunication, redundant efforts, or faulty implementations. On the other hand, communication tools integrated with the development and operational infrastructures can speed up the delivery of business value to the organization. How a team structures the very infrastructure on which they communicate will directly impact their effectiveness as a team. ChatOps is a branch of DevOps focusing on the communications within the DevOps team. The ChatOps space encompasses the communication and collaboration tools within the team: notifications, chat servers, bots, issue tracking systems, etc.  In a recent blog post, Eric Sigler writes that ChatOps, a term that originated at GitHub, is all about conversation-driven development. "By bringing your tools into your conversations and using a chat bot modified to work with key plugins and scripts, teams can automate tasks and collaborate, working better, cheaper and faster," Sigler writes.  Most teams have some level of collaboration on a chat server. The chat server can act as a town square for the broader development teams, facilitating cohesion and providing a space for team members to do everything from blowing off steam with gif parties to discussing potential solutions to real problems. We want all team members on the chat server. In our team, to filter out the noise of a general chat room, we also create dedicated rooms for each project where the project team members can talk about project details that do not involve the broader team. More than a simple medium, the chat server can be made intelligent, passing notifications from the development infrastructure to the team, and executing commands back to the infrastructure from the team. Our chat server is the hub for notifications and quick interactions with our development infrastructure. Project teams are notified through the chat server (among other methods) of any build status they care to follow: build failures, build success, timeouts, etc. Chatbots are autonomous programs that operate and live on the chat server. We have two bots that operate on our chat servers built using the jabber-bot Ruby library. Other chat bot options are python-jabberbot, Lync chat bots, and GitHub's Hubot. Chat bots can be as simple or complex as the team needs them to be. Our proof-of-concept "DevBot" started small by allowing us to type in commands to dynamically get a command line tip from the website commandlinefu.com. While initially frivolous, the development of this bot allowed our team to develop rapport and trust with one another. As team members realized the power of what the bot could do, it quickly morphed into a utility that would allow us to log work activities to a database, or query the status of the build server. Our bots are written in Ruby and allow us to quickly add commands and integrate with external and internal resources as allowed by policy. After the success of our first bot, we created a second bot that allowed direct interaction with our issue tracker from the chat window. We could create cases, get a list of active cases by user, and resolve or open new cases from the chat window. Essentially, this gave us the power to contextually create cases based on the current conversation without leaving the chat window. For example, team members are having conversations with someone and realize they need to adjust a task estimate, they do so from the chat window. They do not need to load a separate tool. They send a chat message to the bot to update an estimate of a particular task. The code below, for example, would look for the word "estimate," a case number, and the estimate value in a chat message. When the bot sees the appropriate message, it executes a command adding the estimate to the case number supplied with the credentials of the user chatting with the bot. def estimate(user, msg)     casenum = msg[0]     estimate = msg[1]   fb = fb_session(user)   fb.command(:edit, :ixBug => casenum, :hrsCurrEst => estimate) end bot.add_command(     :syntax => 'estiamte <CASENUM> <HOURS>',     :description => 'Adds entered hours as estimate to specified case.',     :is_public => true,     :regex => /^estimate\s+(.+)\s+(.+)$/   ) do |from, msg|     #user = prepuser(from)     estimate(from, msg)     puts "#{from} estimated #{msg[1]} hours on case #{msg[0]}"     "#{from} estimated #{msg[1]} hours on case #{msg[0]}" end The chat bot can be an excellent way to on-board new members of a DevOps team. Coding and implementing new features for the bots allows new team members the opportunity to interact with various systems at a deep level very quickly. The team members get a feel for how the infrastructure and team work together, and they do it on a relatively low-risk project. By building functionality onto a chat bot, team members learn the issue tracker, version control, chat server, and build server, to name just a few. Veteran team members immediately see how the new team member adds value to the team. Finding ways to make communication more effective and actionable will go a long way to extending DevOps capabilities of a team.  Every two weeks, the SEI will publish a new blog post offering guidelines and practical advice to organizations seeking to adopt DevOps in practice. We welcome your feedback on this series, as well as suggestions for future content. Please leave feedback in the comments section below. Additional Resources To listen to the podcast, DevOps—Transform Development and Operations for Fast, Secure Deployments, featuring Gene Kim and Julia Allen, please visit http://url.sei.cmu.edu/js. To read all the installments in our DevOps series, please click here or on the individual posts below.  An Introduction to DevOps A Generalized Model for Automated DevOps A New Weekly Blog Series to Help Organizations Adopt & Implement DevOps DevOps Enhances Software Quality DevOps and Agile What is DevOps? Security in Continuous Integration DevOps Technologies: Vagrant DevOps and Your Organization: Where to Begin DevOps and Docker Development with Docker Continuous Integration in DevOps  

Natalie Rhodes - User Researcher at DWP Ben Holliday recently posted a Digital Academy blog called ‘Understanding the problem’. It’s an introduction to understanding the problem when you’re working with a team on a service in discovery. As well as developing a research plan to get started, he explained the importance of finding ways to record (and keep a record of) everything you’re learning. In this post, I’ll explain how to do this in more detail.   Before you start Before you start, consider working in a pair. You’ll find it’s much easier if one person is asking questions and someone else is concentrating on capturing feedback. Capture observations using sticky notes One of the best tools you can use for capturing observations is the sticky note. They’re great for grouping observations later when you get to your analysis. Use these 5 tips when writing sticky notes - if you don’t want to use sticky notes the same principles apply to keeping research notes. Write one observation per sticky note The more information each sticky note holds, the harder it is to understand and organise later. You might miss important information by writing more than one observation or adding too much detail to a single sticky note. Capture the thing, not your interpretation of the thing It’s important to capture what’s happening, not what you think it means. Interpretation comes later. Right now, you want an accurate record of what people have said and done. Make sure you know who said what Label your observations so you can identify the people taking part in your research. An easy approach is to give each person you speak to a number, then label each observation you write down with the corresponding number. Don’t use jargon, acronyms, or shorthand Write observations so other people can understand them. You shouldn’t have to explain these to people if they’re clear and concise. Make sure other people can read your handwriting - using uppercase for legibility is a good idea. Personal data Finally, don’t capture personal information, which could allow someone to be identified. Names, national insurance numbers, addresses, etc., shouldn’t be recorded. You need to think: "If I lost this, could someone identify who this person is?" Making sense of your observations Once you have got all your interviews done, the next step is to look for common themes in your observations. A good technique for this is affinity sorting. As many people from the team as possible should be involved in the analysis stage. Involve everyone who’s been directly involved with your research. Affinity sorting You’ll need a big space, preferably a wall to post up your observations. Using affinity sorting, organise each sticky note into related groups. This isn’t an exact science so don’t feel that once you have put a sticky somewhere you have to leave it there - that’s the reason we use sticky notes. To summarise: Read your first sticky note and stick it up on the wall Read the second sticky note. Is it related to the first observation or is it about something different? If it’s different, then stick somewhere else on the wall. If it’s the same, group this with your first sticky note. If you’re not sure put it somewhere close by. Read out each observation, if this helps. Make sure everyone involved in the room is clear about what happened when the research took place. Keep going until you’ve sorted through all your observations. This can take as long as a few hours, depending on how many observations you’ve captured during your research. You should be able to see themes emerging from your groupings. At this stage label your groups as insights - the interpretation of what we think each group of data actually means. Make your findings visible  It’s important to make your research as visible as possible to your team. Get a wall, a board, or a window - anywhere you can display user needs or insights from research. There’s no hard and fast rule about how to write up your research, but concentrate on communicating key themes or insights to your team. Keep records of what you’ve learnt each time you do research so you can go back to it and review it, and compare it with all the new things you are learning.

By William R. NicholsSenior Member of the Technical StaffSoftware Solutions Division As software continues to grow in size and complexity, software programmers continue to make mistakes during development. These mistakes can result in defects in software products and can cause severe damage when the software goes into production. Through the Personal Software Process (PSP), the Carnegie Mellon University Software Engineering Institute has long advocated incorporating discipline and quantitative measurement into the software engineer’s initial development work to detect and eliminate defects before the product is delivered to users. This blog post presents an approach for incorporating formal methods with PSP, in particular, Verified Design by Contract, to reduce the number of defects earlier in the software development lifecycle while preserving or improving productivity. Formal Methods and PSP Created by Watts Humphrey, PSP incorporates process discipline and quantitative management into the software engineer’s individual development work. PSP promotes the exercise of careful procedures during all stages of development, with the aim of increasing the individual’s productivity and achieving high-quality final products. PSP emphasizes the measurement of software development work using simple measures, such as source lines of code (SLOC). These measures allow software developers to evaluate the effectiveness of their approach and answer the following questions: How is the process working? Is my process effective? If it is not effective, and I need to make a change, what should I change? What were the impacts of that change? In essence, PSP is a scientific approach that software developers can use to evaluate their effectiveness. The Team Software Process (TSP) applies the same measurement principles in a team environment, and has been found by Capers Jones to produce best-in-class productivity and early defect removal. Research conducted by my collaborator in this research, Diego Vallespir, director of the Postgraduate Center for Professional Development, University of the Republic Uruguay (CPAP), found that removing defects in the unit testing phase can still be expensive, costing five to seven times more than if they were removed in earlier phases of PSP. Dr. Vallespir’s research also found that 38 percent of the injected defects are still present at unit testing. In addition to myself, a team of researchers led by Vallespir along with doctoral student Silvana Moreno (Universidad de la República), and professor Álvaro Tasistro (Universidad ORT Uruguay)—theorized that opportunities existed for improvement in the early detection of defects using TSP. Our team felt that the answer might lie in formal methods. Formal methods use the same methodological strategy as PSP: emphasizing care in development procedures, as opposed to relying on testing and debugging. They can also rigorously establish that the programs produced satisfy their specifications. Formal methods hold fast to the tenet that programs should be proven to satisfy their specifications. Proof is the mathematical activity of arriving at knowledge deductively, starting with postulated, supposed, or self-evident principles and performing successive inferences, each of which extracts a conclusion out of previously arrived-at premises. Verified Design by Contract (VDbC) is a technique devised and patented by Bertrand Meyer for designing components of a software system by establishing their conditions of use and behavior requirements in a formal language. With VDbC, software developers metaphorically set up a contract to define certain expectations of their software. In particular, VDbC has been proposed in the framework of object-oriented design (and specifically in the language Eiffel), and, therefore, the software components to be considered are usually classes. The corresponding specifications are pre- and post-conditions to methods, respectively establishing their terms of use and corresponding outcomes, as well as invariants of the class (i.e., conditions to be verified by every visible state of an instance of the class). In the original VDbC proposal, all specifications were written in Eiffel and are computable (i.e., they are checkable at runtime). VDbC involves software developers, essentially, setting up a contract to define certain expectations of how the software will behave. VDbC is compatible with test-driven development, which requires defining test cases prior to developing code. This aspect of VDbC is also compatible with PSP, which has always emphasized developing some test cases early on in the software development lifecycle as part of the design process. It is important to note that developing test cases is not the sum of design, but rather the development of test cases is an aspect of design. Our Approach Designs are most effective if you have some way of verifying the design formally. Different types of design representations, for example, pseudo-code to represent logic or module decompositions to represent structure, support different levels formality. The leverage of additional formality comes from the rigor with which the design can be verified. Our approach involves using the framework and instrumentation of PSP to evaluate how the designer, in this case VDbC, affects the results. The PSP script supports consistency and measurement by defining the logical sequence of steps (for example, plan, design, code, review code, unit test) that must be followed when building code. With VDbC, we added specific phases, activities, and outcome criteria to the PSP script  (For more information about PSP scripts, please see Table 2 in the following SEI technical report.) to show that we can measure how much effort has gone into the phase set up the contract requirements conduct verification check that the design is complete and that the design is correct As explained in detail in our technical report on this approach, PSP and VDbC: An Adaption of the PSP that Incorporates Verified Design by Contract, our combined approach resulted in new phases of software development and modified other phases already present in PSP. The resulting adaptation of PSP, hereafter referred to as PSPVDbC, incorporates new phases, modifies others, and adds new scripts and checklists to the infrastructure. Specifically, the phases of formal specification, formal specification review, formal specification compile, test case construct, pseudo code, pseudo code review, and proof are added.In the remainder of this post, we will present the phases of our combined approach, indicating the activities that are to be performed and the modifications introduced in the scripts with respect to the original PSP. Planning. The activities in this phase of PSPVDbC are the same as in ordinary PSP. For example, Program Requirements ensure a precise understanding of every requirement. Size Estimate involves carrying out a conceptual design (i.e., producing a module [class] structure). Resource Estimate estimates the amount of time needed to develop the program. For this, the PROBE method (This SEI technical report includes a description of the PROBE method). is used, which employs historical records and linear regression for producing the new estimation and for measuring and improving the precision of the estimations. Task and Schedule Planning is for long-term projects. These are subdivided into tasks, and the time is estimated for each task. This planning is unchanged in PSPVDbC. Defect Estimate Base is for estimating the number of defects injected and removed at each phase. Historical records and the estimated size of the program are utilized for performing this estimation. In PSPVDbC new records are needed to estimate the defects removed and injected at each new phase. Finally, the planning script in PSPVDbC is the same as in PSP, given that the corresponding activities are unchanged. Design. This phase defines the data structures of the program as well as its classes and methods, interfaces, components, and interactions among all of them. In PSPVDbC, the elaboration of the pseudo code is postponed until the formal specification is available for each method. Design Review. This phase is the same as ordinary PSP and uses its development script describing the steps to follow in the review. A sample development script is included in Table 13 of our technical report. Test Case Construction. We want to investigate the cost effectiveness of test case construction and unit testing when formal methods are used. Problems with unit test include the cost of test case construction, maintenance of test cases, number of test cases required, and a failure to achieve comprehensive test coverage. We want to determine if it is practical to reduce or eliminate categories of test in the unit test phase when using these formal methods. To answer this, the following must be known: cost of test case construction cost of unit test construction defect density entering into unit test yield of the unit test phase types of defects entering and escaping unit test Formal Specification. This phase must be performed after the design review. The reason for this is that reviews are highly effective in detecting defects injected during design and need to be discovered as early as possible. In this phase, we begin to use the development environment that supports VDbC. Two activities are carried out in this phase: Construction consists of preparing the environment and defining within it each class with its method headers. Specification, in which we write down in the carrier language the pre- and post-conditions of each method as well as the class invariant. Note that, within the present approach, the use of formal methods begins once the design has been completed. It consists of the formal specification of the produced design and the formal proof that the final code is correct with respect to this specification. Formal Specification Review. Using a formal language for specifying conditions is not a trivial task, and both syntactic and semantic defects can be injected. To avoid the propagation of these errors to further stages, and the resulting increase in the cost of correction, we propose a phase called formal specification review. The script that corresponds to this phase contains the following activities: In the review activity, the sentences of the specification are inspected using a checklist. In the correction review, all defects detected during the review are removed. In checking, the corrections are reviewed to verify their adequacy. Formal Specification Compile. Any development tool supporting VDbC will be able to compile the formal specification. Since this allows an early detection of errors, we consider it valuable to explicitly introduce this phase into PSPVDbC. In particular, it is worthwhile to detect all possible errors in the formal specifications before any coding is carried out. A further reason to isolate the compilation of the formal specification is to allow the time spent in this specific activity to be recorded. Pseudo Code. The pseudo code phase allows software developers to understand and structure the solution to the specified problem just before coding. Describing the intent of the design in a programing neutral language helps the developer bridge the more abstract design to the concrete implementation. The documentation later supports peer review that the code actually implements the design. Thus, the pseudo code of each class method defined in the logic template is written down. Our approach advocates that the pseudo code be produced after the compilation of the specification in order for the specification to serve as a well understood starting point for design elaboration in pseudo code. Writing down the pseudo code just before coding allows us to follow a well-defined process in which the output of each stage is taken as input to the next one. Pseudo Code Review. A checklist is used for guiding activity in this phase. The activity pseudo code review is added to the development script. The pseudo code review script is proposed for use in this activity. An example of the script follows: Produce a pseudo code to meet the design. Record the design logic specification templates. Record defects in the defect recording log. Record time in the time recording log. Code, Code Review, and Code Compile. Just as in PSP, these phases consist of translating the design into a specific programming language, revising the code, and compiling it. The description of these activities in the PSPVDbC development script is the same as in the PSP development script. Proof. An addition to PSPVDbC, this phase provides evidence of the correctness of the code with respect to the formal specification (i.e., its formal proof). A computerized verifying tool is used that derives proof obligations and helps to carry out the proofs themselves. Unit Test. This phase is the same as in PSP. This phase is relevant because it detects mismatches with respect to the original, informal requirements of the program. These defects can arise at several points during the development, particularly as conceptual or semantic errors of the formal specifications. The test cases to be executed must therefore be designed right after the requirements are established (i.e., during the phase test case construct) as indicated above. Post Mortem. This is the same as in ordinary PSP, and its description in the PSPVDbC development script. However, several modifications must be made to the infrastructure supporting the new process. For example, all new phases must be included in the support tool to keep track of the time spent at each phase, as well as to record defects injected, detected, and removed at each phase. Our intention in this research was to present the changes in the process to incorporate VbDC. The adaptation of the supporting tools, scripts, and training courses is a matter for future work. Conclusions and Future Work By definition, in Design by Contract (and therefore also in PSPVDbC) the specification language is seamlessly integrated with the programming language, either because they coincide or because the specification language is a smooth extension of the programming language. As a consequence, the conditions making up the various specifications are Boolean expressions that are simple to learn and understand. We believe that this makes the approach easier to learn and use than the ones that have previously been explored. Nonetheless, the main difficulty associated with the method resides in developing a competence in carrying out the formal proofs of the written code. This is, of course, a challenge common to any approach based on formal methods. Experience shows, however, that the available tools are generally of great help in this matter. Tools used with the Architecture Analysis and Description Language (AADL) modeling notation are used to model embedded systems and is supported by tools such as Osate and TASTE.  There are reports of cases in which tools have generated the proof obligations and discharged up to 90 percent of the proofs automatically. We conclude that it is possible, in principle, to define a new process that integrates the advantages of both PSP and formal methods, particularly VDbC. In our future work, we will evaluate the PSPVDbC in actual practice by carrying out measurements in empirical studies. The fundamental aspect to be measured in our evaluation is the quality of the product, expressed in the number of defects injected and removed at the various stages of development. We are also interested in measures of the total cost of the development. We welcome your feedback on our work. Please leave comments below. Additional Resources To read the SEI technical report on which this research is based, PSPVDbC: An Adaptation of the PSP that Incorporates Verified Design by Contract, please visithttp://www.sei.cmu.edu/reports/13tr005.pdf. To read the book, Software Engineering Best Practices: Lessons from Successful Projects in the Top Companies by Capers Jones, please visit this url.

On May 20, @shrmnextchat chatted with Blake McCammon (@rblake) about Millenial Influence. In case you missed this awesome chat, you can see all the great tweets here:   [View the story "#Nextchat RECAP: Millennial Influence" on Storify]    ...

Hi, I’m Kate Bruckshaw and I’m the Product Owner of a digital project to improve the way that people repay money to DWP. I graduated from the Digital Academy in July 2014. We’re transforming the way we operate to create automated, efficient services designed around understanding service user needs. As Andrew Besford recently wrote DWP is a huge multi-channel business, where customers (our users) depend on us for information and support. A big part of the work we’re doing to reduce fraud and error, involves collecting overpaid benefits. In 2013/14, we recovered nearly £1 billion but there’s more to do to increase this and to make it easy for people to pay back money. Managing the recovery of overpayments can be complex. We’re dealing in debt owed to government and with people in diverse situations, sometimes with very different needs. We want to make it easier for users to deal with us so they don’t need to phone or post stuff and can just go online to complete a repayment in a way that really is simple, clear and fast. We also want a service that works for everyone but we know how important it is to start with something that’s viable. Eric Ries promoted this as the Minimal Viable Product (MVP) and it translates well as a question: ‘what’s the simplest thing that could possibly (probably) work? This isn’t about spending months setting-up a big IT project, instead it’s about making things happen by doing things differently. Roo Reynolds' blog post is still one of the best reads on how we now create new services. Getting to alpha For the last few weeks, we’ve been working out what might be possible and what we can do quickly, knowing that we can continue to improve it (it’s an agile project and we’re iterating). There’s no better way to do this than by building something and sharing it with users and that’s what we’ve done, starting with paper prototypes and progressing to a ‘click through’ in PowerPoint before we code. This kind of approach doesn’t aim for perfect design but we’ve found it’s hugely powerful: being able to go through a user journey, show someone an interface and listen to their experience has been the best way to understand what people really think of your build. Hearing someone tell you it’s ‘sweet’ is even better. We’re aiming to complete our alpha very soon and to begin testing a fully working prototype as a beta. And that’s where you get a real sense of how transformation is starting to change DWP and of the power of collaboration. Working in the Transformation Hub Our project is running as part of a community, alongside other teams doing other digital projects and a rapidly increasing number of seriously talented people now based in our Leeds Transformation Hub. Things happen here. We can grab User Researchers for a pop-up session, get @BenHolliday to test out our prototype or take ten minutes with @mortimer_leigh to learn from the experience of launching our Carers Allowance Digital Service. What’s more, we’ve been working across government to look at re-using existing components (H/T to Ollie McGuire and his team in HMRC for sharing their source code with us), using new tools (30 minutes on a Google hangout instead of hours travelling by train) and finding out how doing things differently translates into testing new approaches with our delivery partners in commercial and technology teams. The service we’re building will be simpler, clearer and faster for people to use and we’ll have made it in a way that’s simple, clear and fast for the team.

By C. Aaron CoisSoftware Engineering Team Lead CERT Cyber Security Solutions DirectorateThis post is the latest installment in a series aimed at helping organizations adopt DevOps.  Regular readers of this blog will recognize a recurring theme in this series: DevOps is fundamentally about reinforcing desired quality attributes through carefully constructed organizational process, communication, and workflow. When teaching software engineering to graduate students in Carnegie Mellon University’s Heinz College, I often spend time discussing well known tech companies and their techniques for managing software engineering and sustainment. These discussions serve as valuable real-world examples for software engineering approaches and associated outcomes, and can serve as excellent case studies for DevOps practitioners. This posting will discuss one of my favorite real-world DevOps case studies: Amazon. Amazon is one of the most prolific tech companies today. Amazon transformed itself in 2006 from an online retailer to a tech giant and pioneer in the cloud space with the release of Amazon Web Services (AWS), a widely used on-demand Infrastructure as a Service (IaaS) offering. Amazon accepted a lot of risk with AWS. By developing one of the first massive public cloud services, they accepted that many of the challenges would be unknown, and many of the solutions unproven. To learn from Amazon’s success we need to ask the right questions. What steps did Amazon take to minimize this inherently risky venture? How did Amazon engineers define their process to ensure quality? Luckily, some insight into these questions was made available when Google engineer Steve Yegge (a former Amazon engineer) accidentally made public an internal memo outlining his impression of Google’s failings (and Amazon’s successes) at platform engineering. This memo (which Yegge has specifically allowed to remain online) outlines a specific decision that illustrates CEO Jeff Bezos’s understanding of the underlying tenets of what we now call DevOps, as well as his dedication to what I will claim are the primary quality attributes of the AWS platform: interoperability, availability, reliability, and security. According to Yegge, Jeff Bezos issued a mandate during the early development of the AWS platform, that stated, in Yegge's words: All teams will henceforth expose their data and functionality through service interfaces. Teams must communicate with each other through these interfaces. There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no  shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network. It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols -- doesn’t matter. Bezos doesn’t care. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions. Anyone who doesn’t do this will be fired. Aside from the harsh presentation, take note of what is being done here. Engineering processes are being changed; that is, engineers at Amazon now must develop web service APIs to share all data internally across the entire organization. This change is specifically designed to incentivize engineers to build for the desired level of quality. Teams will be required to build usable APIs, or they will receive complaints from other teams needing to access their data. Availability and reliability will be enforced in the same fashion. As more completely unrelated teams need to share data, APIs will be secured as a means of protecting data, reducing resource usage, auditing, and restricting access from untrusted internal clients. Keep in mind that this mandate was to all teams, not just development teams. Marketing wants some data you have collected on user statistics from the web site? Then marketing has to find a developer and use your API. You can quickly see how this created a wide array of users, use cases, user types, and scenarios of use for every team exposing any data within Amazon. DevOps teaches us to create a process that enforces our desired quality attributes, such as requiring automated deployment of our software to succeed before the continuous integration build can be considered successful. In effect, this scenario from Amazon is an authoritarian version of DevOps thinking. By enforcing a rigorous requirement of eating (and serving!) their own dogfood to all teams within Amazon, Bezos’s engineering operation ensures that through constant and rigorous use, their APIs would become mature, robust, and hardened. These API improvements happened organically at Amazon, without the need to issue micromanaging commands such as "All APIs within Amazon must introduce rate limit X and scale to Y concurrent requests," because teams were incentivized to continually improve their APIs to make their own working lives easier. When AWS was released a few years later, many of these same APIs comprised the public interface of the AWS platform, which was remarkably comprehensive and stable at release. This level of quality at release directly served business goals by contributing to the early adoption rates and steady increase in popularity of AWS, a platform that provided users with a comprehensive suite of powerful capabilities and immediate comfort and confidence in a stable, mature service. Every two weeks, the SEI will publish a new blog post offering guidelines and practical advice to organizations seeking to adopt DevOps in practice. We welcome your feedback on this series, as well as suggestions for future content. Please leave feedback in the comments section below. Additional Resources To listen to the podcast, DevOps—Transform Development and Operations for Fast, Secure Deployments, featuring Gene Kim and Julia Allen, please visit http://url.sei.cmu.edu/js. To read all the installments in our DevOps series, please click here or on the individual posts below. An Introduction to DevOps A Generalized Model for Automated DevOps A New Weekly Blog Series to Help Organizations Adopt & Implement DevOps DevOps Enhances Software Quality DevOps and Agile What is DevOps? Security in Continuous Integration DevOps Technologies: Vagrant DevOps and Your Organization: Where to Begin DevOps and Docker Continuous Integration in DevOps ChatOps in the DevOps Team  

Naomi Stanford - Organisational Design The other week, Nic Harrison talked about "fixin' to get ready" with the momentous task of transforming DWP. So what does getting ready mean? For me, it means two things: Stopping all the horseholding we do Learning how to leapfrog. Horseholding Horseholding is hanging onto things that are standing in the way of transformation. It’s the sort of stuff that new joiners notice before they start to conform, and the sort of stuff that long servers know is meaningless and non-value add but have given up trying to battle with. The word comes from a story about soldiers in the Second World War who stood to attention before a gun was fired thus holding up the actual firing process. When asked why they did this it took a long-retired Colonel to remember that in the days when guns were drawn by horses the soldiers had to hold the horses as the gun was fired to stop them bolting when they heard the noise. Seventy years later the standing to attention was still an unquestioned representation of this.  I’ve noticed a few DWP horseholdings For example, I got a great email the other day. The first paragraph read ‘DWP has been asked to seek expressions of interest for around 20 facilitators …. Grade is not critical.’ The next paragraph read ‘If you are interested in this excellent development opportunity please send your details including your name, grade, job role’. Hmmm, grade isn’t critical but please tell us what yours is...? Is ‘grade’ one of the things we’re horseholding? Others are layers of governance, hard-copy documents, bureaucratic processes (dare I say the performance management system?), the language of command and control - ‘commissions’, ‘what’s the exam question?’, ‘Who’s marking the homework?’ and acting in functional areas not across the Department. Leapfrogging Leapfrogging is what we have to do now. Think of telecoms in many African countries. They’ve gone straight into mobile phones, mobile banking, and other mobile technologies never having had fixed landlines. We have to look similarly at where we can effectively leapfrog. Right now, for example, as we design DWP for 2020 we are moving from doing small scale redesigns to meet specific programme needs, towards digitalising services, new ways of working including Smarter Working, and creating new(ish) delivery models. But this is not enough. To get to transformation we have to leapfrog that middle ground and go for fundamentally restructuring in a way that radically changes our Departmental shape, size, operating model and partner relationships. One lovely, albeit small, experiment we could build on this is running at Loxley House, Nottingham. I had the chance to visit and see a leapfrog in action - an integrated team made up of local authority and DWP employees, managed by a DWP team leader, working in a local authority building (the building is open 24/7), delivering a professional service that secures jobs for local unemployed people. The many practical challenges they faced have been overcome by having a clear, shared goal and a collaborative/non-hierarchical working style. Being even bolder we could imagine no DWP but a tiny Civil Service that is just a Secretariat, or a Civil Service which is only a policy maker or one that is entirely ‘e-enabled’. (Look at Estonia for a current leader in this field). Leapfrogging takes nerve and energy and it can be scary. It is also exciting, challenging and (I think) absolutely essential. To really transform we have to give up horse-holding and start to leapfrog. Think of your horseholding examples and let me know what they are. Now consider where the leapfrog opportunities are and let me know on those too. Email your thoughts to Naomi

by Scott McMillanSenior Member of the Technical StaffSEI Emerging Technology Center This blog post was co-authored by Eric Werner. Graph algorithms are in wide use in Department of Defense (DoD) software applications, including intelligence analysis, autonomous systems, cyber intelligence and security, and logistics optimizations. In late 2013, several luminaries from the graph analytics community released a position paper calling for an open effort, now referred to as GraphBLAS, to define a standard for graph algorithms in terms of linear algebraic operations. BLAS stands for Basic Linear Algebra Subprograms and is a common library specification used in scientific computation. The authors of the position paper propose extending the National Institute of Standards and Technology’s Sparse Basic Linear Algebra Subprograms (spBLAS) library to perform graph computations. The position paper served as the latest catalyst for the ongoing research by the SEI’s Emerging Technology Center in the field of graph algorithms and heterogeneous high-performance computing (HHPC). This blog post, the second in our series, describes our efforts to create a software library of graph algorithms for heterogeneous architectures that will be released via open source. The Opposite of an Embarrassingly Parallel Problem In computer science, the term embarrassingly parallel problem describes a situation where the same operation or set of operations can be executed on different data simultaneously, thereby allowing the distribution of data across many computing elements without the need for communication (and/or synchronization) between the elements. The problems are relatively easy to implement on high-performance systems and can achieve excellent computing performance. High-performance computing (HPC) is now central to the federal government and many industry projects, as evidenced by the shift from single-core and multi-core (homogenous) central processing units (CPUs) to many-core and heterogeneous systems, including graphics processing units (GPUs) that are adept at solving embarrassingly parallel problems. Unfortunately, many important problems are not embarrassingly parallel including graph algorithms.  Fundamentally, graphs are data structures with neighboring nodes connected by edges. The computation to be performed on graphs often involves finding and ranking important nodes or edges, finding anomalous connection patterns, identifying tightly knit communities of nodes, etc. The irregular structure of the graphs makes the communication to computation ratio high for these algorithms—the opposite of the ratio found in embarrassingly parallel problems—and thus extremely hard to develop implementations that achieve good performance on HPC systems.We are targeting GPUs for our research not only because of their prevalence in current HPC installations (e.g., for simulating three-dimensional physics), but also because of their potential for providing an energy-efficient approach to the computations. We are investigating different approaches, including the linear algebra approach offered by the GraphBLAS effort, to enable the efficient use of GPUs and pave the way for easier development of high-performance graph algorithms. Implementation As detailed in our technical note, Patterns and Practices of Future Architectures, the first milestone in our research was to implement the most basic graph algorithm on GPUs: the breadth-first search (BFS), which also serves as the first algorithm included in the Graph 500, an international benchmark specifically tailored to graph algorithms that measures the rate at which computer systems traverse a graph. The benchmark for this algorithm is divided into two kernels: graph construction (Kernel 1) breadth-first search traversal (Kernel 2) The Graph 500 also provides the specification and a reference implementation for generating graphs with the desired scale-free properties (using a Kronecker generator); a parallel pseudorandom number generator (PRNG); guidelines for how the kernels are invoked, timed, and validated; and procedures for collecting and reporting performance results from Kernel 2, which is used to rank the systems on the Graph 500 list. An early accomplishment of our work was to decouple those kernels from the benchmark’s reference implementation. The resulting code is not part of either kernel and is invariant with respect to the specific BFS algorithms implemented and forms a software framework within which we develop and evaluate our BFS algorithms. The resulting framework, written in C++, allows us to directly compare different implementations, knowing that the graph properties and measurements were consistent (for a more detailed explanation of our implementation, please see our technical note). Next, we concentrated our efforts on evaluating a number of different data structures for representing graphs. Due to the low computation-to-communication ratio, a graph’s representation in memory can significantly impact the corresponding algorithm’s performance. The following data structures and computer architectures were evaluated using systems from the ETC cluster containing up to 128 CPU cores, and dual NVIDIA GPUs: Single CPU, List. The baseline single CPU implementation is a single-threaded, sequential traversal based on Version 2.1.4 of the Graph500 reference implementation. This baseline is the simplest implementation of BFS and results in poor performance as a result of the unpredictable memory accesses. Single CPU, Compressed Sparse Row. Much research has already been performed and published to address the memory bottleneck in graph algorithms. One popular alternative to the list structure on CPU architectures is called the compressed sparse row (CSR), which uses memory more efficiently to represent the graph adjacencies by allowing more sequential accesses during the traversals. Using this data structure resulted in improved performance and allowed larger graphs to be manipulated. Single GPU (CSR). The baseline GPU implementation is an in-memory CSR-based approach described in papers by Harish and Narayanan. The wavefront-based approach in this algorithm is very similar to the behavior in single CPU implementations, except the parallelism requires the graph to synchronize between each ply of the traversal to ensure the breadth-first requirement is upheld. Multi-CPU, Combinatorial BLAS (CombBLAS). This approach is the precursor to the GraphBLAS effort mentioned in the introduction and is detailed in a research paper by Aydin Buluc and John R. Gilbert, which is an "extensive distributed-memory parallel graph library offering a small but powerful set of linear algebra primitives specifically targeting graph analytics." CombBLAS is a multi-CPU, parallel approach that we use to compare to the parallel GPU approach. Our results for these implementations are shown in Figure 1 below. The complexity of programming these architectures is a primary concern, so the performance of the BFS traversal (Kernel 2) is plotted against the relative amounts of code required both to build the efficient data structures (Kernel 1) and implement the traversal (Kernel 2). Figure 1. Performance BFS traversal (Kernel 2) relative to source lines of code (SLOC), a proxy for implementation complexity. The results from the two single CPU implementations confirmed that the data structures used to represent the graph can significantly affect performance. Moreover, the CSR representation leads to an order of magnitude improvement in performance as measured in traversed edges per second (TEPS). Using the CSR data structure in the development of a single GPU traversal achieves another order of magnitude performance improvement. The Challenge Note that, as performance of these implementations increases, so does the amount of code required to achieve that performance (about 30 percent more for CSR, 60 percent more for CSR on GPU). Advanced data structures for graph analytics is an active area of research in its own right. Add to that the increasing complexity of emerging architectures like GPUs and the challenges are multiplied. To achieve high performance in graph algorithms on these architectures requires developers to be an expert in both. The focus of our work is therefore to find the separation of concerns between the algorithms and the underlying architectures, as shown by the dashed line in Figure 2 below. Figure 2. The architecture of the graph algorithms library that captures the separation of concerns between graph algorithms and the complexities of the underlying hardware architecture. It is similar to the architectures of computation-heavy scientific applications that depend on highly tuned implementations of the BLAS specification. The graph algorithm library we are developing for GPUs is aimed at achieving this goal. If we are successful, our library will hide the underlying architecture complexities in a set of highly tuned graph primitives (data structures and basic operations), allow for easier development of graph analytics code, and maximize the power of GPUs. Part of the benefit of our approach is that the graph analytics community will be able to take advantage of it once it is complete. Enter BLAS There is a robust effort already underway in the academic research community focused on data structures and patterns. One approach to address this challenge has been suggested by the graph analytics community. Called GraphBLAS, this effort proposes to build on various existing technologies used commonly in the high-performance scientific computing community. GraphBLAS proposes to build upon the ideas behind sparse BLAS (spBLAS) to represent graphs and a parallel framework like Message Passing Interface (MPI) to scale out to multiple CPUs. A proof-of-concept implementation of this approach, called CombBLAS, was recently released by Aydin Buluc and John R. Gilbert. We also implemented this approach on our 128-CPU system and showed (in Figure 1) that we could scale to multiple CPUs and achieve more than 30X performance improvements over the "Single CPU, CSR" approach. We also achieved a 2X performance improvement over our GPU implementation. Just as importantly, however, we achieved this improvement with 15 percent less code by leveraging existing libraries. There exist similar technologies for GPUs and our goal is to develop a library that hides the underlying complexity of these approaches, and implements a number of key graph algorithms. We presented our work to the community at a GraphBLAS birds of a father session at the 2014 IEEE High Performance Extreme Computing Conference (HPEC).Collaborations and Future Work Our research bridges the gap between the academic focus on fundamental graph algorithms and our focus on architecture and hardware issues. In this first phase of our work, we are collaborating with researchers at Indiana University’s Center for Research in Extreme Scale Technologies (CREST), which developed the Parallel Boost Graph Library (PBGL). In particular we are working with Dr. Andrew Lumsdaine who serves on the Graph 500 Executive Committee and is considered a world leader in graph analytics. Researchers in this lab worked with us to implement and benchmark data structures, communication mechanisms and algorithms on GPU hardware. Dr. Lumsdaine’s team has experience in high-level languages for expressing computations on GPUs. The team created a new language, Harlan, for expressing computations of GPUs that is available via open-source. Researchers at CREST have used Harlan as a springboard to implement graph algorithms on GPUs and further explore programmability issues with these architectures. This work provided insight into graph algorithms, but also additional insights into Harlan and where it language can be extended. We are also collaborating with Dr. Franz Franchetti, an associate professor in the Department of Electrical and Computer Engineering at Carnegie Mellon University. Dr. Franchetti is also involved in the GraphBLAS community, and is performing research in the programmability of graph algorithms on multi-CPU systems. Through our collaboration, we are hoping that multi-core CPU programming can translate to GPU programming (GPUs can be thought of as many small CPUs on a single core). This year, we will also be collaborating with the CERT Division’s Network Situational Awareness Group (NetSA). Working with their network data, we will compare the performance of our approaches on commodity GPU hardware with results they have achieve using a supercomputer specially designed to perform graph computations at the Pittsburgh Supercomputing Center. Many supercomputers have multiple compute nodes (CPUs) with accelerators, like GPUs, connected. For many applications, the accelerators are vastly under-utilized, often due to the complexity of the code needed to run efficiently on them. Our long-term goal is therefore to release a library that will allow our customers and the broader HPC community to more easily and more efficiently utilize them in the growing field of graph analytics. We welcome your feedback on our research in the comments section below. Additional Resources To read the SEI technical note, Patterns and Practices for Future Architectures, please visit http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=300618.