SAFe For Government

TRANSCRIPT

00:07

All right. Good afternoon, everyone, and welcome to the safe for government webinar. And my name is Charles Mattox, I’m founder and principal of the i Four group but also represent i Four D solutions, which is a contracting joint venture with the federal government and want to give you all a brief, very brief overview of how agile is being adopted, and the federal government today and some of the tools that are available to any agency in the federal government or state and local to use agile methodologies to successfully implement some agile practices within the federal government. Just a little brief overview of AI for D. And to give you an idea of what you know, our focus is it is on primarily safe training and consulting. We have cybersecurity and cloud development capabilities. It’s a joint venture between the i Four group and DK W solutions, a big focus again on agile practices, enabling government agencies to become more agile within their IT environments, their cloud development environments, etc, Top Secret, Top Secret clearance facility and a lot of other capabilities that might be of interest for any government agency looking to become more agile, and as a certified company, as well, and a lot of past performance and experience with contracting with the federal government. But let’s jump right into kind of the agenda for today in terms of agile in safe in the federal government. By the way, before we get started, please feel free to put anything in the chat. Any questions, we’re going to reserve about 15 minutes at the end of this this session for q&a and to answer any questions that came up. So again, feel free to to chat your questions, and we won’t take them real time. But we’ll reserve some time at the end to address those questions. So first thing on the agenda is to talk about federal requirements and how they meet agile development, you know, that typically, we think that the federal government is, you know, it’s still it’s like a big dinosaur, it moves slow, but how can it become more agile? So federal requirements meeting agile development, then we’re going to actually talk about some of the specific practices and kind of the mindset and the new approach on how the government can adopt an agile methodology such as safe and to still have compliance and regulatory requirements being met along the way. All right. A big focus on that part of the discussion is on meeting compliance, meeting the regulatory requirements, and how does that How is that done? And we’re going to talk about the Lean QMS method on how that’s how that’s accomplished. All right. So jumping right in here, the problems that we see within a lot of our big federal agencies with government, IT and software development jobs and things of that nature, especially when you have a government agency contracting out with multiple vendors that come together to produce on a mission for a particular agency, what are those, those typical problems, and so we’re showing some here on the screen that probably resonate, especially if those on the call in the webinar here, have run a program before and have seen some of the challenges with managing multiple subcontractors. These are the typical challenges you face, right dependencies, you know, especially when you got one vendor dependent on another or you’re the vendor is dependent on some information from the agency itself, and getting that information in a timely manner. And then that causes a delay in the net snowballs, and these delays just keep piling up. And then we’re ultimately to the very last posted on here, late delivery or late discovery in terms of problems and things that that are needed to be solved complexity, how do we manage a complex project, again, require multiple vendors requiring multiple inputs, multiple discovery points in the process? These are these are definitely challenges how do we how do we simplify that complexity and make it more manageable

 

04:35

so that we can actually have some successes and stay on track and make progress as we want to? Little visibility, little transparency into the progress, the progress right every vendor every subcontractor on a big you know, program is doing things their own way or managing things in their own particular environments using different tools. to track their work and actually report on how they’re making progress, how do we how do we create more transparency around that and have a more collective unified approach on how we can manage our programs. Sometimes we have this mindset that, you know, validating all the requirements up front, getting our designs in place, boom, we’re often running that is the success factor, passing that phase gate makes it all easy thereafter. Now, that’s not necessarily the agile mindset. It’s not the agile mindset at all we’re gonna get to that. But you know, committing to early designs and requirements to early on in the progress and that kind of basically anchoring our trajectory of how we envision completing the you know, completing the solution in our program actually causes a lot of anti patterns and dysfunction. And we want to think in a more iterative way, and have a process that supports an iterative way. And processes supports a replanning a rethinking, reevaluating of our approach. So we can continue in a a true, a more true to the end solution approach. Another typical result of multiple vendor contracts and working with a lot of different groups in a program is that there’s never really a way to systematically improve across across the entire group, right? So the program each again, each team is kind of doing that in their their own right, but it’s never a collective a programmatic, a systematic way of improving across the entire mission for your agency, and how do you get better and better and improve these practices, root cause analysis some of these areas in your program to continually improve? So so this is kind of a list of, you know, there’s many more, I’m sure that anybody on the in the listening to this webinar will will can testify that this is just maybe the tip of the iceberg. And there’s so much more in terms of problems that that need to be addressed. And it’s because of this, this, you know, just managing a complex mix of different contractors in a in a complex solution that we’re trying to work on together is inherently causing a lot of problems in our environment. So well, here meets safe the Scaled Agile Framework, and I’m going to use my annotate here just so we can have a little fun and because government, so one big input to this big framework, it looks maybe complex on the on the surface, but you know, through some analysis and training on this, it actually can be very simplified in a few hours on, on how this actually can work in in an actual environment. But this framework was actually created back in 2012. So it has a lot of history. 2011, pardon me, the

 

07:50

official launch is in 2011. We started working with it in 2012. So like, a year right after. So we’ve been working with this framework and helping clients for a long time now. So see, see, we see a lot of the iterations and the different versions of the framework as as it’s come to fruition today. And government has played a big part of how the framework has transcended into today’s state and today’s version. So it plays a big role. As you can see, it’s right at the top there a key input into how the you know, the most upstream input into your requirements, or however you envision a program working with this framework. But this this whole framework here on scaled, agile framework.com, all of these things are clickable. So some of the things that we’re sharing with you today, you can actually see for yourself and in more in depth learnings against Scaled Agile framework.com is one of my colleagues to put that into the chat. And you could you could see exactly that spelling of the URL that can take you right to this website. So again, all of these, all of these areas here on this, this, call it the big picture are clickable, full white papers, downloads, a whole lot of templates, a whole lot of goodies on here are available for you to actually go in depth and find out more about the framework and how it can be applied in your environment. But this is what we’re going to be talking about. It’s basically creating a I like to call it a mental model on how we organize our teams, our vendors, our artifacts, our approach and our mindset to large scale product delivery. And so it pretty much covers soup to nuts, anything that you can think about in terms of how you need to get a product built and shipped out the door to your customer. Everything from budgeting, everything from how you aligned different allocations to operational versus development capacities. You know, it covers the whole thing soup to nuts. So, again, we’re going to touch on this very small sliver of it today and how it’s applied and from regulatory and compliance perspective in the government environments. But that, you know, again, just for your own research and your own learnings, I would definitely advocate for you to go out and take a look and check out, you know, more depth on some of these artifacts and these these constructs here in the safe framework.

 

10:17

Alright, so going on to the next slide. So, just kind of given, you know, wrap our minds around what we’re going to be talking about here. So a lot of the agencies we see on the left side hand side of this slide here, a lot of requirements from the industry, FDA, you know, there could be CMMI, you know, DL dl 178, and a particular actual FAA requirement that I’ve worked with in the past on building software, how can an agile approach still be robust enough, and have the grit and the, and the technical, you know, constraints within it, for us to feel comfortable that yeah, it’s, it’s good enough to actually build our products and ship them off to the government, especially maybe in safety, critical environments, and the such in the light. So, and I would even say that, you know, the Agile constructs, if applied properly, in done in a right way, and following a framework and a template, such as safe, can give you the guidelines to actually produce higher quality, higher safety, and higher, you know, a higher level of assurance that we are delivering the right product and your traditional approaches. So traditionally, the approaches Yeah, I mean, done, right, some of our traditional methods, especially if you had good people, managing the project, good project managers that just, you know, made sure that they’re covering the right things, and we had a process in place covering the right things, you know, is basically kind of a no how thing, but the Agile way of working actually kind of just forces your hand, I mean, you got to do it a certain way, you got to evaluate at, you know, at the incremental stages. So it’s definitely a more robust method to assure and I say quality assurance, assure compliance, assure quality, in validation verification along the way. And so that objective evidence that can be seen at the far right, that, you know, we can, we can make sure that we’re getting out the right product to our customers at the right time. So this just can kind of get your mind wrapped around. Well, why is that the case, I’m making this claim about Agile is a more robust only gives you higher levels of assurance, than your traditional ways of working. So if you can see kind of the waterfall approach AR traditional approach, up top, you know, we’re validating various stages of the lifecycle of the software, we use software as an example of the software development lifecycle. So validating stages does not necessarily mean you’re actually validating, working validated software. And I, again, I’ve been in this situation before, and that’s why I became an actual an advocate of agile development is from my history working in the aerospace industry and trying to get things like FAA compliance, push through, it’s because when you validate a requirements phase or design phase, that does not mean that you’ve actually achieved the safety critical software component that’s necessary to deliver to your, you know, to the government or to your client that are, you know, the the Air aircraft carrier aircraft contractor that you’re working with. So again, it’s the small cycles of validating the working system that actually meet those requirements. And the sooner that you can get that level of validation and verification, the better. And that’s why we see at the bottom, we have these small cycles of verification and validation happening. And that way, we can kind of assure that stamp of approval that that little blue ribbon there means that yeah, we’re getting, we’re getting sign offs at these various stages. We call it continuous compliance. Another good thing to look up on that Scaled Agile Framework website continuous compliance, verifying we can achieve some of the end results early on in the in the in the development cycle versus waiting to the very end, and finding out Wow, we got it all wrong, we have to actually, you know, do it all over again. Or we got to a big rework cycle that’s going to take months and months to get accomplished. We, I wish we would have known that six months ago, we would have put a fix in place and got that taken care of right away. So that kind of goes to our Lean mindset approach. We always call this kind of agile and lean, agile, lean go together. That’s why safe is known as an Agile and Lean framework. So these two kind of work hand in hand to accomplish these goals that we’re looking for. PDCA if you see that on the slide here, Plan, Do Check Act. So it’s something that if you’ve been in the industry for a while and you’ve had any exposure to lean, you know the short cycle or sometimes known as the Deming Cycle cycle. That’s what we’re talking about here, you plan a little bit, you implement you do you check, you validate, and then we act or adjust. And that’s how we continually move through the process and make sure that, again, kind of this concept of continuous compliance is, that’s how we verify early on, that we’re on the right track and not going off track. Which, by the way, reducing risk is your main objective as a leader. So leadership, leadership of a program leader, your mission leader, and an agency, what is your primary responsibility, when you’re subcontracting out this this work to a vendor or a collection of vendors, it is reducing risk of delivery, not delivering the right thing on time, or in getting it on time and getting the right thing into into production? And how do you manage risk along the way. So managing risk through these phase gates is not an ideal state, which you can see here, what we’re talking about. On the left hand side, the phase gate approach gives you this artificial trajectory that you are moving towards an end state because you’re validating requirements, you’re validating design documents, but you’re not validating the system, you’re not integrating and getting feedback that way, taking action through plan, do check back mindset to move forward in the right direction as we which we can see here on the incremental delivery side, on the on the right side of the picture here. So it’s kind of in a nutshell, is our mindset approach to how we should be building product and thinking about product on our on our large scale programs within the federal agencies. How do we take this more iterative way, and taking this to heart not just in terms of how we build the product, but how we contract with our contractors. So that goes into setting contracts in place with subcontractors and vendors that actually are expected to work in this way and produce results in this way, produce feedback in this way. All right. Again, I point you to the Scaled Agile Framework website, there’s an article and there’s videos on agile contracting, and how you can set up agile contracts that actually support this way of working support how you would set up a statement of work and set up a contract with a vendor in a budgeting process that is incremental, and in line with this and so that way, even from a contracting standpoint, we’re able to meet some of these objectives. Okay, so switching gears onto now, now, how is the federal government actually supporting this way of working? So the Federal Acquisition Regulation call it far for short, there is actual a tech far handbook out there on how to support this, this these Agile processes in our digital ways of working within the government. A lot of a lot of government solicitations out there now are requiring an agile method be applied to how we deliver on that particular it work or software development initiative. And a lot of this came came to fruition back in, you know, in the two administration’s go to an Obama administration, if you remember, there was a big healthcare.gov fiasco, that, you know, it was late and you know, that requirements weren’t met, I think that, you know, site was blowing up and doing all types of crazy things. But, you know, if anybody’s worked on those traditional waterfall ish projects, you recognize the pattern, right? When it’s time to release and it’s time to go live, that’s when all the issues come to, to fruition. And that’s when things hit the fan. And so learning from that, that’s kind of how this this new initiative, this tech fire was, was given birth to, and that’s why agile methods are, are highly recommended in the government space. You know, you might need to do some more education within your particular agencies to to, you know, to present these approaches, but I, you know, working with a lot of federal

 

19:03

folks that work in these different agencies, particularly, they’re setting up centers of excellence, and various like this, the Small Business Administration and the GSA organization that actually initiates a lot of these solicitations. They’re looking to make this more and more of a standard and almost a mandate that we need to always look towards using Agile methods when contracts are rolling out because preventing those fiascoes like the healthcare.gov website and making sure we can reduce risk. We don’t have an endless pot of cash. I mean, I’m a taxpayer, I know that you probably are too. We don’t want to spend our taxes on a lot of iterations of things that could have been found years ago and things just being delayed, you know, delayed, delayed, delayed and boom, we don’t even have funding anymore to actually continue with this initiative. So again, these type of things that you know being more or lean, and being more, you know, being more focused on accomplishing these objectives in a timely manner. And being the hyper focused nature that Lean and Agile applies to our development initiatives is always the way to go saves us money, it saves us time, it actually builds better products in the end. And so another thing too, too, a lot of these ways to actually, like I mentioned about contracting in a modular way.

 

20:29

That is also specified on how that can be done. In this tech far, they’re getting us contract guidance given on how to set up these contracts to subcontractors and vendors. And I’ve actually had the opportunity to work with some some at the SBA that are actually drafting up some of these, these requirements and how you contract with vendors, and what would be the Agile way to do that. And so it’s in place, it’s in motion. And so those that are listening to this webinar, that please feel free to reach out and definitely have some resources that you can reach out to to get more information on how this is being applied in the various agencies, and how you can set up contracting requirements and RFQs and RFPs that actually meet some of these, these goals here. So just jump we’re not going to jump into so that’s kind of the part of the the discussion, I wanted to have about just the overall approach, why Lean and Agile is, you know, a good fit for today’s government agencies, state and local, and how even there’s been moves made in the the actual contracting part of it part of the government to actually make make sure that these requirements are in place in RFQs, and RFPs. And also to just the contracting approach, and the budgeting approach, where these contracts are even kind of falling in line as well. So these things are definitely in motion. So now we’re going to talk more about the execution, how do you actually run an agile program in the government? How do you actually, you know, look at your mission that you’re on? And how do you make sure that it’s going to be achieved? If you’re going to take these agile methods and apply them? How could you actually make it work, so I’m gonna get some high level give you some high level understanding on how that’s done using the safe constructs. So the primary safe construct that is used is what we call the value stream, the value stream is all the people processes, tools, everything that you could think of, so when we’re thinking about that, in terms of getting our mission accomplished, it’s all the vendors that we’re working with is the agency requirements as the fires is the different agencies that are providing different requirements is the actual systems that still exist, that we need to modify and enhance and improve. So all those things together, that constitute the value stream that needs to, we’re touching all these different points in the process with, you know, with different vendors to actually accomplish some solution value. Alright, so that’s our construct, we got got a big, you know, traditionally, it’s a big program. So very similar to our traditional programs. But there’s, there’s a clear difference here, we’re working in a way to continually optimize and streamline value flow across all of these groups. And so how that actually happens is we have these different, you know, these different silos. And you can imagine there’s these different silos for different subcontractors doing different things in the process, different agencies, different groups within our agencies, needing to contribute to the value stream and getting things done. And there’s inherently a challenge working across all these different boundaries, right management challenge, political challenges. There’s this contractor doesn’t like you know, necessarily working with another contract group. And so how do we get everybody to play nice in the sandbox and work together? And so that’s what we’re given with this value stream construct, we’re giving a way of working that basically consolidates all these value streams into what we call release trains, and safe. That’s what it’s known for an agile release train. It’s a collection of processes and constructs that allow multiple vendors, multiple groups, multiple teams to seamlessly work together and deliver value together and in a most optimized way, and building solutions together and approving together, delivering capabilities and and features together. All right. Also, too, there is planning constructs within the framework that allow this collection of

 

24:39

teams and vendors to work together to produce long term roadmaps for solution deliverables. So from a business planning perspective in your agency and your mission, you know, you’re having trouble sticking to a particular roadmap that we’re implementing to the safe contracts allow us to visually Eyes long term goals as well as the short term goals that Agile is known for. Right. But also to a long term roadmap. I know that oftentimes that’s not, you know, the thing that we think about in terms of agile delivery is we think agile delivery, oh, it’s just that two week and you know, that two week delivery cycle and I’m in a program where I need to understand what I’m delivering next year. Well, the constructs allow that type of thinking from a portfolio and solution level, and how you iterate along those those long term lines, as well as the short term deliverable line. So you can plan out these milestones, these releases these these different versions and iterations of your solution that you’ll be seeing off in the future. How do you manage that from from an agency perspective? So typically, in this the safe constructs, there’s always three ways of looking at delivery? There is the technical delivery aspects, you know, technically, what do we need to have in place, from a platform perspective, from a development practice perspective, from a technical compliance, technical architectural perspective, and things of that like, and those things might be caught up specifically in some of the files, you need to have a certain level of validation in place to make sure that, you know, it’s acceptable, and you need to have some compliance checks along the way to make sure that that’s happening. Then there’s another aspect in the in the safe constructs on the business value perspective, from the end user functionality and feature perspective, you know, what are we wanting to have in the system? What did the screens need to look like? What does the user experience need to be? What type of functionality and business rules do we need to have set up in our system for it to be, you know, meet the functional requirements that we’re asking about? And then the third aspect in the safe in the safe constructs is the process perspective? What are the planning cadences that we’re going to stick to? And what is the planning approach that we’re going to have? What are different artifact levels and require, you know, in say, in Agile, we don’t call them requirements anymore? We call them features and user stories and tasks that need to be done. And so how are we going to lay those things out? What are going to be the requirements on how we manage those on a day to day basis? What are what is our visual management going to look like in terms of what’s sitting in the backlog, what’s in development, what’s an analysis, what’s in progress, and UHT in QA, and etc, in verification and develop validation step. So Kevin, a visual management system around that. So the third aspect in all of the state constructs usually revolve around a process management perspective and an agile in safe, we use Lean and Agile, Lean and Agile approach. So some of the tools are scrum Kanban, XP, some of those agile practices there. Alright, so So oftentimes in the in the, in the safe approach, so this is from an agency’s perspective, and we’ll have business owners or product owners that set the direction for a particular vendor or subcontractor, and that vendor or subcontractor may then have its own agile team implementing on a certain feature or initiative in that way. And that agile team may have its own typically may have its own product owner, and Scrum Master and development team, or sometimes the pattern might be that the product owner is on the the agency side, where are the scrum master in the development team members are on the vendor side, there’s been various patterns that we’ve seen work well, but those are some of the common patterns that you might see in some of these agency and vendor

 

28:51

setup in that way. All right. From a budgeting perspective, so let’s just jump to that. So when setting up one of those those agile contracts with a particular vendor, it kind of simplifies down to we have a fixed cost per pi, right, you know, you have a vendor, they’re going to, you’re going to issue them a purchase order for a particular, you know, so many PI’s worth of work, there’s a certain number of features that are in the backlog that we we have on the docket to get accomplished within within a certain timeframe. So again, might be certain features were expecting at these various milestones. And if not, you know, it rolls over to another pie and there might be, you know, certain incentives to get certain features definition of done within a certain number of PI’s and how that works. But essentially, that’s kind of how in a very simplified way we can see how we’re contracting out work to various vendors and various buckets of work to to actually get the the mission account Push with a fixed cost per psi approach across our program. So from a meeting regulatory and compliance on this, we’re going to go into this aspect now that, you know, as mentioned, in the safe framework, there are methods called Lean killed, there’s methods basically baked in the Lean QMS approach that’s been documented in the safe framework on how to meet some of these verification and validation requirements. Okay, so things things like, you know, code, test code coverage, things like, you know, I’ve seen like in the FAA example, for example, you have environmental testing requirements, you have to meet certain industry standards in terms of testing in terms of, you know, we just call it shake, and bake, you gotta shake the device for so long. And while it’s running an operation, simulating urine, you’re in the flights and in flight, environment, right, hot, cold, and you’re meeting these type of test requirements. And so again, along the way, we’re incrementally validating, and being able to show that we’re passing and meeting these requirements along the way versus having that saved up to the very end. So this has given you an idea of how you might look at these regulatory and compliance requirements. I stated this upfront that, you know, Does this meet our agile manifesto, the mindset that we’re trying to accomplish working software over comprehensive documentation, customer collaboration, over contract negotiation, responding to change over following a plan, we welcome changing requirements. No, it’s not. Requirements changing is not necessarily a bad thing. You know, we say that that’s to the we say that, that, you know, requirements change needs to happen. That’s the competitive advantage of the customer. Right? They must, there must be a reason why we need to change those requirements, it’s not something that we should be, you know, pushing back on so hard that it just makes life miserable. To change requirements, no, we should have a process in place that’s adaptable, that these changing requirements, it perfectly makes sense. That’s how we set up our process. That’s how we set up our our ways of working to be agile in basically accept those changing requirements and put them in place very quickly. Because some of these requirements like safety, quality, those things are set in stone, you know, those some of those, we call those non functional requirements, a lot of those things will not change. But some a lot of times functional requirements will, you know, the user likes this now, because we they’ve gotten feedback on something, we showed them a demo, then now they actually see that the system working in this way might be a whole lot better system than then before. So functional requirements often do change, nonfunctional is not so much. But But this way is this having this process this way to adapt is very important. And that’s why we we believe in it. And it’s been so successful up to this point. So a good way of looking at the various levels of verification and validation.

 

33:11

My one on one of the guys I used to work with isa say, used to give me this analogy. And I always like to use it now. But I give it a give a credit to him for for saying it. He says he like Charles if if you no verification is if I gave you the specs to build a hammer, and you go into the lab and you build that hammer. And you bring that you bring that hammer back out of the lab and you give me a demo on that hammer. And we’re reviewing this hammer and it’s a 10 foot hammer with a big tree trunk of a log as the as the handle and this huge thing big as a car as the head of the hammer, you know, we’ll be looking at the hammer. Yeah, that’s a hammer. But that’s not the hammer that I asked for. Right? So verification is yeah, you build a hammer. But it’s not the hammer that the customer necessarily asked for. So there’s, there’s some user acceptance, validation of the system that has not been done yet. So validation is just that it’s actually validating that we have built the right hammer for the customer. So that level of validation and you can see the gentleman there, you know, looking at the solution, and yeah, that is the right hammer that we that we have asked for it meets the meets the specs. It’s been verified, you know, to the, to the details that meets the requirements of the hammer of a hammer, but it’s not that that big hammer was not the hammer we asked for but now when we basically shrunk down that hammer to a use of usable size for the customer. It’s been validated to be the right hammer. And then lastly, is this hammer actually suitable to actually operate in the actual environments that we’ve asked for? You know, maybe it’s a you’re bringing it up into it the International Space Station, or we’ve actually we’ve brought it, you know, into a, a hot and humid environment where, you know, maybe if it wasn’t the right, non functional requirements in place at that point the the wood on that on the handle rots away and it’s only good for like one year, or something of that nature. So the compliance requirements need to be met as well, and making sure that we actually make sure that we have those compliance checks in place as well, a lot of times compliance also comes down into probably the thing we hate a lot, but it’s necessary is the documentation. You know, documentation, you know, oftentimes can be treated as a deliverable in above itself. Because you know, one of the things that we used to do in the aerospace environment, you need to have show that you basically have traceability reports, making sure all from what the customer is asking for all the way down to the test cases that have passed, that there’s traceability that every single requirement is being met in the customer specification. So that level of traceability is you know, having a traceability document is oftentimes a compliance requirement, especially for safety critical hardware and software systems.

 

36:22

So again, I pointed to you, I’m not going to cover this too much in depth, but point you to the Lean QMS article on the Scaled Agile Framework site definitely gives you some good good understanding in ideas around the statutory regulations and how they can be met, as well as regulatory customer. And just overall industry best practices are all kind of aligned together with that lean QMS approach. So let’s take a look at inaction how some of these things get satisfied. As mentioned early on, and, you know, it’s unfortunate, we don’t have an opportunity to go very in depth on the mechanics and the cadences of the agile and safe approach. But if you could think about it, agile, technically, you have these two weeks sprints, and then safe adds on a larger cadence, which is called the program increment cadence, which can be eight to 12 weeks worth of work, okay, and so, and we’re looking at that big picture, thinking about it from a mission or program perspective, I’m in an agency, I’m running this program, I have this 12 week increment, to I’m sorry, a 10 week increment in which I have a planning process in an alignment process that can align multiple vendors and subcontractors to a common mission. And so with that, we set objectives and milestones in place at a big planning event called PI planning at the beginning of the program increment. And we plan out all the objectives for all the different subcontractors and vendors and whoever’s a part of our mission for that time period for that 10 week period. At the end of that 10 week period, we evaluate, we evaluate, you can see there at the next pie or program increment, we evaluate compliance, quality, manufacturability, usability, all the things that we’re looking for, that’s just a small subset, obviously, but we could be looking at functional requirements, you can see that’s all started based off of a system demo. So we’re demonstrating, you know, firsthand, visually, you know, whether it be on a remote in a remote setting or an in person, you know, at at a at an actual in person event, we’re actually demonstrating that we’re meeting these, these actual requirements that we’re shooting for in terms of the objectives of our program. So one of the key Agile characteristics is that we don’t evaluate progress based on, you know, checklist or phase gates, we evaluate progress on visual demonstration of the working system. I like to say that’s like, you know, Missouri, is to show me state, you know, agile is like Missouri, you got to show me what you got, you can’t just talk about it, or show me a checklist or a PowerPoint deck about how you’re making progress on your program or your mission, you have to show the system in motion. So also to at that point, at that pi milestone point, we actually get an opportunity for customer feedback, vendor feedback, stakeholder feedback, leadership feedback, is another alignment point in the process in which we can actually get feedback on some of these regulatory and, you know, these these critical requirements that we need to make sure we’re on track to meeting. Alright, so this is validating our learning. We always say that in an agile environment, you’re continuously learning what your customer wants, up until the very point where you’re, you know, technically you’re done with the the program, you’ve deployed the solution and even at that point, you’re still learning about what the customer likes and dislikes about the solution to go went back into the next iteration of enhancements and improvements upon the solution. But we’re always in a learning mode, especially when we’re in the beginning stages of any program. And deploying and building and deploying and building and deploying up until maybe a first release. We’re constantly learning. So validating those learnings, every PII is very critical to making sure you’re, you’re reducing risk on not hitting your milestones and goals for your particular program. You could also see to at the very end, the improvement stories, another key construct within the safe framework is we don’t we don’t just call it relentless improvement. We don’t just call it continuous improvement gave you the answer before the the end there. But we call it relentless improving, it’s not an option, part of the the constructs is set up so that we have these root cause analysis sessions at every PI milestone to make sure we’re addressing the root causes of why we’re not making the right progress, why we’re not potentially building the right things, why we’re having a problem communicating across from one vendor to the next to the next, let’s put in some concrete action items and call them improvement stories to make sure someone’s taking action on that in the next PII. So we don’t face the same things that were that happened, the last guy doing the same thing over and over again, and thinking you’re gonna get different results, especially if it’s what that’s called,

 

41:21

I think that’s called the definition of insanity. If I’m not mistaken, we don’t want to run insane programs, we want to run SANE programs and actually achieve the right results. Alright, so just to give you a little bit more crisp details on what we actually evaluate, from a critical metric standpoint, at each program increment is we get, you know, in the weeds in terms of what the functionality is, what functionality has been completed up to that point, quality metrics, in terms of, you know, productivity metrics, you know, just to make sure we don’t have any issues with, you know, a particular vendor versus another are we actually staying on track is one vendor actually suffering from not having proper engagement with our product owner, one of our agency, one of our agencies that they should be working with and getting the requirements in a timely manner. So addressing all problems, from soup to nuts, making sure that we have a closed loop process and making sure we’re continually improving. Alright. And then a couple last slides are almost about to finish up here. Just to give you a little bit more insight that the the Framework also has a very robust and templates are provided to be able to construct the artifacts, the features, the epics, the you know, the epics, that’s the big, you know, sometimes you have these big initiatives that span multiple PI’s, you know, how do you document properly, something that takes, you know, six months up to a year, you know, there we call, we have like a lean business case template to address that we have a budgeting process to making sure that those get there’s there’s a funding approach to how we understand the funding model around those and the different allocation that we might want to associate with compliance and regulatory, you could see or you call them enablers, you know, exploration, architectural infrastructure, non functional areas that we need to make sure that we have in place a lot of times to this could be in the documentation area, you know, these enablers, we need these things in place in order to actually get these features across the finish line. So and also to what the features, there’s templates around those, how do you create a business type of benefit hypothesis statement? How do you write the right acceptance criteria? For certain features? How do you break down features into user stories? How do you break down the big epics into vertically slice features? Right? The proper definition of some of these artifacts actually drives a lot of the agile ways of working across your missions and across your programs that you’re running. So it’s very critical, even upstream that if you’re properly, you know, following the framework and using some of the tools and the templates in place, that just creating the right intent, the right definition of the problem allows for vendors to actually implement things in a more agile way, if you create a, you create a set of features for them in a very horizontal, you know, componentize way, we’re going to be building waterfall things all the way back over again, you’re gonna set yourselves up for building things in a way that you have to integrate everything at the end, and you’re basically kind of replicating a waterfall process, but just using sprints, and in a framework to kind of manage a waterfall process and we don’t want that. So a lot of times it starts starts up front. How do we define our requirements and our agencies? How do we define the requirements, call them the features and get away from the requirements language? How do we define the features and the initiatives and the capabilities that we’re Looking for in our agencies in a way that drive agile development practices. So again, there’s a, there’s some templates and some methods around that, that are in the framework that help your agencies and your missions to adopt better practices around that. And then once we have those vertically sliced art artifacts in place are capabilities, features and stories. Then when we’re validating at each milestone and increment, we have something of kind of concrete nature to evaluate compliance on, we have something we have a model, we have a, we have some working software that we can actually show we’re meeting compliance on that we’re meeting the requirements on that we’re meeting some of the safety critical areas of the system that that are important to us to make sure that we’re moving the ball forward. So that is kind of it in a nutshell. And I’ll stop at this slide. Just to take questions. I know we got 10 minutes left. And so I asked my colleague, Amanda, Did anything come in the chat, any questions or on any of the content that we were we’re sharing? Looking here? Any questions? I’ll kind of pause here for a moment to take any questions or comments or anything that came in to check out here,

 

46:19

I think here comes one. So question here came in NYC, the issue wasn’t the agile, but the accounting, has the federal government address capital versus expense in Agile? So great question, Bill. So again, in the in the some of the COEs that are establishing some of the guardrails around lean budgeting and adopting the Agile are adopting the safe approach to that. There are some there’s some patterns on how it’s being addressed. And so again, it’s right now it’s there’s nothing across the board, you could just say all federal government has not adopted not even adopted, you know, what I just showed here in the in the in the slide deck around just establishing a fixed cost per PII contracts with vendors. I mean, that’s even, you know, a little bit that’s kind of on the on the cutting edge. So, you know, to your question across the board, but there is some, they are experimenting with it. And I just don’t have a personal exposure to any, any program that’s actually adopted, the constructs that are being represented, but I know they’re being pushed. And so I know that they’re trying it out. And it’s still in kind of experimental stages. And but it’s still in the motion. And they’re definitely trying to make headway in this area. Because I know those guys, I actually trained several those folks that actually working in GSA, the COA, and they’re trying to push some of these initiatives, but it’s just very slow to adopt, you know, these are new new constructs. And but I’d say that, I’d say definitely, it looks bright, you know, the future looks bright, because, you know, they definitely see the challenges with using the old expense capital versus expense approach with a waterfall program. And that just doesn’t match up with your agile ways of working, which is very iterative, and you’re addressing, you know, you know, capital and expense areas in a different way. In that agile ways of working. Follow up another follow up question here. Do you use hybrid to address that gap? Does hybrid work with safe so um, you know, it’s just me I’m because I’m a I’m not an agile purist. But one of the things that I am a lean agile purist, I would say that I never see an opportunity where you cannot apply a lean way of working. So you may not say, agile, you know that, you know, we’re not fully agile, but can you know, and I understand there’s some programs that you call it waterfall, waterfall ish, but, you know, maybe we can still lean out some of those constructs there. I think there’s never an opportunity. You know, even on a waterfall program where we can’t lean up and find a way to accelerate value, get better alignment, apply some of the lean construct. So to me again, there’s so long story short, yeah, there’s always going to be a hybrid type model. But I always say that the hybrid is not just agile with waterfall, it’s at agile with trying to lean out your traditional approach, because we always want to improve upon collaboration, you know, feedback and all those things that we’re trying to achieve and lean can you know, using some of those lean ways of working can help us do that, but just never. I would never just say that agile, use traditional waterfall ways of thinking is the way to go. But you know, agile and lean, agile and leaning out your traditional ways of working hybrid. Yes, go for that one. All right. All right. Now, another question came through. Are there any success use cases that we could reference to drive home the variability of safe in government? So definitely there’s some there’s some definite good use cases, I would say the GSA is one use use case. And actually, those on this, this on this webinar will will send out a some some reference links to those to those examples. The GSA SBA is another example. I know, NASA is definitely we’re we’re looking at a lot of those going after a lot of those contracts, actually, personally, so yeah, they requested to be agile. And you can think about it, a lot of the the agencies that might have challenges with which they all do, but challenges with not an unlimited budget. They are a slim budget, they’re gonna want to be more agile, right? If you if you could do more with less. That’s the whole agile mindset, right? That’s why startup companies use agile, they don’t have an endless budget, they want to validate assumptions and get value delivered, you know, get value out there fast and know whether or not they’re moving in the right way and try to have some some quick successes. So it’s funny, those are the those are the agencies that are attending to adopt

 

51:24

agile, more more quickly than those that have very large budgets, I’d say not even playing and pointing fingers. But by just do your own research and find that out. From your perspective, what is a better way to adopt safe by government, federal, state, state federal level, I don’t think it’s necessarily at a state or federal level, in terms of which is a better way to adopt. But you know, again, it kind of goes down to the to the point where what I was just talking about when you’re trying to get some type of technical. Particularly technical is the technical area, software development, IT management, and you’re working with limited budget, and live in a time, those are the best cases for applying agile, those are the best business cases to adopt an agile approach when you’re trying to do things that way. So and that can happen at state or federal levels. So I would say, though, that just in my experience, is that the the federal has more exposure to a lot of these agile approaches, particularly safe, in my experience in those that the state local groups that we’ve been working with, you know, they’re not they’re not hip to safe, quite yet. I mean, it’s still kind of, you know, it’s something that’s just not on their radar. But the federal agencies have definitely made, you know, this kind of, you know, there’s there’s safe as their go to for running a program and agile program in the agency. So that that definitely is a better grounds for testing out some of these these methods. All right. All right. Anything else coming through? I think that’s about it. All right. Well, this was definitely an awesome conversation. And definitely stay in touch for those on the webinar, we’re going to be having more of these and definitely touch on some other topics, you know, agile and government and various other agile topics. And, you know, we have some coming up on cybersecurity dev SEC ops, data analytics, and various other webinars and workshops. So definitely would love to have you join us again. And oops, oh, yeah, one more question. Before we leave. One more question coming through. All right. You’re gonna get the drumroll, please. Okay. Government churns change slower than the corporate and private sector. How do you see that impacting the training and transformation plans to Agile traditionally, projects programs on the federal side? Were scoped out budgets several years in advance. How can we impact that? So great question. I mean, this just kind of goes to the root of change management. And I’ll get one minute left. So I want to make sure we make sure time is honored here. So this kind of goes to the root of change management, how do we drive change? So there are some you know, there needs to be what we call in every any agency trying to adopt these methods, you need to have what we call a guiding coalition are some some champions of this change. And those champions of this change. You got to kind of set up this community of practice like messaging this kind of It’s just it’s just pulsating. It’s just pulsating through the organization. And because change takes a long time, as you pointed out, you know, sometimes it’s just not the right time. I mean, it is what it is, you know, we a lot of our groups cannot change as fast as we like, you know, there’s a lot of, you know, carryover and a lot of workforce that are kind of in their, in their seats and have been there for a while they’re

 

55:22

looking to retire. And, you know, we got to deal with what we’re dealing with, right. So change is gonna come incrementally, maybe there’s new programs and new projects, new startups, pilots, that we can implement these these new practices on, to actually show and prove like, Hey, here’s a clear example of how we just saved money and delivered value three times as fast as we did on that other program. So you got to five, be creative, you got to have those those change agents along with you, and be creative and find ways to actually show that these things are working, get the information out there, get the learning out there, have workshops, have communities of practice, have training opportunities, where you can actually have individuals trained on some of these methods. Because at the end of the day, what I’ve seen, I haven’t seen anybody come out of any of the classes that we’ve taught to actually still be anchored in their own ways of thinking, they definitely see oh, yeah, this will work. It’s just a matter of is our leadership going to allow this to happen. And so leadership, allowing the change to happen, and driving the change and guiding the change is kind of the crux of it, and how do we keep up the momentum and the pressure and things of that nature. So with that said, it’s thank you guys again, for joining the webinar. We’re going to end it here. On that note, we will email all you guys I mentioned the links to some of the success cases, which I think are very valuable for you guys. But other than that, again, thanks a lot for joining and we will see you next time. Have a great day everyone. Bye bye