From the Crows' Nest

Ken Miller (00:09):
Welcome to from the Crow's Nest, a podcast on electromagnetic spectrum operations or mso. I'm your host, Ken Miller, director of Advocacy and Outreach for the Association of Old Crows. You can follow me on LinkedIn or email me at host at from the crows nest.org. Thanks for listening. In today's episode, I welcome Jeff Kojak and Olivia Letz from the George Mason University Baroni Center for Government Contracting within the Costello College of Business. We're going to dive into current efforts to reform the planning, programming, budgeting and execution process, or PPBE, which is an annual cyclical process to determine and allocate funding for the Department of Defense. It's one of the three major decision support systems for defense acquisition that continues to confound some of the most knowledgeable experts in DOD today. The process itself is laborious and confounding at times. It's also a legacy process that too often challenges the defense department's ability to move technology and capabilities from the lab to the field with speed.

Ken Miller (01:10):
This reform effort is just underway with the new administration and congress coming in January. We simply don't know yet the outlook for the reform, but what is certain is that we need to find a way to continue it. We simply cannot continue with the same approach and expect this process to adapt on its own because of the complexity of this topic. My interview with Jeff and Olivia is in two parts. Today is part one. Part two will be released on December 18th, our last episode of 2024, and in that episode we'll get into more detail with the recommendations for reform. Before I get to them, just a word about next week, A OC 2024 is upon us. The global EEW community is going to descend upon National Harbor Maryland at the Gaylord Resort. Next week, December 11th to 13th, we have a great trade show and symposium planned for our attendees to discuss electromagnetic warfare within the context of great power competition.

Ken Miller (02:08):
As I mentioned FTCN from the Crows Nest podcast, we'll be there all week with special episodes. We will not be recording our regular A OC Member subscriber bonus edition next week on Tuesday and releasing that next. We will not be releasing that next week as we normally do. We'll actually return to that edition in early January, 2025, but we will be there. The podcast will be here at the convention all next week with special episodes available to all of our listeners. I'll sit down each day with our keynote speakers, discuss their message and perspective. On Wednesday, I'll be interviewing Major General Anne-Marie Anthony, director of Gem O under us stratcom. And then on Thursday I'm pleased to have both General Thomas busier. He's the commander of Air Force Global Strike Command and Vice Admiral Craig Clapperton, who's commander of Fleet Cyber Command Navy. And then on Friday I'll wrap up with major General Elizabeth Michelson, deputy Chief of Staff for Operations and NATO's Joint Force Command Norfolk.

Ken Miller (03:16):
But that is not all that we'll be doing. We'll also be walking the exhibit floor, talking to exhibitors, attendees, recording certain segments and uploading those up to LinkedIn and social media. So if you are unable to attend or even if you can attend and you want to stay up to, to speed with who we're talking with during the week, follow us on LinkedIn and we'll be uploading segments throughout each day. And then on Thursday of next week, on December 12th at four 30 in the afternoon, we'll be doing a live recording from the main stage, a live episode from the main stage at the convention. I have some special guests with me. We will talk about the convention, we'll field questions from the audience. It'll be a really fun interview, fun conversation with guests as well as with the audience. Alright, so without further delay, I want to bring you part one of my interview with Jeff Kojack and Olivia Letz on PBBE reform.
Ken Miller (04:14):
Let's listen in. Alright, I am here with Jeff Kojak and Olivia Letz from the George Mason University Costello College, Baroni Center for Government Contracting. I have asked Jeff and Olivia to join me here on the show for a special two-part series taking a look at the PPBE process, which is planning, programming, budgeting, execution process in DOD. They have been a part of a study to take a look at necessary reform of that process through a special commission that was established. So through the GMU Barone Center, they contributed a special report identifying some of the issues surrounding the process and as well as findings and recommendations. So without further delay, Jeff and Olivia, thank you for joining me here on from the Crows Nests. It's great to have you. Alright, so just before we get started in talking a little bit about the PBBE process, Jeff, I just want to have you share a little bit about the Baroni Center and what its purpose is in terms of its participation in this process.
Jeff Kojak (05:19):
Thanks, Ken. So the bar center is the only academic center that is squarely focused on government contracting. There are some think tanks that have some offices that deal with this, but we are the academic center that's anchored to examining, supporting, researching, advising when it comes to government contracting. And that's both for the government as well as for industry, and then for other academia faculty and whatnot across the country. So we're there to bridge the of understanding between these diverse elements.
Ken Miller (06:02):
This reform effort actually got started in the FY 2022 National Defense Authorization Act. So it was passed roughly in the end of 2021 depending on the timing on that. But it is set forth a commission to take a look at this process. And then your center contributed a report that helped kind of shape both the interim report that was released last year in, I think it was September, 2023. And then there was an implementation plan and then the final report came out in the spring. And now we're at the stage where we move forward with having to try to implement, at least from a legislative perspective, certain reforms that will affect the process. And so we're at the beginning stage of that implementation, I would imagine. Could you talk a little bit about just the timeline of this effort before we get into the actual process, the actual PPB process and the pieces of the reform effort?
Jeff Kojak (07:02):
Wait, Ken, when you say the timeline of the effort, what effort,
Ken Miller (07:06):
So the effort of the commission being set up, when did the Baroni Center begin participation in it and your report and just kind of piece it together? Because it's been over a couple years. I'm not sure when the commission actually took off.
Jeff Kojak (07:24):
I see, I see. So as you mentioned, national Defense Authorization Act for 2022, what's important about that is that came from the authorizers, right? So just as a baseline is that the National Defense Authorization Act, both coming out of the Senate and the House of Representatives, that comes from the authorizers, that comes from the Senate Armed Services Committee and the House Armed Services Committee. So they were the origin of that legislation. It was passed by both houses and signed into law. And what it stipulates is, hey, there needs to be a commission and on PPB reform, and that commission was stood up at the end of 2223. It was fully manned and they began work. And in 23, 1 of the things that they did was they wanted case studies to support their work, to inform their work, to provide examples of what's right and what's wrong or what's effective and what's ineffective. And they came to the baroni Center for Government Contracting to help them with these case studies to understand examples of what's effective and what's ineffective. And as you mentioned, Ken, is that the interim report came out in the autumn of 23 and then the final report came out in the spring of 24. And there's been quite a few public events both hosted by the commission and hosted by Baroni in person and webinar that have supported the commission's findings and our findings, which are ultimately wrapped into the commission's findings. And

Ken Miller (09:13):
At least the white paper that you released was in September of this year. That's what first attracted my attention to your effort and began the effort to have you on the podcast. So like I said at the top of the show, we're going to break this into two parts, two episodes. The first part is going to be talking about kind of the background of PPBE process, some of the challenges of it, some of the motivations for strengthening it and reforming it so that technology transition from the beginning to the field of system is a little bit more efficient, allows for more innovation and so forth. So I wanted to get to begin, this is one of those topics that I think that everyone will say in the DOD sphere, many people say it's absolutely important, but yet at the same time it it's legacy. We need to update it and so forth. But then when you ask people what is it, the knowledge of actually the process itself escapes a lot of people. So I want to take time to kind of go back and take a look at the process. And with that, I'll turn to you, Olivia, with regard to the PPPE process, again, it's planning, programming, budget and execution. It's an encompassing process. Can you describe a little bit about what is it and who are some of the chief participants in this process? At different points in time over the years of a program or initiative,

Olivia Letz (10:46):
PPPE affects everyone in the defense industrial base. There's no one who isn't affected by it, but many entities have a very, very large stake in it. You got three service secretaries, five service chiefs, the office of management and budget, four Congressional Defense committees, and the whole defense industrial base. All the contractors comprising many companies. So each military department conducts the PPPE process of its own accord slightly differently. And over the years, PPPE has changed slightly. Some secretaries of defense have changed the flavor a bit to delegate more centralized authority or more back to the services. But it's fundamentally remained pretty much the same since 1961 and it serves a very important purpose. And while the executive branch is supposed to maintain leverage as an executor of the budget, Congress has that kind of main authority. It's the constitutional authority to provide for the common defense.

Olivia Letz (11:45):
And as we see through PPPE reform efforts, it's still to give some of that authority up sometimes. But it was established in 19 one technically by a comptroller Charles Hitch who was directed by our longest standing Secretary of Defense McNamara, who had just been made president of Ford Motor Company as Kenny was elected. And Kennedy subsequently asked him to be Secretary of Defense. So he turned on the big Ford Pay and went to DOD. And he kind of instituted this systems analysis, continuous analysis refinement. It was very quantitative oriented emphasis on facts and figures and applied what at the time was cutting edge management of businesses and applied it to defense bureaucracy. And those features remained today. But prior to PPPE, formerly PPPS planning program, budgeting system services kind of had their own budgets and pushed their preferred weapon systems. But this made it more difficult to organize budgets. So PPPE fixed some of those issues, but that buffer of cold logic as we see in reform efforts isn't always perfect for moving at the speed of light to accommodate new technologies.

Ken Miller (12:57):
Yeah. So you mentioned, so it started in 61, and so obviously back then, this was kind of, I don't want to necessarily say it was revolutionary, but it was a major shift in how we tried to tackle this issue of planning and programming DOD initiatives.

Olivia Letz (13:16):
And sorry, I danced around the main purposes to align long-term strategy to budgets. So

Ken Miller (13:21):
Back then it was kind of is sought to fix a lot of the existing problems, especially come in the midst of the Cold War. But now over the last 50, let's see, 50, 60 years, what are some of the challenges that have arisen with the process over the last 60 years? I mean obviously now we're not budgeting when you're talking about trying to align long-term strategy and budgeting and programming with the services who have the primary responsibility to man train and equip, along with you have the service chiefs and the COCOMs that are responsible for fighting over these decades. A lot of it's a process that's proven to be difficult to navigate and has become, at least from my understanding, a bit more complex. So can we talk a little bit about the specific process of where things begin and the steps that are taken through the process to identify and align the long-term strategies with the budget authority?

Jeff Kojak (14:22):
So to begin with on the planning that's referred to in PPB, what that is is that's generally pointed to as the National Defense Strategy, the National Military Strategy, those are aligned to the National Security Strategy. Those documents that come out from the White House and from the Office of Secretary of Defense and from the chairman of the Joint Chiefs of staff. Those documents happen on a episodic. They're aligned to, for example, the next national security strategy will come out in 2025 or the beginning of 2026 depending on the administration. And then the National Defense Strategy and the National Military Strategy are aligned to those. So that's the planning part. And those happen well in advance of everything for programming. When we talk about programming, what we're really speaking about is that the services, the military services themselves, their program offices, their acquisition sections of their services, and the same goes with the defense agencies.

Jeff Kojak (15:32):
They build a putative budget, a budget request inside of a thing called a program Objective memorandum of Palm. And that happens up to two and a half years before the budget is actually executed. I say again, two and a half years before the budget is executed. And then those service Palms Poms program objective memorandums, those will go to the service secretariats service Secretariats will oversee a sort of vetting of those, and that becomes a budget estimate submission. Those go to the Office of Secretary of Defense, principally the which CAPE is the cost assessment program evaluation. And they go through a process called program Budget review, which all the components of the Office of Secretary of Defense, such as the Under Secretary of Defense for research and evaluation or the Under Secretary of Defense for acquisition and sustainment, the services, the combatant commands, every stakeholder you could ever imagine inside the Pentagon, inside the Department of Defense, they all do program budget review and that's in the autumn.

Jeff Kojak (16:51):
And then that will go, ultimately, there's a thing called pass back, which OMB office and Management of budget at the White House and the executive office of the President, they do that with the other departments. They don't really do pass back with Department of Defense because they're part of program budget review, but there is some sort of dialogue between OMB and DOD. And then the president's budget request is submitted to Congress the first Monday of February unless it's after an election, after an inauguration. And then who knows when that budget is submitted by the White House. And then from the first Monday in February until some period of time, Congress debates it. And this has commonly taken all the way through Christmas, which actually goes into the next fiscal year for which this is concerned. And so then once Congress passes it into law and the president signs it, then the money is allotted, it's appropriated, and then it's allotted back to the Department of Defense.

Jeff Kojak (18:09):
Even when as part of PPB in the E for execution, there's an opportunity for reprogramming, which means that the Department of Defense can go back to Congress and can say, Hey, we want to change things, and Congress can endorse that or not. And then execution can go on for many years after the nominal fiscal year. Execution can go on depending on the color of money and the year it can go on for many years, SHIP building money can go on for 10 years. So PPV has not only does it have myriad stakeholders, but it has myriad steps and it's governed by statute, it's governed by regulation, it's governed by policy directives, and it's governed by practice. So the amount of complexity that's involved is a moat for people actually understanding it and being able to function with it. A

Ken Miller (19:19):
Couple of things that you said that kind of jumped out is just the number of stakeholders that are involved in the process. From my vantage point, from advocacy, we deal a lot with Congress and they come, I don't want to say come in at the end, but certainly they come in at that point of budgeting the congressional defense budget. But before then, you mentioned over two and a half or more years before there's even a request, this process is going on. How has the complexity increased over the years to the point where it is perceived to not necessarily be the streamlined process we need for today's innovation purposes, getting the technology into the field?

Jeff Kojak (20:07):
So again, what are we talking about? We're talking about in today's dollars, $850 billion, right? So the largest chunk of discretionary federal funding. And who is the discretionary part of that? Right? So Liv brought it up, the power of the purse of Congress. So Congress decides the wealth redistribution of that 850 billion, and that is a lot of money and people like to make money. So what has happened in PPBE is that two of the colors of money, and when I say colors of money, I mean appropriations categories. So there's military personnel, there's operations and maintenance, operations and maintenance, there's military construction. But the two big ones of principal concern inside of PPB are RDT and E research and development, test and evaluation, and then procurement. And when we talk about acquisition, we mean those two things. We mean RDT and and procurement combined. That's what we mean when we say acquisition.

Jeff Kojak (21:24):
So Congress, as Liv brought up, is informed and influenced by the defense industrial base as is the Department of Defense, right? So as Liv said, the defense industrial base sells things to the Pentagon and it lobbies Congress. And so industry is in business to make a profit. And so industry and Congress have played together, as has the Department of Defense to require either by directive or by practice, is to have excruciating detail in how the money is spent in RDT and E and procurement. And because that budget is designed two and a half years before execution, things change. I mean, our family budgets change from one month to the next. The defense department's more complex than our family budget. And for it to be planned two and a half years prior to execution, it is hard to swallow. And so a huge concern inside of PPBE is the amount of the constraints and the restraints that are placed on how the program offices inside the services, the program offices inside of the defense agencies spend their money. And those handcuffs, those constraints on the program offices slow things down. It allows them not to be reactive or proactive. It allows them not to have the full agility and flexibility to adopt new technologies when they arrive to respond to new requirements that come out of the combatant commands, for example. So that lack of responsiveness to technology and requirements is tied to the amount of excruciating detail that is imposed on the budget through the process.

Ken Miller (23:51):
Now you mentioned the role of reprogramming in the budget and execution phase. How has reprogramming, has that taken on a much deeper significance in the process because of the lack of responsiveness or has that always kind of been important that reprogramming phase?

Jeff Kojak (24:11):
So reprogramming has always been part of the process. It has increased in its importance, its saliency and the debate around it since the global war on terrorism began, and the reason why is because during the Cold War, the requirements were fairly static. But once nine 11 happened and we were embarked on the global war on terrorism, it seemed that requirements were changing and that we were actually fighting an enemy that was thinking and reacting to us. And so of that, because things were changing along the way, there was a reprogramming meant more to the Department of Defense, and it meant more to the defense industrial base responding to things. One thing on reprogramming is that the reprogramming amount, a critical part of the debate around reprogramming is the amount that's allowed. So there's below threshold request BTR, and there's above threshold request a TR. And the division line between that now per congress is $20 million. So if it's under $20 million Congress, Congress has just says yay verily. But when it's over $20 million, Congress wants to hear about it and be involved. And then there's also a caveat to that, whereas if it's an item of special congressional interest, then that also bears a TR reprogramming a request to Congress.

Ken Miller (26:00):
So we mentioned that the commission for reform was established in FY 2022. Could you talk a little bit about leading up to that, how has the evolution of technology, both from threat and what we have to do, you mentioned a little bit about the global war on terror shifting. Obviously now with this commission was introduced before the war in Ukraine, but you can see even now the war on Ukraine is pushing constant change in how we think we need to respond from a defense perspective, from military perspective, how has this process been put under greater stress in recent years that kind of stimulated the need for a reform effort? And maybe related to that from a congressional perspective, how has the process in Congress broken down to the point where you mentioned the budgets aren't usually passed until December, sometimes the appropriations are part of a CR up until the new year, the next year. Obviously there's a lot of uncertainty with the congressional budgets each year. How has that stressed the process leading to the reform effort?

Olivia Letz (27:13):
I want to jump in with something I really like that a 2020 white paper from the center said with our former colleague Eric Lofgren, he noted how the best military systems don't often respond to requirements. So transformative tech, like anything from F 16, F 18 to GPS stealth are iterative processes that are developed in highly non-linear ways. But the PPPE as you can, there's lots of charts, step-by-step. It's a highly linear process, and flexibility is clearly important, but lacking in PPPE, which is one reason the PPPE commission for form was established. And the final report says the time lags in the PPPE process, those give us adversaries more time to innovate as well. And another big feature that we've sort of touched on, just the massive amount of complex information that's displayed, how the dollar amount accounting for inflation as well has gone up so much that defense industrial basis much more complex than it was in the sixties.

Olivia Letz (28:18):
Congress is getting almost 2000 budget lines. So how that complex information is displayed has a big part in increasing the complexity of congressional DOD relations. So justification books are J books composed in the budgeting phase. They're broken down instead of a portfolio sort of layout, which many people have advocated for an important sort of reorganization of the budget to just the portfolio of missions instead of these many, many budget lines breaking every aspect of a defense program into highly separate lines. So this is a big contributor to how crazy it can be with Congress and back and forth on budget lines and markups. And I don't know, Jeff, if you have any further comment on that as well.

Jeff Kojak (29:06):
Understanding Congress is very important to understanding PPB. Many people when they think of PPP, they anchor solely to what happens inside the Department of Defense. They anchor solely to the sort of debate between the military services trying to satisfy their requirements and the combatant commands trying to satisfy their own different requirements. What's also important to understand is that Congress itself is not a monolith. Congress itself is a coagulation, an aggregation, is that a better word? An aggregation of very disparate interests. So you have the authorizers, you have the appropriators. Not only do you have a hundred senators and 435 members in the house, and then not only are divided red versus blue and different geographic local interests. And there are different political donors among the defense industry, but they are divided against each other from an authorities perspective. The authorizers authorized the appropriators appropriate. And think of that in a sense as that the authorizers can provide a ceiling on how much money is spent, and the appropriators actually cut the check.

Jeff Kojak (30:34):
And I think it goes back to the origin of the commission. It was the authorizers that asked for the commission to do its work. The appropriators did not ask for this. And the bulk of the recommendations that the commission made and the bulk of the work that, and Liv can speak in great detail about the case study findings is that the bulk of the observations that have been made are actually quite frankly germane to the appropriators, not the authorizers. There are things that need that can and should be changed. For example, in the DAR, the defense acquisition regulation DOD 7,000 point 14, but the real meat is dependent on the appropriators. And so in effect, the commission's report makes all these statements about what the appropriators need to do differently. And the commission is ultimately an instrument of the authorizers. The divisions between parts of Congress are important to understanding what is really being debated when it comes to PPB.

Ken Miller (31:59):
It seems like the process needs to become today more flexible and responsive to some of the changing threat environments and technology development efforts. But it seems like there's so many steps along the way where things can fall short of their goal. And even from an EW perspective, and I know Jeff, you have some EW background, which was great to learn. It is really hard to even identify some of the initiatives in EW in the defense budget because it's just so, things are so hidden and even terminology and justifications that are in all the books that come in with the budget don't necessarily point you in the right direction of where things are at. So is it safe to say before, I want to get into talking about the commission itself, but is it safe to say that it is really become just a process where too many things can go wrong at too many points and we just need to have a more adaptive, flexible, and responsive system in place to help at least with some of the leading technologies?

Jeff Kojak (33:09):
Yeah, I think that that's totally fair and wealth dated. That is a key summary. And it goes back to Liv's point about how many budget line items, how many program elements there are inside of procurement and RDT and E each. The fact that there are so many of those budget line items and program elements, each of those has a little fence around them. And so that there's a lack of agility. And then you brought up the fact that we don't even know what's there. I mean, so much in the electronic warfare world is in fact classified and it should not be publicly available for our nation's enemies to see. So there is enormous complexity as you say, and that enormous complexity and detail does constrain us from being agile and adaptable.

Ken Miller (34:14):
So the ND, the Defense Authorization Act of 2022 established the commission. So could you talk a little bit about how that commission was set up, the steps that it followed to produce both the interim report, the implementation plan, and the final report and just where are we at and what can we expect moving forward just at the 30,000 foot level? And I'll turn that over to you, Olivia,

Olivia Letz (34:38):
Where we stand right now. So final report was released in March, and the commission itself had been comprised of, was it 14 people, Jeff, but they all had very experience and defense, industrial base and government. And it was two years of painstaking interviews and research and many separate studies within that were submitted. And our study was one of several that tried to provide real tangible sort of cases or examples of agencies doing things right by PPB or PPB, not doing right by some defense agencies or services or programs. And so currently where we stand is DOD release implementation plan in response to the interim report. But as of right now, from what we hear for other COD is currently working on a plan to address the 28 recommendations released in the final report from last March. And there's things that can currently be done at the service level and are being done, for example, all that about common data platforms and data sharing and delegating more authority to the program, the PEO level, those, and even consolidating budget line items, which was a big factor in our case studies and in what the PPE Reform Commission released in its own reports as well.

Olivia Letz (35:56):
There are efforts underway to do that right now, and the commission's final report proposes this major overhaul of PPPE, but we've also had a lot of events in our webinars and looked into ways that also the PPPE process can be navigated now. It does serve an important purpose and there are ways for many programs to succeed within that context as well.
Ken Miller (36:19):
It was interesting that there was an implementation plan released after the interim, but before the final report, what was the reasoning for that and not waiting until the final report was out. And also I want to kind of drill into what you had mentioned, Olivia, where there are some aspects of the report that can be implemented without even Congress getting involved. DOD can just do it. And then there's other things obviously that Congress has to probably authorize because it's changing frameworks or guidelines or regulations. So talk a little bit about that process of why the implementation plan came out and then where the breakdown is between what can be done in DOD and what needs authorization from Congress.

Jeff Kojak (37:02):
The interim report came out and it was, even though not everybody agreed with what it had to say is it said many things that were commonly agreed to. And it was briefed to the Deputy Secretary of Defense, Kath Hicks, and she has long experience in this realm, and she recognized that there were things that could be done immediately and that should be done immediately that would benefit national security. So she asked her folks to come up with an implementation plan immediately, not to wait for the final report, but to immediately do so and they did so. And there are items, as Liv alluded to, that are germane to the Department of Defense. So for example, establishing a classified and unclassified communication enclave with Congress, for example, creating a common analytics platform, doing a better job at auditing, doing a better job at training. So those things are internal to DOD that don't really require Congress to be involved, so to speak.

Jeff Kojak (38:16):
Then as Liv noted is that the Department of Defense creates these things called justification books and these justification books, these J books, they spell out in excruciating detail by program element and by budget line item where the money goes for procurement and RDTE, the consolidation of those lines can be done by the Department of Defense without authorization or the appropriators stipulating anything from Congress. It can be done. Now Congress can push back and say we want greater detail, but ultimately those J books are drafted by the services and overseen by the Department of Defense so they can do what they want to do. And so that is also internal to the department. There are things that are external to the department that require Congress to be involved. So for example is new starts. So a huge complaint with regards to PPB is that new starts are forbidden under a continuing resolution.

Jeff Kojak (39:35):
And of course there is always a continuing resolution. Ever since 1977, continuing resolutions have been the name of the game for the last 30 years. There's only been three years that didn't have a continuing resolution, 95, 97 and 2019. So continuing resolutions are the name of the game, but new starts and what is a new start. It is not just a new program, but it's also theoretically an answer to a new requirement or a new technology. And so those things are forbidden under a continuing resolution that's governed by the appropriators, that's governed by section 1 0 4 inside of every continuing resolution. The reprogramming is governed by the appropriators in the annual appropriations section 8,005. The way money can flow between different colors, different appropriation categories through transfer authority. Again, the regulations cover this, but this is ultimately defined and controlled by the appropriators. So when it really comes down to it, the appropriators have the power to make things more agile or not

Olivia Letz (40:49):
The or Congress's way of responding to the final report was all the language in the Senate. NDAA for FY 2025 remarked on the commission's final report findings and commended certain things like it's advice on reprogramming thresholds and the budget line item consolidation and directed the comptroller to provide periodic briefings on implementing the recommendations and the establishment of a cross-functional team. But we don't know who would be on that team yet.

Ken Miller (41:17):
Let's talk a little bit about the baroni GMU BARONI Center involvement in the commission study. They reached out to you, they were looking for some case studies, the BARONI Center put together your report. So could you talk a little bit about the role that BARONI Center played in the commission and the case studies that you contributed to the report

Olivia Letz (41:43):
Commission on PPBE reform? The BARONI Center was tasked with a very specific goal and that was conduct a review of how the PPBE process supports joint efforts, capability and platform lifecycles and transitioning technologies to production. So we were also asked a couple additional questions on technology transition. Our higher valued opportunities were gone at the expense of continuing lower value programs. And when we say, sorry, I don't hear the background. Okay, I can hear you guys. Okay. So you can interpret higher value programs in terms of desirability and support of the war fighter or DOD strategy. We weren't thinking dollar value necessarily, but high value in terms of its serves war fighter goals. And we were also asked that another kind of sub-question is the PPE process a significant root cause of failure to reallocate resources to higher valued uses accessing from JS or small a acquisition process?

Olivia Letz (42:41):
So another important background detail here. So JS stands for joint capabilities integration and development system for requirements. And when we say small a acquisition process, we mean defense acquisition system or the as for buying items and the multiple pathways for operations and sustainment including the adaptive acquisition framework, the acquisition system as a whole PPB is part of big A acquisition, one of three elements of JIS and DAS. And in many cases in the case studies, those are all very intertwined, but for our case studies. So we looked at four programs and two organizations that were chosen for operational importance relevance to PPBE related issues or use of cutting edge technologies. And our methodology was a lit review and reviewing of budget documents, interviews, primarily relying on interviews with program offices and key people associated with the programs and organizations, including industry folks. And a big part of our research was we wanted to see in our case studies concrete examples for PPPE, also directly exacerbated Valley of Death and previous reports. Brony Center has examined the valley of death and understood it in that context as a time lag between evidence of a desirable effort and allocation of funds to a program of record. So our case study research supported that element of the PPE dialogue, common criticism that it promotes a valley of death. So where programs are just basically dying in a waiting room before they can be rolled into services palm processes. So that's the sort of baseline of our research, which made us way part of it into the final report.

Ken Miller (44:36):
So what were the key findings then of these six case studies? You said you did four programs, two organizations. Could you talk a little bit about each of those and what findings emerged from the six case studies?

Olivia Letz (44:49):
Sure. So first we were assigned by the commission already wanted to look at three major autonomous system programs throughout the services. So a Navy program, army program, air force program. And these case studies were undertaken by a colleague, Dr. Ed t did a great job on these and they had some common lessons for sure. So we had the commission also wanted a space development agency to be examined. But as we started conducting interviews and doing some initial literature reviews, we also decided on Army's Titan program and joint Joint Rapid Acquisition cell or JRE for the additional programs we decided on, but for three autonomous systems. So we had Air Forces collaborative Combat aircraft or CCA we call it. So unmanned combat air vehicles, which will be part of Air Force's Next Gen air dominance initiative deploying six gen fighters. And we had Army's robotic combat vehicle or RCB program developing autonomous and semi-autonomous prototypes that are part of their next gen combat vehicles, family of ground combat vehicles.

Olivia Letz (45:52):
And we had Navy's large and medium unmanned surface vessel program. So part of the Navy's effort to expand fleet with more low costs and adoptable vessels, which will accommodate various payloads. And then moving to space development agency, a newer agency makes use of commercially procured cheaper satellites to create layers of satellites and their main goals to have a proliferated war fighting space architecture, PWSA, supporting surveillance navigation, deterrence, communicate various joint functions across the war fighter. And we had army's tactical intelligence targeting access node, which again we cited on as we were conducting our initial lit review. And Titan is meant to eventually phase out legacy ground systems and use massive amounts of intelligence with sensors real time and enable AI supported target recognition and long range precision targeting. And joint rapid acquisition cell was kind of more of a different case study for us. And it's been around longer than any of the other programs and agencies we described in since 2003, I believe. And the JA uses rapid acquisition authority RAA to address critical needs that are identified by the combatant commands. And these needs are submitted as joint urgent operational needs or joint emergent operational needs. So juons and GNS through a very special process. And these are meant to be fully rolled and sustained by a DOD entity, usually in military service. And JE facilitates that whole process. And we were initially assigned to the three autonomous case studies. And again, all of them are new except JR

Olivia Letz (47:32):
Case studies are mildly very modern and most are still in the earlier stages are highly classified. So it's actually hard to assess full technology transition if we're understanding it in terms of transition to production, which makes these case studies interesting for future monitoring and analysis. Like when a ca achieves its full proliferated warfighting space architecture, it'll be a fantastic case study in the future as well. But we find the use of technology transition as a concept to be pretty broad and we took care to assess it in terms of being funded through the development stage. So most of these programs we describe are highly experimental prototyping modes currently. And another interesting thing about JA that distinguished it was that it doesn't always field these super high tech solutions, but it was a good case study for showcasing PPPs difficulties, incorporating other urgent needs, including smaller adaptations to support the war fighter.

Ken Miller (48:29):
I was going to ask, is it safe to say that the attributes of being highly experimental, you mentioned a lot of the autonomy autonomous capabilities I'm sure related to that, artificial intelligence, ai, those are the attributes that are putting stress on the current PBE process with how fast these technologies are moving.

Jeff Kojak (48:54):
Not only are the technologies emerging, but also the personnel inside the Department of Defense that are responsible for the requirements process, J Sids that Liv mentioned, and the personnel that are responsible inside the program offices for actual acquisition, they are unfamiliar with these emerging technologies, right? To some degree on the directed energy front, there is a greater familiarity because industry and the government have both adapted to directed energy. But when it comes to software, when it comes to large language models, computer vision, anything on the artificial intelligence side, there's a lack of savviness. I would offer a lack of subject matter expertise. And I think that the electronic warfare community can relate to this because there are obviously personnel inside acquisition and experimentation and testing with regards to electronic warfare that they are very savvy to the state of the technology. But there are plenty of other people are not up to speed with what is cutting edge when it comes to electronic warfare and spectrum management. And so it is this specialized, highly technical, emerging technologies are not always the easiest things to set requirements for, set budgeting for and then actually go through the process of acquisition.

Ken Miller (50:43):
That will conclude this episode of From The Crow's Nest. I want to thank my guest, Jeff Kojak and Olivia Letz for joining me Part two of our conversation where we discuss, get into more detail about the recommendations. We'll release that on December 18th in a couple of weeks. So please take note to come back and download that episode when it's available. Also, please take a moment to review, share, and subscribe to this podcast. Subscribers will receive two additional bonus episodes per month, as well as the opportunity to participate in Live Studio audience for the recording of those bonus episodes. Also, we are in the process of releasing a new podcast series, our CTO series designed for chief technology officers where we deep dive into technology issues of the day. Season one is on its way out, it's available, and season one is powered by L three Harris, the six episode season. We're releasing episodes four, five, and six later in this week. But you can go to crows.org/podcasts for more information on that series. That's it for today. Thanks for listening.