From the Crows' Nest

Ken Miller (00:10):
Welcome to From The Crows' Nest, a podcast on electromagnetic spectrum operations, or EMSO. I'm your host Ken Miller, director of advocacy and outreach for The Association of Old Crows. Thanks for listening. Today, I am very pleased to be joined by Mr. David Tremper. He is the director of electromagnetic warfare, or EW, in the Department of Defense Office of The Undersecretary of Defense for Acquisition and Sustainment. Before we begin, I want to thank our episode sponsor, Northrop Grumman Corporation. Northrop Grumman provides full spectrum superiority. Their innovative, multi-function interoperable solutions ensure war fighters have full spectrum dominance to make real time decisions no matter the environment or domain. Learn more at ngc.com/ew.

Ken Miller (00:50):
All right. I'm here with Mr. David Tremper. He is the director of electronic warfare in the Office of the Undersecretary of Defense for Acquisition and Sustainment. David, it's great to have you on From The Crows' Nest. Thanks for joining me.

David Tremper (01:01):
Hey. Great. Thanks Ken for the opportunity. I think this will be a great talk.

Ken Miller (01:05):
I appreciate it. I had you on the show a few months ago, in May, at our Army Cyber Electromagnetic Activities Conference. Real briefly. I grabbed you and did this... Pulled you into the studio on your way out for a brief interview, and I wanted to take some time to talk about some of these key issues a little bit more in depth with you, so I appreciate you taking some time to join me. For our listeners' sake, you know, I want to take a step back and begin with your position because it's extremely important in OSD but it's also very complicated. There's a lot of pieces to what you do and your mission. So, I was wondering if you could just take a few minutes to tell us about the role that you play as director of EW and how it relates to other offices and responsibilities in OSD.

David Tremper (01:50):
Sure, Ken. So, Acquisition and Sustainment is responsible for Department of Defense, obviously acquisition policy and ways that we get and purchase new systems, and that's different than research and engineering, our OSD R and E, which focuses on the science and technology, more of that six one to six three, even six four level. Those are the different budget categories. So, Acquisition and Sustainment looks at the services acquisition programs of record, so the big programs, what they're buying, how they're achieving milestone decisions. And then, we in OSD particularly track the big ones, the ones we call A Cat One D's. We... OSD A and S has oversight of those big ones and then all the smaller ones, A Cat One C's and below, we monitor those to just... More or less on a capability scale and look to see how those capabilities are evolving and how those services are investing in them.

David Tremper (02:38):
So, I'm on the acquisition side of that, so I'm focused really on the acquisition of new capabilities, so those big programs like C whip and next gen jammer, [inaudible 00:02:47]. All those things are acquisition programs, so we within A and S and the EW office, we track those for EW and I as the director of EW really become a lobbyist for EW in that acquisition role. So, as we see acquisition programs and the services making changes to their EW programs, whether it's an increase in funding or a decrease in funding... Maybe they're divesting in a program. Maybe they're eliminating a program. I look at that from more of a DOD EW perspective to make a determination whether that is a capability issue for the Joint Services and then, given that, I act essentially as a lobbyist within the Pentagon to go around and draw attention to that change, whether it's a good change or it's a bad change.

David Tremper (03:29):
And then, as we go across the course of the year, we look at capability shortfalls. We look at where the threats are evolving. And, we take that information to senior leadership within the Department to give them awareness of what's the state of DOD EW. Where are we falling behind? Where are there opportunities? And then, we try to use that information to make wise investment decisions moving forward on the acquisition side.

David Tremper (03:50):
We do work with OSD R and E, research and engineering, to try to tie together new capabilities with acquisition. We help to work the transition in stories with acquisition... Or, with the R and E side. So, really, we're there looking at it from a let's build the capability, let's get it in the field. We work with the R and E folks to see what technology's coming down in the pike and how do we road map that into a transition plan.

Ken Miller (04:12):
So, you basically touched on all the different parts of a program development in some form or fashion. How do you keep track of all the competing interests? Because, if you look at EW, you have your systems but you also have your features or software or capabilities that are parts of other systems and then you have, obviously, your service interests. You have domain interests. You have OSD, civilian military balance of priorities. How do you keep all that together in one job?

David Tremper (04:40):
Yeah. That is a challenge and I would say you have to also map that to the broader DOD priorities and capabilities. So, I can champion for EW, but I have to be careful about making sure that I'm championing for EW given the priorities that the Department has, right? So, I don't want to walk in and say, "You have to get rid of this other capability and fund it in EW," when perhaps that other capability is just as important. So, there's an important balancing act that happens just there within the priorities.

David Tremper (05:07):
And then, as you go lower and you start looking at the programs and what they're doing and their budgets and you look across the services, it becomes even more complicated because what you will find is that as EW specialists... And, I'm sure many of the listeners would agree with this. EW's not platform-centric, right? It's not dependent on a specific platform. You can very often find EW capabilities. They could be used on an aircraft. They could be used equally well on a submarine at periscope depth or on a ground vehicle. So, we have to keep track of that too because the Army may be investing in something that is useful to the Air Force or the Navy and we want to make sure that we're being efficient with our investment. And, that's not to say stop redundancy. There is some goodness in having redundancy. There are applications, unique requirements for our defense systems. But we also have to be careful to make sure that we are not building the same system or the same capability twice.

David Tremper (05:57):
So, I have a staff of EW civilians and contractors and they have different parts of our portfolio. They're responsible for monitoring different programs. They're responsible for working with R and E. And then, we collect all that information to try to create the acquisition road maps, see what the quantities are that are going out there, make sure that those quantities are sufficient, make sure that the capabilities are keeping up, as programs age, making sure that they are remaining relevant, that they're not becoming obsolete. That's a challenge in and of itself. So, there's a lot of going out and talking to different organizations. There's a lot of interacting with intelligence community, interacting with other offices, interacting with the services, and then doing that balancing act of how you bring it all together.

Ken Miller (06:35):
And, over the years, there's been a lot of momentum, some progress, on reforming the acquisition process, making it a little bit more efficient, getting programs to the field a little bit sooner. I was speaking on a panel and someone asked a question about acquisition and I said, "Look, I'm not an expert on acquisition, but I only attended about two courses trying to explain it and I left more confused." And, someone from the audience says, "Well, that does actually make you an expert." So, when you're talking about acquisition, it's kind of hard to know where to begin. Where have we made progress in the acquisition of EW and where are some of the areas that we need to address more intentionally or are lagging behind?

David Tremper (07:11):
I think in terms of progress, we're starting to see some of the goodness from the S and T community regarding software-defined systems, open architectures for EW. We're starting to see that migrate from that S and T world into acquisition programs and you see this in places like the Army. Right? So, the Army, in one of their PEOs, is mandating the use of C Moss, which is an open architecture for doing EW systems. What that allows you to do is migrate capabilities and technologies from one platform to another, right? And, that, in and of itself, significantly shortens the development timeline for each of those programs because they can leverage the other program's investment. So, there is some acquisition efficiency and investment efficiency that happens through those [inaudible 00:07:53] open architectures and we're seeing that start to happen on the acquisition side.

David Tremper (07:57):
In general, you're also starting to see what we call middle tier of acquisition programs, which are an acquisition process for accelerating the acquisition of systems, specifically to accelerate it. Any time you say accelerated acquisition, you're going to absorb risk, right? That's what happens when you try to accelerate acquisition. That's part of the reason acquisition is so long, is that they're trying to tamp down risk as much as possible. Soon as you accelerate it, you start to incorporate risk, and one of the challenges there is that very often that means that you're investing in new technology in parallel with integrating that technology into a system. Right? So, acquiring the technology. So, you've shortened the timeline. You've accepted risk. That creates non-traditional budget profiles because typically you would see a budget profile for how the research and engineering is done and then how the acquisition is done and they would have a very distinct down slope for R and E and up slope for acquisition.

David Tremper (08:48):
What happens when you do accelerate that is you see these non-traditional budget profiles where they appear more to be parallel investment. And, as soon as you do that, right, you start to create alarms that trigger cuts. There's a recognition that that acquisition program is not investing wisely. There's what we call concurrency. And so, you start making cuts to programs because that budget profile is not a traditional budget profile. That's one of the challenges we see with accelerating acquisition, is that acceptance of risk and that non-conformity to traditional budget profiles.

Ken Miller (09:20):
How do you address that? Because even though there has been reform efforts, you still have silos across DOD. So, someone could be making a decision on a program based on some trend that they see looking at the program from one perspective and they might make a decision on that. It's like, oh, okay, well, you know, there's too much risk or it's not performing the right way, and make a decision. But they're in a silo that is separating them from maybe other criteria that is important for making sure that that system or that capability gets to the field. So, how do you cut down those silos or bring those silos together a little bit so that the right decision is made at the right point in time so we have a proper understanding of the risk that we need to accept?

David Tremper (09:58):
That's exactly what my job is for EW in acquisition and sustainment, is working across the programs of record, understanding where they're going, understanding what the trades they're making, as well as then working with the flag level leadership and the services to understand how they're balancing that capability against their other priorities. And, when we start to see mismatches and we start to see concerning situations like you just described, the decision that is made from a particular perspective that doesn't account for another perspective, we have to grab that. We have to do a quick analysis and then take that in front of senior leadership and say there's an issue here and get senior leadership awareness on that issue so that we can bring the data forward and do the analysis to determine whether that's a wise decision or whether that decision was made with limited information.

Ken Miller (10:41):
I want to kind of shift gears a little bit. In talking about managing the portfolios of EW, one of the biggest challenges that you're undertaking is understanding or managing the shift in DOD from thinking about EW to EMSO and this migration of thought and what it means for capability development and programs. So, I wanted to get your thoughts on how DOD is accomplishing that. You know, when you look at certain Venn diagrams and it shows, okay, EW is, you know, three parts. Electronic attack, electronic protect and electronic support. And then, you add spectrum management, it becomes EMSO. But it's so much more complicated than that because it's how those individual pieces fit together with adjacent capabilities in the defense world.

Ken Miller (11:27):
So, EMSO is a lot more than just combining things. It's how you relate those capabilities to other missions and so forth. So, I was wondering if you could pull that thread a little bit, talk about how DOD is transitioning from EW to EMSO. And, what are some of the areas that it's looking at in terms of where it's going?

David Tremper (11:45):
That's a great question. You know, one of the keys to that is that term EP. [inaudible 00:11:50] if anybody's heard me talk over the past year, you've heard me talking about the lack of real oversight on EP. So, when we talk EW in the Department, we're predominantly talking about electronic attack and electronic support. Right? EP is typically not in there and part of that reason is that EP is a feature that [inaudible 00:12:06] system puts in it or a PNT system puts in it or a comms system puts in it. So, the EW community is not informing those other communities, radars, comms and PNT, what their features are or really reviewing what their features are to determine that those are the right features to install.

David Tremper (12:22):
And so, what ends up happening is those acquisition programs will cut their EP features because it's a risk mitigator from their perspective. They don't use it for [inaudible 00:12:30] operations. They typically don't need it to meet their requirements. So, the EP is lost in radars, PNT and comms. Where we fall short is that that doesn't circle back to the EW community to say, "No, you need to go back and fix that." So, EP is not really being managed or monitored the way it should be. We've started to see that change over the course of the past three years and one of the ways is through this EMS survivability component that has shown up in the JSIDS process.

David Tremper (12:56):
So, JSIDS is the acquisition bible. It's how... The JSIDS manual is the book that all the acquisition programs have to follow. It directs how they do their programs, what they have to provide answers on. One of the new answers that they have to provide as of 2018 is how they're addressing EMS survivability. [inaudible 00:13:14] has shown up is now forced the radars, the comms, the PNT communities, to start looking at how do they address EP and then providing to the requirements review boards their plans to do that so that those requirements review boards can make a determination whether their EP is sufficient or it's insufficient.

David Tremper (13:32):
So, when we start talking about EMSO, there's a lot that is buried in that term EP, mainly because we have not traditionally managed it well, and so as soon as you say that EMSO is EW plus spectrum management, well there's now a hidden term in there that didn't used to be in there, which is EP. And, as soon as you open up that EP term inside of EMSO, you have to start talking about radar capabilities and comms capabilities and PNT capabilities. So, where we have traditionally talked about EA and ES when we're talking about Department of Defense EW and even electromagnetic battle management, for instance, in the EWX comm... Those are three heavily discussed topics. As we migrate to EMSO, what we're going to need to start seeing is an incorporation of what are my radar capabilities? What are my comms capabilities? What are my PNT capabilities that achieve EMS survivability and allow me to have EMS superiority in that contested and congested environment?

David Tremper (14:28):
And, that incorporation of those domains also becomes a balancing act because what we're trying to talk about is EMS survivability. There's a fine line between EMS survivability and capability. We don't want to overwhelm an EW conversation with a new radar capability conversation or desire for a new radar capability. But in that EMSO world, it becomes a delicate balancing act to maintain an EW conversation with those others.

Ken Miller (14:54):
How does that affect your ability to be an advocate? Because, you know, it's one thing to speak about, okay, we need EMS survivability in our equipment and in our other systems. But it's another thing to address it from the perspective of the radar, from the comms, and to say, okay, you need this one particular feature and you need it early on in development.

David Tremper (15:16):
Yeah. It's... It's extremely difficult and what we've been trying to do lately is work closely with the strat comm Joint Center for Electromagnetic Readiness, and that is a center that's existed for probably about the past 15 to 20 years. They were formerly known as JEPEC. And, one of their jobs is to go out and exercise EP on near fielded or fielded DOD systems. And so, they'll take out the jammers. They'll... They'll see what happens when comm links go down and they'll expose all of those vulnerabilities. It's organizations like JCER that start to provide awareness to the Department at large and the programs individually that they have an EP problem.

David Tremper (15:54):
And, what I have learned in talking to acquisition programs on the comms side and the C Two side is that they will design EP features in at the start of their program. However, what they discover is that not until they bring it out into the field do they realize whether those EP features are sufficient or not, and very often they're not. So, they have to go back. They're not resourced to go back and fix those EP features and very often they've already cut that field testing of EP that would enable them that insight. So, really, what needs to happen is a preservation of that field testing that allows them to understand the EP vulnerabilities so that there is awareness, right? So, step one is that awareness piece.

Ken Miller (16:31):
Does that mean we need to put more money or emphasis on testing earlier in development to make sure that these systems are being tested for EP so that we understand the vulnerabilities before it's too late to either go back or it's too costly to go back and fix it?

David Tremper (16:48):
I think EP needs to be a feature that's in the designs and awareness is established on how effective that EP is very early on from modeling and simulation, digitization of those systems, all the way through operational testing, to validate that the EP works. Otherwise, we end up with fielded systems that don't discover they have an EP problem until they're in the field, and at that point, we're trying to fix it with Band-Aid fixes and we can't achieve EMS superiority with a bunch of Band-Aids.

Ken Miller (17:15):
With this emphasis on EP, it touches on something else that, you know, we've talked about in the past. It's this drive toward multi-function systems. So, could you talk about how multi-function systems is changing the EW conversation?

David Tremper (17:31):
One of the things it does very effectively when you talk about multi-function EW, multi-function RF, is it inherently brings in spectrum deconfliction, and we've seen this on particular platforms in the past that I've worked on, where you have very limited real estate on which to put antennas and so as soon as you start turning communication systems on, even low power communication systems, you start interfering with high sensitivity receivers that have antennas that are close by. And, that could be on ground vehicle. It could be on a submarine at periscope depth. Because those have historically been stovepipe systems, creating that spectrum deconfliction between those systems is hard, right? Because those systems weren't necessarily designed to work collaboratively together and so you end up putting notches in and doing all kinds of strange things to try to get the systems to be compatible and interoperable.

David Tremper (18:20):
And, effectively, what ends up happening is you find out that physics is going to outmaneuver you and you're going to have to turn one of them on and one of them off to really get successful performance. What multi-function gets you is it suddenly allows you to control that deconfliction within the software. So, sharing front end antennas, sharing software, sharing applications, sharing digital components. It suddenly allows you to, on the fly, within the device, determine who has access to the antenna, how do I effectively deconflict those system. And, now I have very efficient use of the spectrum that doesn't require I go back and I put filters in and I try to notch out things on antennas. It makes it much easier to do that spectrum deconfliction problem.

Ken Miller (18:59):
And, this makes spectrum management all the more important in the conversation and probably increasing importance in the future as we try to address this growing understanding of deconfliction.

David Tremper (19:11):
Indeed. And, I think what happens is you have to look at the various requirements of the different RF systems that are using that device. A multi-function RF system that does comms and it does EW and it does Sig Int... Maybe it even does PNT. Well, those things all work well together because of their analog power requirements. Soon as you say I want to do a radar... If you're going to say it's a high power radar, well, high power wants narrow bandwidths and so it challenges your ability to do EW on that system. There's the balancing act that happens when you're doing these different applications into a multi-function device.

David Tremper (19:42):
There's also an acquisition challenge that happens because we don't acquire RF functions as functions. We acquire them as subsystems or even systems. So, when we start talking about multi-function, it challenges the acquisition paradigm because now what we're asking acquisition programs to do is to share a piece of hardware instead of building it themselves. Right? So, now you have a comms acquisition program providing software in wave forms to potentially another acquisition program, and that's challenging our historical acquisition paradigm.

Speaker 3 (20:15):
Providing full spectrum superiority across all domains. That's defining possible. Giving war fighters the freedom to act across the spectrum, especially in highly contested battle spaces, can seem impossible. At Northrop Grumman, we've pushed the boundaries of possible across the spectrum for decades. Today, our cutting edge interoperable multi-function technologies are a revolutionary leap in spectrum dominance. How revolutionary? Imagine detecting the precise location of an adversary long before they ever detect you. Or, better yet, denying or degrading an adversary system without them being able to do a thing about it. Your freedom to shape the spectrum is an overwhelming advantage in every domain, an advantage made possible by Northrop Grumman's unique software-defined, open, safe, secure, architecture solutions.

Speaker 3 (21:03):
It's all part of our commitment to ensure our war fighters have full spectrum dominance to make real time decisions, no matter the environment. That's defining possible. Learn more at ngc.com/ew.

Ken Miller (21:14):
So, you mentioned acquisition of multi-function systems and I want to kind of go back and talk a little bit about budget because when we talked in May, the president's budget had just been released and [inaudible 00:21:26] we've gone through a summer of Congressional hearings and everything of that nature. And so, today, a few weeks from the end of the fiscal year, how does the defense budget address spending for EW in light of multi-function systems? What are some of the things that we need to be keeping an eye on? And then, kind of related to that, in a hearing earlier in the spring in the Armed Services Committee, it was mentioned that about 40% of the budget, EW budget, is focused on about 10 programs. So, how does that influence what you're trying to do as the director of EW in OSD in the budget process? What do you contribute to this oversight process between Congress and DOD?

David Tremper (22:08):
We look at all the different marks that show up in the bills as they're making their way through and then we try to determine what's the DOD impact, what's the joint capability impact of those marks on those specific programs. And then, we work through the Office of the Undersecretary for Acquisition and Sustainment to identify things that we need to go back and put what's called a reclama in, things I've raised a concern about that say, "Hey, this... This is going to have a joint impact. We need to be careful of it."

David Tremper (22:34):
We also reach out to the acquisition programs that are affected and we ask them what's going to be the acquisition impact of these cuts or plus ups that they get, and then based on that response, we attempt to represent that response from more of the purple OSD perspective as opposed to the service specific reclama, because they will also submit service specific reclamas to the cuts that they see and then we'll do that from the OSD side.

David Tremper (22:56):
And then, from the DOD EW wide, we collect all those cuts, we collect all those marks, and we try to make a determination of what's the broader impact. And, I think one of the challenges that we see there is that if you look at individual programs and you assess those individual programs, you'll find programs that need to have money taken away and so cuts will be made. But you'll find that when you add up all of those cuts, you end up what I like to consider death by a thousand paper cuts, that a lot of small cuts are going to have a significant impact on the DOD-wide investment of EW. And so, when you total all those things up, we have... We end up with a much bigger number than what those individual programs are.

David Tremper (23:32):
So, we have to be careful of that, too. So, we pay attention to that and when we see large numbers come from those thousand paper cuts, we then go to the Undersecretary of Acquisition and Sustainment and we inform them that there's a large amount of cuts here and they should be looked at collectively from a DOD EW perspective to make sure that we are not effectively divesting in EW by cutting all of these individual programs.

Ken Miller (23:53):
On Capitol Hill, there's a group called the Electromagnetic Warfare Working Group. A number of members of Congress part of that group, that they've taken it on themselves to basically be champions and raise awareness on the EW matters in Congress. And, one of the things that they've said over the years is that when Congress is looking at the budget, it's basically a microcosm of DOD. They're looking at it very program-specific. They're not able to see the justification or the perspective used for making certain cuts or for even tracking how EW is spread throughout the budget. So, there's this need for a Rosetta Stone concept where you can actually somehow figure out, okay, here's what's funded. Here are the cuts. Here are why the cuts are made and what the impact is.

Ken Miller (24:34):
How can DOD and Congress kind of address that moving forward so that you don't have multiple bodies making budgetary decisions without more comprehensive perspective on what's going on for capability like EW?

David Tremper (24:51):
That's a fantastic question. I would love to kind of create an annual cycle where we do something like that, where we work with the Hill and we talk about what we've perceived the cuts or what the potential impact the cuts are. I know that in a recent NDAA, there was a request for an annual budget sufficiency report that looks at a wide variety of EW programs and what's their budget for those programs. And, it's more or less a snapshot on here's what the Department considers their EW programs and here's what the investment [inaudible 00:25:19] on those programs. The taking the purple look at it and saying, okay, if you cut this program here and you cut that program here, here's what happens to the acquisition schedule, here's what happens to the [inaudible 00:25:29] delivery timeline... I think that's something that my office is trying to pay close attention to. That's why we put reclamas in when we see cuts. I would love to have that conversation and make it annual cycle with the Hill and talk about here's what we think are going to be concerning issues if they are cut.

David Tremper (25:43):
Currently, we do it through DOD leadership, which I think is probably the preference for the DOD. But we're happy to have that conversation about what we perceive the impact of cuts to be.

Ken Miller (25:51):
I want to move on to another topic that's obviously closely related to EW and multi-function systems, and that's the rise in use of collaborative systems, artificial intelligence and machine learning. And, want to get your thoughts on how that is changing what we're able to do on the EMSO front and what are some of the developments on the horizon that are really going to shape our ability to achieve EMS superiority.

David Tremper (26:18):
In a previous life when I was at DARPA, I managed cognitive EW program, AI, ML, of application to DOD EW. My background is S and T. My first engagement with really acquisition side is now, now that I'm in acquisition and sustainment side of the program. There's a lot of challenges with bringing AI and ML and my fear has been the observing a lot of hand waving about cognitive EW will save the world. Right? You don't need to worry about that because cognitive EW will take care of that. I think there are a lot of opportunities for cognitive EW and AI and ML, particularly shortening decision timelines, shortening the time it takes to create or to adjust a system or adapt a system to recognize a threat that has never been recognized before. I think there's been a lot of work at DARPA to look at how do you use machine learning and AI algorithms to do that and look at the class of threat rather than specific threats and use that to determine what threats am I looking at.

David Tremper (27:11):
I can tell you the challenges that I see as we approach acquisition is the types of things that are required for caring and feeding for AI and ML algorithms are not traditional things that acquisition has to deal with. Right? So, when an algorithm currently shows up for an acquisition program, they integrate it. Like, just a generic algorithm, not even a AI or ML one. Maybe a piece of software. It's integrated into the system. It meets the requirement. It tested and then... And, effectively, it goes out and it's fielded.

David Tremper (27:39):
When it comes to AI and ML, you start to have to think about lifelong learning of those algorithms. They're going to have to learn. There's a trust issue that's established with the operators. They go to schoolhouses. They learn how to use systems. They recognize how those systems work and what the limitations or opportunities are. If you've got a cognitive system that the system may be changing its mind potentially over the course of time based on data that it's collecting, the operator's potentially going to become confused and not trust the system. The operators need to start learning how to trust the systems. We need to learn how to train the algorithms because right now whenever you do a cognitive or AI ML type of program in EW, you have to go out and find data. You need to go out and find data to train those algorithms.

David Tremper (28:21):
When I was at DARPA, one of the things that we experienced when we would pitch an AI or an ML program was that the very first industry day where we would talk to industry about the program we were working on, the potential vendors would be very excited about it and the first question they'd ask is where's the government furnished equipment data that they need to train the algorithms that are going to be used in the system, and that's a challenge. Finding that data's a challenge, so we're going to need to start to think about data warehouses where operational data can be housed so that algorithms can be trained. There's a whole new logistics infrastructure that I think would be required to really integrate, acquire, maintain and sustain cognitive systems.

Ken Miller (28:59):
You mentioned trust and it's a very interesting word and... Because, you know, in order for you to trust something, you have to assume that there's at least some adherence to familiar rules, that you kind of can model a particular outcome. And, what we're finding, it seems, with AI and machine learning and deep learning systems, is that it's challenging our assumptions. It's challenging everything that we know and it actually in many cases doesn't follow the rules that we have set out at the beginning. Are we actually missing out on all the benefits we could obtain from artificial intelligence by trying to fit it into our assumptions?

David Tremper (29:36):
Yeah. That's a huge challenge and this gets back to the data thing. As soon as you try to provide data to train an algorithm well, you have to groom that data. And, to be efficient, you really want to groom that data, label the data, provide it in such a way that the algorithm can learn. But as soon as you start manipulating the data from a human perspective, you start to limit the potential opportunity that the machine has to make discovery with the data. So, from my perspective, live operational data is invaluable to training algorithms. It falls on the floor in our operational systems right now, it's not... It's not really being recorded. It's not being stored. It's not being warehoused. So, what we end up with is we end up trying to create data either through modeling and simulation or some other mechanism. And, as soon as you start to artificially create data, you start to run the risk of removing the features of that data that the algorithms could use to learn even better than the human has learned from.

David Tremper (30:30):
And... And, there's opportunities out there to make noisy data, to try to make realistic data through GANs, or generative adversarial networks, things like that, to try to create data that is not groomed per se. But again, the challenge is that AI and ML greatly benefits from its ability to not think like a human and as soon as we try to provide data that has been processed like a human or how we've traditionally processed it, then we start to lose out on the advantage of some of those systems.

Ken Miller (30:58):
You mentioned you're trying to train the algorithm or the system through the use of algorithms. You know, in the past, those decisions of how you collect and share data are all at the senior military decision making level, whereas now we're talking about having to write the algorithm to train the system, which really drives the decision making or the opportunity space down to the programmer or even early on in the development of the system because if you don't have the algorithm right or if you don't understand the algorithm that you're trying to build for the system, you're not going to get the data you need. How does the acquisition system adapt to this driving of decision making down to the algorithm level?

David Tremper (31:43):
That's a great question and I think we're going to have a lot of learning about that over the next few years as we see a large number of AI and ML types of programs evolving from that S and T world into the acquisition world. This is where I think it's going to be a challenge for acquisition and we're lining up studies to look at what's the acquisition impact of AI and ML. Right? What is the logistics implications of having training algorithms, fielding algorithms, sustaining algorithms? What's the impact on operator trust? Do we have to have AI and ML training in schoolhouses at the same time the operators? So, effectively, the AI and ML algorithms would be virtually learning at the same time that the operators are learning so that you establish a relationship between the operators and the algorithms.

David Tremper (32:27):
There is a significant challenge, I think, to how we bring AI and ML into acquisition for all these reasons, that cognitive systems almost have to treated as if they are human operators learning, and that's not the way we acquire systems.

Ken Miller (32:40):
Because a lot of times when you talk AI, you talk about having a man in the loop at some point in the decision making process. What we're learning about deep learning now, it's more of like a man outside of the loop trying to understand that loop because it's moving so much faster than a lot of times we can even understand and we have to trust something that we've built to make decisions or to model our world in a way that we might not really fully comprehend. So, it challenges our assumptions going in and how we can make sense of the decision making process of the machine.

David Tremper (33:13):
Yeah. That's right. And, I think for EW, you particularly want to have the human on the loop and not in the loop, right? So, one of the big advantages of AI and ML for EW is that EW's a speed of light operation. A few bits that get out there out of control could reveal a lot of information about the battle space, and whoever controls that awareness of the battle space is going to effectively win. So, when you get down to the real value of AI and ML for EW, it's in managing that speed of light fight. So, light's traveling 180 plus thousand miles per second. It's instantaneous, for all intents and purposes. So, when you think about how do I make a decision on information that is moving instantaneously between nodes in a battle space, you can only really manage that, the control of that information, using machines that are effectively making decisions at the speed of light.

David Tremper (34:01):
So, in that regard, you can manage strategy. The human can manage strategy. The machine can manage tactics. And, you really end up... And, the real value I see for those systems is in that speed of light spectrum warfare.

Ken Miller (34:14):
That's all the time we have for today. David, thank you for joining me. Great conversation and I'm sure it would be great to have you back on the show again in the future. I do appreciate your time joining me this morning.

David Tremper (34:25):
Absolutely, Ken. Thanks. Any time. I'm happy to come on and talk.

Ken Miller (34:28):
Thank you so much. Appreciate it.

Ken Miller (34:30):
That will conclude this episode of From The Crows' Nest. I want to thank my guest, David Tremper, for joining me. Also, please join me next week for our next episode coming out on September 15th. I am joined by Mr. David Weinberger. He is the author of Everyday Chaos: Technology Complexity and How We're Thriving in a New World of Possibility. Also, I want to thank our episode sponsor, Northrop Grumman Corporation. Northrop Grumman's multi-function interoperable solutions create full spectrum superiority for our war fighters across all domains. Learn more at ngc.com/ew.

Ken Miller (35:02):
Thanks for listening.