Insights from the Space+AI Summit

Good morning. Uh So as Jenny said, we're gonna talk here in our first panel about artificial intelligence and how it applies to space Domain awareness. And what I first want to do is introduce our three panelists starting with Lieutenant Colonel Ashton Harvey. Colonel Harvey is currently the Chief technology officer at the national work ons office specifically in the inw ground Systems program office. Prior to that, he was a services chief fellow at DARPA and he's held many roles across the Department of Defense and the Air Force. He also holds a Ph.D. in engineering and operations research from George Mason University. Welcome, Colonel Harvey. Next up is Major Sean Allen. The major is currently at Space Systems Command where he has the honor of being the inaugural chief of the SD, a tap lab. Prior to that, he was a mission director at the NS DC. He's done a lot of work on hardware and software prototyping and when he was at the Space Security defense program, and he also worked a lot on OPIR and SAC systems. Welcome, Major Allen. And our third Panelist is Doctor Pat Biltgen Doctor Biltgen is a leader in Booz Allen's Artificial Intelligence Group where they apply artificial intelligence across the defense intelligence and space clients. Uh He spent more than 15 years in the defense industrial base working on lots of national security missions. We have the honor of realizing that he just released his book on AI for Defense and Intelligence that's out there on Amazon. So I'm sure we're all gonna quickly pick up a copy in the lobby or rush out to read it so we can talk to pat about it. Doctor Biltgen holds a bachelor's, a master's, and a Ph.D. in aerospace engineering from Georgia Tech. Welcome, Doctor Biltgen. Thank you. So, before we dive into this topic of artificial intelligence, I want to say, it's fascinating to me that we're here talking about this. Growing up as a kid prior to the internet prior to cell phones, mobile phones, as we know it today. UmArtificial Intelligence. The only thing we ever heard about it was when we were watching Star Trek every week and we heard Captain Kirk in the, and the group talk about this all and powerful computer with all this intelligence on the enterprise. And so I find it amazing that we are living it now. And where they were foreshadowing in TV series and movies about what artificial intelligence would do in space. This team and the, the team across the nation is making it happen. So we're gonna talk about what is that making it happen, and have a little bit of discussion of what it is and where we should be focusing. So we're gonna dive in, the first topic I'm going to ask Major, I want to start us off with and I think we start with the basics, since this is a very complex topic, umand we only have a short amount of time. Major, maybe you could start our discussion a little bit about talking about just the basic foundational needs that we have in the government with regards to AI. Before we start diving into maybe the solutions and the priorities. OK. Thanks, appreciate that. Something I will say is I'm the only person here who doesn't have a Ph.D. So I, I have done, you know, some technical program management, but I am an operator. So all of my perspectives are me sharing the observation about how we implement AI from the operator's perspective. So, one question that we were talking about earlier just before we came in, why don't we see widespread adoption of AI, you know, ChatGPT took off this last year. This is not new. What makes it hard to adopt from in the operational context Instead of giving you the list of reasons why I think it, you know, it's hard to adopt. I saw something change when Colonel Raj Agrawal took over this last summer as the space based Delta Two, the SD A and battle management commander. He made one simple change in how he described kill chains. But one of the buzzwords has been, let's get the kill chains to close. We've said that for several years. He turned that and he said my priority to avoid operational surprise is to detect the start of a kill chain. And for whatever reason, that small change in language has converted very well or translated very well to software engineers, machine learning ops, folks, data scientists to say I can do event detection. That's something that I can measure. And so, that's where I've been focused has been on. What does it mean to avoid operational surprise by detecting the start of a kill chain? And then how can I break that down into a set of tasks that we could implement AI to help us with? So is that getting at it no small challenge, right? Is any of your others want to add to that? Well, well, Tony, one aspect would be you know, it's major Allen's point if you're trying to detect those early precursors, uh the adversary is trying to prevent you from doing that. And so a lot of their signatures are very weak signatures or they're trying to deceive you with certain things and humans tend to have cognitive biases or preconceived ideas of this is what I think that means. And the enemy knows that too. So an area where AI can help in trying to identify that back to the comment about big data is that it's really good at finding latent patterns and weak signals. And OK, it may not always be right, but it can at least suggest those to the operator and say here are five things that I thought were weird and the operator would go, oh, you know what, I didn't notice four of those. And that's a very common thing that we see in a lot of cases. in almost every domain where the AI will find non intuitive things, the human usually dismisses those as the AI being wrong. And in many cases, actually, it was just something we never imagined. And in that case, Pac maybe you could talk about or others. So as I understand it, AI really in some ways has the potential to be more powerful in the human brain, but it needs time, right? It needs data to learn and takes time. So how does that work for and against the desires? We have to speed up and improve the kill chain and speed of the mission at the same time? Be accurate. So I'd like to share my thought on that. If you can't clearly define what those objective functions are that you're trying to train a model to detect. then more data is not gonna help you more training time, more CPU hours, whatever is not you, we need to think more clearly and be more specific in what are the tasks that are required for battle management. So I think one challenge in the Space Domain Awareness ops community is it's complex. There are many, many things that must go that, that we have to do correctly to, to achieve space superiority. But then it had, there's a tendency for it to be all things to all people. So if SDA is all things to all people getting more AI sprinkled in on top of that and hoping the operators are gonna adopt it, it's probably not an effective strategy. So I think Colonel Gerald's comments about, you know, the way that we are going to achieve space superiority, avoid operational surprise means we first must detect the start of a kill chain. That, that is a slightly different way of thinking about this problem because now I can say how are, what are the finite list, the number of ways I could be surprised what are the specific attack vectors by which an adversary can um can, can come at me, right? And now we can start saying to detect an event or what are those weak signals? Those may end up being indicators that an adversary is attempting to surprise me by mimicking a payload when he a different type of system or pretending to be debris when he's actually a payload. Those are the, those are things I think we can actually specify get data train models and incorporate them into battle management functions, right? Yeah, I think that's a great point. So well, Sean is more on the operation side and I've spent many hours playing at phone calls where he's out on the floor trying to work through different problems. I sit in a spo and we build things and so I try to think about how do I create a problem that I actually can solve? Someone hands me a stack of nebulous requirements. We need, we need more things, more, faster and more better. How do I turn that into something I can actually turn around and give to folks like Pat and say, OK, here is a decomposed problem. Here are objective measures, here's where you're going. So I think setting, you know, your example is Colonel Ol setting a good framework to scope a problem to give shape to it and to decompose it into subportions that make them chewable, understandable problems that reduce the the mental complexity of trying to frame the problem for someone. So I don't have to solve all of SDA I can focus on. How do I detect that this event is about to happen? How do I detect that a maneuver has happened? How do I classify that? A maneuver? It is non nominal. How do I do these different types of things? building that systems engineering structure around decomposing the problem providing good IC DS API S between those problems allows you to scope it down to a manageable size where people can really start to chew on it. And I think that that's, you know, one of the things you've seen out of space systems command and the NRO there's been a lot of thought leadership in recent years trying to understand how to decompose that problem well and communicate that to our industrial base. So that folks like Pat or others out there can actually put real code behind that and turn that into approaches that can automate things that are typically manual processes. And as we have trusted automated methods, we can then get to the point where we can gain the operators, operators trust to institute very understandable AI approaches to solve those problems where it's appropriate. Yeah. And, and Tony, I think that's a critical point. I'm glad that Major Allen used the word Sprinkle. That's, that's a word that we use a lot is like, oh, let's just Sprinkle a little AI on it or like, let's do AI but to the colonel's point, it's very important. Like systems engineering has fallen a little out of favor cause you're like, oh it's got these processes and I got to decompose these things and let's just prototype it and see what happens. There is a place for that about getting things in front of the operator and saying, what do you think? But you spend a lot of cycles bringing the wrong rock if nobody can articulate the problem. So most of the systems that are successful in the Department of Defense are where the operators and the nerds can work together and say, hey, I got this requirement. It passed down through Colonel Harvey's spoe and was mistranslated 19 times from the original person you talked to. So now that we see it, what do we really mean by maintain custody of object? Let's figure out what each word means. Sounds really pedantic to say we're gonna do a definition exercise of each word in that sentence. But then you understand, like to maintain custody, you need to see it all the time, one shot every five minutes, you don't want to lose it for an hour. Like help me understand what this word means from an operational standpoint because a lot of those nerds don't have the operational experience. And so we can imagine from a physics or math standpoint, this is what I think they mean, but those systems are never successful, you have to talk to the operators and say, show me what you're doing and let me try to understand how I might make your life better. And it sounds like what we're, we're talking a lot about here is the challenge that both government and industry have working together really is to, to harness the technology push that's going on in artificial intelligence, but responsibly apply it in a rigorous methodology. So we can get incremental advances to our operational community to help the mission, right? Not just spend a lot of time and money, just pushing the edge and hoping something is gonna give us because we're learning, but we also need tangible outcomes fast. So I think that leads us into the next kind of topic to dive in just a little deeper. And, and that is, is where do we see right now? And in the coming years, some areas that you think that AI is particularly suited for Space Domain awareness, you, you started talking a little bit about the fact that there's so much out there we want to collect on. and it'll be more and more as time goes on and part of this will help us to make sense of it and then decide, you know what to do in the back end. Can you talk a little bit about? and maybe kind of Harvey lead us off here is where do you see some specific areas that maybe not big art requirements but smaller requirements we see evolving that can point people that are trying to bring solutions to the table. Yeah, absolutely. So, I think Sean again, you know, had a really good insight there about defining the objective function. If the government is not clear with what we want. And I, and I don't mean, you know, in words, I mean, in, in the mathematical where we're trying to go, what are the numbers that we're trying to hit? What are the things we're trying to do? Folks will never really get there, right. So you see that with there's been a lot of research post public public publicized in Amos around sensor tasking. A lot of it has been focusing on how you do sp tasker better, right? You know, it's, it's I, I want to maintain objects in space that aren't moving and you know, don't lose them. Well, that's not super interesting. We can do that. We have, we have an algorithm that can do that. There's a lot of other things that don't have good algorithmic approaches in that sensor tasking world that there are other tasking types that the 18th SD SS is defined to turn those into actual objective functions to then bring together what is the value of going to look at something after it maneuvered to refine that, that state versus what's the, the, the, you know, the, the resources we spend to do characterization shots on an object that we're not quite sure if it's, if it's actually what it says it is turning that into an objective function will help folks actually really solve that. But until that happens, folks are gonna spin on that, we've seen a little bit of good research at Amos where people started to some of them, but not on all of them. Kind of moving away from the decision problem of sensor tasking you look at at classification and we talked a little bit about like AI algorithms are really good at looking about a lot of a lot of streaming data and saying this is a change detection. This is, you know, this is not nominal, this is nominal. There's a lot of places where that can be a really good opportunity to apply AI techniques to those. Um And then, you know, just because I feel like I'm required to talk about ChatGPT. You know, there are a lot of opportunities to leverage large language models to look at unstructured data that otherwise we couldn't pass into an algorithm very easily to potentially pull out insights whether that be news reporting in foreign languages because they can read the foreign language, I can't. And or that be, you know, looking at Twitter posts, media, video, turning video into text that can then be searched to be able to look for other left of launch indicators that might tell me I need to spend more cycles looking at something. Can I jump on? Absolutely. It's my opinion that if you want your innovative new research prototype AI thing to gain adoption in the operational setting, it has to be very clearly attached to a battle management function. So if you can't articulate that in a short sentence, then the rest of the research paper is gonna go unread. So one place where I see missed opportunity is with regard to anomaly detection on orbit. I think there is huge opportunity to make use of existing algorithms that but they're not well integrated because we haven't yet described to the operational community. This is why this type of anomaly when detected gives you decision making power in your protection of an asset. So one example from this past past three months in the lab, if somebody tells me that an object is stable or unstable, that's that's somewhat trivial may not require machine learning. But it might lots of time ragged time series data from heterogeneous network of unc calibrated optical sensors. If you wanna know, you know, the sparse data, tell me quickly whether that target goes from unstable to three axis stable that change. can tell me something very useful about the intent of the target and then I can communicate to a battle manager if objects on this list become stable, that may tell you the next proximity event is potentially hostile, right? So being able to answer those operational questions in that language, not in machine learning language, is gonna help people to adopt your, your whatever it is. If you got some cool long, short term memory thing or whatever, that's doing a neat ragged time series processing rock on, tell me whether or not this target is potentially hostile and if I need to take action against it. So and it sounds like in the example and he brought up ChatGPT, right? We're not talking about at at this point where a lot of people are worried are, are we using AI to replace the human? We're talking about using AI to enable the human? We want our operators to be able to work on things that that they should be spending time on and let now this AI engine start to really deal with volumetrics of the data. And, as you said, merely help us with the back end decision making course of action development and, and things like that. Pat, have you seen any other areas than we were just talking about here? That, that you think really fits this, this near term application of AI. Well, you know, ChatGPT and I are very good friends. So thank you for asking the question. One of the things that's interesting about, OK. So that is a class of algorithms called large language models that has enabled a lot of new applications. But large language models are probabilistic token generators. OK. What they do is they take language and they say I'm breaking these words up into tokens which are parts of speech. And the big breakthrough, the large part was that if you give it enough human language, it says, oh, I figured out how nouns and verbs are related and I figured out how certain facts are related. And by the way, it doesn't always get it right. But it says I can predict the next token in the sequence. One of the things that surprises people is um is its ability to do things for which it hasn't been trained. So I have an example in the book where I said chat GP T here is an Excel file that contains ship movements, find transshipment events. So transshipment events is when two ships come together and they offload something to transfer them to each other. And it says I found six transshipment events and here they are, but you go, it shouldn't know what that is or how to do that analysis yet it produces six events that are correct. And so the researchers have found like it's predicting patterns that are not all based on language. So in space domain awareness, you could use this to take a series of motion patterns and treat them all as tokens and then say, can you predict the next token which would be the future position of this spacecraft? Now we have physics based ways of solving that problem. You go like, hey, I know the math, but it's possible that these algorithms are going to find different math. And some of the companies that are doing um motion prediction for self driving cars instead of doing the kinematics of here's my velocity, my acceleration, my angle and I should be here. They're treating it as a token prediction problem and using large language models to guess where the vehicle should be in the future. So that's an example of a domain where you go like, but you shouldn't be using text based language models to drive a car and some researchers are going well, they work. And as you know, there's like a lot of things in our society where I don't know how that works and someone tried to explain it to me and it doesn't make any sense, but it does seem to work. And so I know that doesn't give us like a good feeling as technical people. Like when Major Allen goes, can you explain to me how this would work? And I go like sir, it's preventing operational surprise. I have no idea how it's doing it and I can't explain it. But have you felt surprised lately? And so that's a kind of a, a weird way of looking at the problem. I mean, like the kernel that checks my requirement is going, this is not going well for you right now, but we're entering a new domain where we're, we're saying like these are algorithms that are solving problems and we may not be able to understand why. One less side, there's a lot of chemistry that we don't understand. Like when you do chemistry in high school and you're like, I have this reaction and then it goes to this. It wasn't until I was in grad school when I was in a combustion course where um Professor Lewin at Georgia Tech was like, by the way, there are hundreds of precursor reactions that happen between those two things and we can't actually observe them like we know that they're there. But we simplified this for you in undergrad and said this turns into this, but there's a lot of other stuff going on that's unobservable by any sensor. And, and that may be true in medicine. That may be true in psychology. That may be true in how I fill up my expense reports. So all those things could be um could be like totally new domains for us where AI could solve a problem. And we really like explainable AI like it's a great buzzword. It's a great concept but you go like we may have to realize like we can't explain it, but it works. So how all, all three of you? So that's a really good segue into. So we are pushing the boundary here, right? Even though in a lot of cases now we're we as humans are trying to develop AI to augment what we do our human operations. We usually use our, we hold our human brain and analysis the gold standard. How do we know as we develop these A I solutions and hundreds and thousands of contractors working with all different parts of the government on different solutions. How do we know or how are we going to work towards trusting? And I don't mean trusting as in, you know, the, the ideas of Terminator, it's trust us that what we said it was going to do meets the outcome. Um especially when that maybe that gold standard is the human and how, how do we now measure a machine against the human performance? So if we talk a little bit about how we see both government and industry embarking it down, you know, we've talked a little bit about the invention of AI, but now how do we get into that operational side of how do I know I can trust it that I can step back and you're not worried about what happens, maybe not tomorrow but a month from now or as it learns, are we feeding that back? So yeah, I'll throw in an opinion. Even if you're doing something super novel, very interesting, you know, whatever large language models for orbit determination, we do need to measure the outcome and evaluate the performance against some benchmark. And there's 50 years of good data science standard practices on how to do that. And in the space domain community, I think a lot of the, the bottom of the data science hierarchy of needs goes unmet. In many cases, access to data, publicly available, expert labeled data sets so that people could throw some spicy new hotness against it and go OK. I knew that there were six ship transit things in this data. how would you know that there are six unless an expert who knows the answer is again, it's supervised kind of a technique. But I'm not the expert in AI. It seems likely that the next few years, we're still gonna see a lot of growth in supervised machine learning techniques. And our community has done a poor job, could do a lot better in solving the bottom of this pyramid. Like let's go get uh all of the commercial G pr data related to a finite set of class label events, maneuver, proximity RF changes, attitude changes, launches reentries, whatever those events happen to be, make those easily accessible to the machine learning community with nice clean labels on them. Like I, I would argue that that's a nice place to start. Then I may not actually care or what. Maybe I won't worry is the right word. Whether you're misusing some cool technology, at least I can measure the performance. And then I can tell you whether I'm gonna spend a nickel on it. So the data is really important. I mean, sorry, go for it. So the data is really important. You make a great point this morning, you made a post about the EM n uh handwriting digit data set that came from back in the nineties. Almost all computer vision algorithms were advanced because of a data set called image net, which was 100 and 20 million images that were just scraped off the internet. And but they were labeled using a data set that was a taxonomy of it was a whole taxonomy of words like it goes down animal mammal, cat and then what type of cat? So the people that were labeling, it had a set of words from a data set called lang net that allowed them to label the images that was a major undertaking. And you go like well, gotta start somewhere. Uh On the satellite imagery side, the government released a data set called X view, which is uh a data set of a million images in 60 different classes that almost all the satellite algorithms have been trained against. And so at some point, you have to, you have to say like how much money are we spending, trying to tune up algorithms against bad data when we could just go, let's just go take the however long it takes five years to go create a gold standard dataset and then you're going to see that progress. And by the way, if you don't believe me, Tesla built the world's largest machine learning supercomputer and they're sucking data off millions of Teslas sending it back to the mothership. And they're going like every time you're in autopilot and you go like this and it goes boo boo and disengages autopilot, you just labeled the data, you said whatever the car was doing, the human did not think was correct. Their data says actually they're right more often than we are. Jury is still out on that. Literally, the jury's still out on that one. But um is that also while my insurance bill keeps going up there? It's you Tony actually remember to your original question, how do we trust it? You know, 50% of us are below the median at any task. So that means like two of us are below median drivers aggressive. OK. So, and then lastly, the trusting part would be if you have that labeled data, that is a gold standard that people have validated, that the algorithms can be run against trust comes with time. And the really tricky thing about A I is, even when it's right, 99.9% of the time, um, this is a phrase DARPA uses the phrase statistically impressive but individually unreliable. It's like you see the one example that's really obviously wrong and you're like, oh, this is never gonna work, but it was right. So many times that you didn't even notice and the one time it's really, really, really badly wrong, like, none of us have ever made a mistake. It just goes like, ah, this is never gonna work. But when you have trained operators, they go, I just can't accept that. And now you're building your way back up from the, the pit of despair. Right? Yeah. So, uh uh for one, it feels like my parents in that sense, right? The one thing you did wrong, they remember. Uh also on the note of trust, I did make a note to review Pat's expense reports for unobservable data. So we, we'll get around to that. Uh Actually Jenny approves them all. Oh, there we go. They're probably fine then. Um So, uh you know, I want to circle back to, you know, uh one of the first interactions I had when uh Sean stood up the the tap lab, was, hey Sean,uh I think you're gonna do great things. Also, I spent a year plus with a bunch of really smart optics people and really smart ma mathematicians trying to build a high quality simulated data set that I knew ground truth because I often I have great space data and I have no idea what ground truth is because we don't actually instrument a lot of the satellites or I don't have access to the GPS receivers on board all the satellites. So I don't really know what truth is and I have no GPS data for debris. So I'm just guessing what, what was actually going out there. But with high quality simulated data, um there's tradeoffs, I I all models are wrong. Some are useful. Hopefully this is a useful one. You know, I really know what ground truth was in the simulation because I made it. So my first thing was take this, it, it, it's not safe to go. Don't take this. You know, we, we to be a good partner. I need it to be able to understand. as his folks are coming up with new innovative ways of approaching problems. What does that look like on a, a data set that I've seen other people work on? I know what the performance is. And I think your point about image net about putting out challenge problems with good data that clearly define a problem, scope it down and say I want here is a, here's a whole bunch of data home of investment. I want you to do this really well. can, can really issue a statement which small teams that can't afford all of that upstart cost that can't build Tesla's supercomputers and get pat to train it every day with with his driving skills to, to be able to reduce that start up cost and allow them to work. So I think, you know, that that is an area in the space domain awareness problem set where we, we have some under investments and a little bit of that is because of classification, you know, ask, just ask Os D Plumb uh about right, Assistant Secretary of Plum about overclassification in the space world. You just made a bunch of statements about it. So, it, it is a challenge. And I, and I, you know, I, we, we talked about two ways to get around it, commercial data is not beholden to dod classification guides, simulated data if done, right? Can sidestep a lot of those problems. So there are ways around it. We just need to find a good, effective way of doing it and establish those problems that are tied to operator needs that can then have good verification validation behind them to build that operator trust. Did you do an OG legend of Zelda reference? Yes, that people like me will not recommend somebody in this room caught it. Yes, I know, I'm sure. So let me, I wanna touch on one more topic before we get a chance to let the people in the audience and virtually ask us some questions. But we, we're delving into different aspects of the human play in this. So I wanna go down that path a little bit more. So let's talk about the workforce like we saw over the last 10, 15, 20 years where um the services and the intelligence community had to really address how do we evolve our workforce to deal with the new cyber technology. Now we have artificial intelligence. How do we see this on both the government and new side? Really pushing the boundaries of our workforce? Are we going to be able to keep up with it? Do we have the people, can we recruit? Is it affect, you know, skill codes and things like that? Because we know the commercial world is pushing on it very heavily. Um We know we have our own classified issues with the workforce and how it limits things. So can we talk a little bit on that, that part of the human element when it relates to the workforce and applying it to the technology? Um especially when we're working in a sensitive topic area like SD A? Yeah, I got, got a thought on that. So I think uh those very technical skills, a literacy and AI software development, cloud computing, all, all of these various technical skills even in technical fields, not everybody is A, you know, enterprise software engineer or a cloud computing expert. Those are those 10 are, these are partitioned out skill sets. So if we want adoption of AI technology, we want trusted data sets that have been scrutinized and measured and built up rigorously. And I want workforce development. This sounds like a multidiscipline kind of activity where I'm, I'm gonna have to have people who are physically co located from very different backgrounds. So we're doing a three month, it's called the Apollo Accelerator, but they're innovation cycles, right? Build a prototype, show it off three months. This first cohort. Um There was one gentleman fresh out of his undergrad did not know what a restful api was but could solve, you know, quadratics and astro dynamic stuff with pen and paper for fun. There were dev ops guys, I had machine learning folks from national labs and every single person on the team benefited very quickly by being in proximity with experts in something different. That sounds trivial, get making the incentive so that people want to show up in the same place. UmAnd talk about something I is a huge deal. If you don't have organic ways to grow cross discipline teams, then you're gonna have to mandate it and that may be very challenging. So, yeah, yeah, I, I think you make a great point there. So being, you know, a guy who sits in a natural program office. I always think t shaped skills, right? You need to be deep in an area. You need to have an expertise that other people can rely on you, but you need to build out an understanding of who's to your right, who's to your left sufficiently, you know how to work with them because you know, I need coders who understand the domain they're working in. I need, I need domain experts who understand how to work in a system engineering process. I need system engineering guys who understand that. What they, what you know, the the thoughts and dreams that they're putting into this Visio diagram have to get turned into code. And, and have enough of an understanding, appreciation of the other folks around them and humbleness to know how to interact with them, to ask the right questions at the right time. And then the connective tissue, I, I, I see the benefits of, of working closely together As a guy living in DC with customers out in Colorado, I will say you can make virtual work. But there is a barrier. And, and you need the right collaboration tools to help bring some of those barriers down so that a conversation 2000 miles away can still look like you're sitting across the table from somebody can still feel organic and, and easy. and not like you're interrupting someone. So those are all important aspects. But yeah, I do think fluency in the techniques to understand what algorithmic approaches might be valid here is, is a very basic thing and you've seen it with, with, with him already, right? You know, he's a very I'm an operator and then he starts talking about, you know, labeled data ETSS and large language models and you know, obviously fluent in the techniques that he's trying to operate around. I think that's important. And do you see coming from uh the program of World program office world also the need to because we're talking about diversity, but diversity also includes leadership, right? Many of us that are in leadership ranks or not, you know that into the technology. So they need to also be educated, right? So that they can be part of this diverse team, making the right decisions and understanding this new technology, right as it comes up. So we're making programmatic decisions, right? Yeah. Yeah. II, I think as a, as a guy sitting in a, in a, you know, doing what ground software or just software development in a place that's used to launching satellites. There's a cultural shock there where, you know, software takes about 2.5 seconds to field when I hit my little get push and I compile and now I've launched a new software version. That paradigm is very different than someone who spends a decade building a satellite that needs to work the first time. I can launch it. I can check it and go. No, that didn't work. Oh, well, let me go fix a few things. so it's a very different paradigm and there's a lot of communication that you need to do to educate them on what your risk posture should be based on. What is the cost of making a mistake? Now, that being said, it's easy to push code. That doesn't mean you should push it directly to prod and then go out for the weekend and on a Friday, of course, you know, you got everything done. It's 530 on a Friday hit. Push, go to prod wish Sean the best. hope his weekend shift goes well. That's a recipe for disaster. But you know, at the same time, if I take the risk posture of every, every push to prod is a spacecraft launch, I'm not gonna innovate in cycles and I close the loop with China. So Pat, I'd like to ask you to kind of wrap this up here before we go for the audience. And I think your perspective coming from probably one of the the largest industry leaders in artificial intelligence. How do you see the workforce challenge and, and how you maybe you're dealing with it? Well, I re with Mr Robinson's comment about the calculator where it was like the calculator is gonna ruin math. And you know, the freak out that professors have about, oh, they're gonna use ChatGPT to write papers. And I kind of go like, I don't know why we write papers anymore. And they go like, hey, Pat, that's a dumb thing to say from a guy that just wrote a book. And you go, like, I know on page four it says the reason why I'm writing this is I don't think people are gonna read books anymore because I actually think it's like if you have this companion that can answer all your questions, you can even be like, tell me a bedtime story and it does, you be like, hey, tell me everything about the James Webb space telescope and it does. And so I think that especially like Children today who are growing up with the calculator that's called AI, there's a magic box that will make text appear OK. You still have to check it and make sure it's correct. You still have to check against references. You still have to say is that my voice that's coming out of the box. But what I often tell our leaders in the intelligence community is like, if you have someone that is 16 years old today that will be entering your workforce when they're 2223 and you sit them down and go, I need you to write me a 10 page paper a 10 page prose report on what's going on in Ukraine. They're gonna freak out. They're gonna go. I haven't written a 10 page paper ever. I haven't ever typed that many keys in a sequence in my life. I would go to my friend and go. I need, here's my prompt and I need a paper on Ukraine and it needs to cover the following topics and I need you to pull this reference and I need a map of this city and I need to know what are the major industries of the city, what it connects, how it works. But that person is gonna know everything that they need to put into crows. And so I think a lot of our workflows are gonna change and the tradition of, you know, I mentioned expense reports, but I also have to do like my annual performance appraisal which the company requires is a giant wall of pros. And I won't say where that prose we can use to streamline that I mean, I'm not going to say where that prose came from, but the our expectations of things that we produce, I think will change and to major Allen's point about it's about operations. It's about avoiding surprise. Like I don't know that I need to write a five page report that says here's how it avoided surprise today. I just need to avoid surprise. So I do think that the young people that are entering the workforce are going to bring a new set of skills, a new appreciation for human machine teaming. By the way, this happened before in the Iraq war. There's a famous quote that says the Iraq war was fought in Chat and it was like the 18 year olds that you deployed in 2003 were used to using Chat. And they're like, oh, there's this thing called Merch on CNET and I'm gonna chat to someone and say the drone is over here, here's what it's armed with, here's the target and they're just chatting like they were at home when they were playing video games. So I do think Tony, you're gonna see that kind of transformation where young people are gonna bring new skills in and we have to try really hard not to squash them and say you have to do it the way we did this in 1990 because by the way, 19 nineties are just like a thing that had like Seinfeld and friends and that's all that people remember from that entire decade. Two great things though. They, they will last forever. So with that, thank you. We're gonna take time to, to turn to the audience. For some questions, our virtual audience can get some lights that are gonna be passed around. I have several questions posed here. And the first is a two part question I'm gonna start with. The second part. Are you concerned about warfighting scenario? And adversarial activity myopia within the future mission development framework? Ok. Well, I think definitely we heard that one and, and I think we have our operational person on here all ready to chomp at the bit at that one. So major, I already uh can you repeat it slow for me? Are you concerned about war fighting scenario and adversarial activity myopia within the future mission development framework? Yeah, I, I'm not sure that I'm either qualified. I mean, yeah, we mean that we have myopia to adversary actions like a bias toward interpreting their behavior. That kind of a thing? Can we get the first part of that question? The first part is, have operators been surprised by Dr MS for quote, being surprised unquote in the space war fighting domain that have been developed by individuals and teams outside of the dod. So you make ad RM you build your system to achieve those goals and it turns out it was a bad DRM is that we're getting at. Yeah, it's absolutely possible. Yeah. And, and I mean that, yeah, multidiscipline is a huge deal here, right? Like you don't have the guy that is the one changing the gears on the truck, isn't the one designing the architecture that he sits inside typically? So? Well, I think on the question too, uh I think maybe not necessarily the perfect intent of that, but we did to touch on this earlier is the fact that, you know, this is a cycle of what we do on our side and how the adversary reacts. And then it's like I call it the the detector of the radar detector and then the detector is Ay. Yeah. So, so I think so one example of interpreting that Tony would be like, OK, we went to the same school that they did, we use the same book. So when you go, hey, what's the best way to do this? You go, well, if you turn to this page, here's the way to do this maneuver with the minimum amount of propellant. And you go, well, why do you think they're gonna do it with the minimum amount of propellant with the same challenge with humans use doing that. That's right. So there an area for AI might be, I would do this with the minimum amount of propellants because only recently we've started to say dynamic space operations and freedom of maneuver. And what if I could refuel things? So I like to conserve propellant because my vehicles are really expensive. But you go, is that what the other guy is doing? Maybe he doesn't care because he can launch vehicles whenever he wants or he can refuel them or doesn't care about burning up a whole rock AI engine could potentially course of action ahead of. If I was building an AI engine to mess with Sean, I would do things like that and I would be like, it doesn't do things that make physics sense or economic sense. It does things that win. And there's, yeah, I just, I want to jump in here now that I understand it better. I do have a strong opinion about this. Yes. we got it going. Yeah. Yeah. Yeah. Yeah. So, I think, you know, time travel is not real as far unless you guys got something going on in the back room. But there are limits to how an engagement can be performed. Physics will constrain those things. That's a math problem. Now does the weapon system that I'm worried about, what is its performance? Now, there's an envelope and that's, you know, informed by intelligence and I can make predictions and estimates and maybe AI can help me with that assessing the intent though, I feel like this is the trap. We be very careful of test the null hypothesis, which is why I'm a little bit concerned about hyperfocus on anomaly detection. You can be a strange guy all day long. If you're not holding me at risk, there's no imminent use of force. So now we need to have a policy discussion. I think we, these things have been ongoing for some time. But what are the norms of behavior for hostile intent? How do you evaluate them? And then we should be messaging those things very broadly. to say there are certain behaviors that we do not tolerate regardless if others imminent use of force. Now. And that, that's a discussion for leadership in the government, right. So by the way, Star Trek postulated AI before it was, it also postulated time travel. So maybe we'll be back here in a few years talking time travel with everyone. Right. So, all right. Do we have another one time for one more question here in the room? Oh, no stealing. So history is kind of rife with some of the examples. And you guys talked about both new technology and operational surprise when you talk about the Trojan Horse Germany bypassing the ma line with fast armor, refitting merchant ships. How will A I help the operator identify those things that achieve operational surprise that we don't know the tactic for that? We don't know the kill chain for how will A I help identify new things we aren't expecting? So this is really around the idea of really early indication work that we're not used to, right? So we're perfect. Colonel Harvey. Yeah. So I'll say you achieved operational surprise when I saw you specifically raise your hands. Yeah, I know that it was gonna be. So you know, we kind of talked a little bit about like how, how do you not get so focused on like, oh, I'm gonna do this and then fuel approach.I will call out a a reach. I think we're a long way from getting there in the space community of being able to throw AI A course of action, determination type stuff. But you've seen in some of the reinforcement learning approaches, you know, before large language models became a thing, everyone was excited about reinforcement learning to solve all the problems. There was a series of successes that eventually got to something called Alpha Star, which was deep mind's approach to solving the Starcraft problem, which was a set of three asymmetric forces um that any two of them would be chosen at a time to go head to head on a complex map with fog of war. We had partial observable. So there's all kinds of problems are analogous to reality, right? War is typically asymmetric, you have different forces, you have different capabilities, you fog of war, you don't see everything, you know what it once look like. You don't know what it looks like right now. All kinds of problems that uh and what they did is they said, OK, here's here's how we think you should solve the problem by giving it millions of playthroughs of how people have done that. And we have a lot of exercise data of how we think people have white carded, you know, the what red's gonna do and what blue is gonna do um get you know, putting that together to teach an algorithm how to start playing the game that is war. And once you have a foundation of letting it play the game, let it fight itself, let it fight itself. So many times that you start to have instances like they showed in alpha star where it perform maneuvers, perform moves that no human had yet been documented as having done. So it managed to find action space that was admissible by the game. that, you know, people had not previously thought it wasn't doing things that people were incapable of doing. They limited. So it couldn't just act faster than people can act because computers are really fast and people can only act to a certain Hertz rating, which is still crazy fast when you get to the really good experts. But they didn't let it do superhuman things, but it did do things that humans had not done. And I think that that's at least one approach that I've seen that might get us to the point where we start to find action space that the adversary could uh explore that we haven't explored, that could then give us a chance to learn how to defeat that those unexplored actions that, that you can start. Once you've seen it in a simulation, once you've seen it in a playthrough, you can start building your indications, you can say, ok, this is what it took to set that up, these are the observables that could have gotten out of this. This is how it could have detected that. But, but you need to, you need an opportunity to see that event to start thinking through how you decompose that problem and how you turn that into a real alert that an operator isn't gonna say that's not gonna happen. That's that, that's not happening. I don't know what you're talking about. So if I pop up an alert today that says, you know, this, this spacecraft is gonna do this crazy maneuver and it's going to, you know, spend a ton of delta V and like circle past three different things and do something we've never seen before. Uh the sean that's working on the off floor today is gonna go That's a mistake and he's gonna throw it away. He needs some kind of confidence that says, yeah, that's a real thing. I've seen that in X rays. I've seen that play out. I understand what this might be doing. I know what I should do now. Well, that's perfect wrap to this session. So I want to thank you all for taking the time. Thank the panel for having such a quality discussion in a very little amount of time. So thank you all. Thank you. 

The subject of our panel though is we're going to talk about this flood of information that's coming from space systems and it's a lot and it's coming from a lot of different sectors. How do we utilize this? The first two panels have talked about that in some applications. And we're going to talk a little bit more about it in terms of how do you incorporate these, what are now becoming huge volumes of information that drive you to getting actionable information or intelligence to your user, to your customer. And we have a great panel here with us today. I'll introduce him here in a second, but I also want to give you one other perspective. So when I showed up at the US GS, uh most people don't know, they think NASA owns the land SAT system, but it's actually owned and operated by the US Geological Survey. I showed up as the director and of course, I was again old enough to have seen the very first LSAT imagery and as a geologist, those were magical images, right? We could see things at scales that just weren't available anywhere else. And at the time I saw that first set of images that came off of lands. That one, it changed my perspective on how you do things with imagery to begin with. But really space based imagery. So when I became the director of the US G SI was interested in how much information we had available and it turns out that we have somewhere around five petabytes of information. Uh just from the land site, 52 years of land site information. What does that mean? Well, how, what does that mean? In terms of scale? That's about 330,000 high end servers in terms of the amount of information that we have. Not only that's all calibrated, right? So it's not calibrated, validated. So it's equivalent of information. And so that was the first thing that led into. Um how do we manage this cloud hosted systems? Of course, revolutionized how we manage data. Now, what do we do with it? That's the subject of our panel here today is what are we gonna do with this space space data? And there's semaphore going on up here. I'm not exactly sure what's happening, you know, in terms of there. OK, good. We're good to go. OK. So with our panels today, we have a, a great panel to work with this amount of information that I just mentioned five petabytes, that's just landside alone. But think about all the commercial sector that's now bringing imagery to the marketplace. How do you incorporate that? What does it take to normalize that information? And so we're gonna get into the AI applications of how we manage this information with the whole objective of how do we deliver actionable intelligence or actionable information that your customer, what does your customer need and what do they need at the speed of decision. And so our panel includes Winston Tree who's here on the end. And Winston is the co-founder and the chief product officer at Albedo, one of the companies that is now doing imagery from the commercial sector. That's a piece of the marketplace that you can now apply to what we were just talking about. He supports the sales product and software engineering teams making satellite imagery imagery more broadly accessible before Albedo. Winston worked at Facebook as a software engineer. So if you really want an interesting post, you know, to put on Facebook here, We were talking about this earlier 15 minutes after we're finished with this panel, just go outside, just look up and wave and you'll get a great post from Albedo. No, I'm kidding. And then next to him is Steve Kay and, and Steve is the senior Director of Azure Space and Microsoft. And before this position, he was a member of the senior executive service as the Deputy Assistant Secretary of Defense for Space Policy, where he had a key leadership role in the establishment of the US Space Force. And Steve was also an active duty Air Force military officer and has held civilian positions in the intelligence community and on Capitol Hill. And in spite of his appearance, he apparently has a lot of experience in a very short period of time in his short lifespan. Uh And then next to him is Brent. Doctor Brent Bartlett is a senior solutions architect with NVIDIA. He's also an image scientist who developed software and hardware to provide in dance solutions to challenging geospatial problems. Prior to this role, he was Chief operating officer at BMIO and co-founder of TIC Technologies and enjoys exploring the intersection of traditional remote sensing with the emerging AI techniques. And then finally, on my immediate left here is a colleague of mine Dr Neil Jacobs, former acting no administrator and early pioneer at Epic and Unified Modeling. And he's worked in both the public and the private sector for the past 15 years. He has also served as the chair of forecasting improvements for the American Meteorological Society and was affiliated with the World Meteorological Association. So I have a great panel and we're gonna kick off with a few questions and we'll probably address them to the individuals but feel free to, to jump in everybody who would like to, can offer opinions on this. But I want to start with Neil and the US government agencies are now acquiring or have available ever growing volumes and types of information from both government and from commercial providers. And this rapidly expanding commercial industry is now seeking ways of delivering actionable intelligence. So when we look at AI and machine learning applications, which we've been talking about today, how can they be applied to generate integrated data solutions to any customer from these different sources? Well, a lot of people, I mean, if you've been watching the news lately, you've probably seen everyone talking about AI being used for weather forecasting. And to me it's a really, really fascinating time to be in weather and climate modeling because of all the new capabilities. But one of the things that I think it's left out of the conversation, whether you're using it for forecasting or not is the training data sets and all these AI systems that are actually being used right now to predict the weather. They don't actually forecast the weather, they actually forecast what the model that they were trained on would say about the weather. And that's a big difference because those models can't go away. It's an initial value problem that going forward in the future is probably gonna be exclusively initialized from space based data. And the problem with that is there's really no way to avoid moving the data around, particularly with polar orbiting satellites because they down link the data at the poles. And this was a shocking number to me coming from numerical weather prediction world with the tremendous volume of data that we had access to. You'll probably be surprised to know that less than 1% of what is actually thrown at a model gets used in the model. And as the datasets become higher and higher resolution both in space and time, that number gets much, much smaller. And the reason why is because you need your weather forecast now, it doesn't do you any good to learn what the weather is gonna be if it takes me three days to make a two day forecast. And with the latency issue being so critical in high resolution rapid cycling models for weather forecasting, the choice, the correct choice. But the frustrating choice was made that lower resolution data with lower latency is better than high resolution data that gets to the model too late. And anytime someone develops a more sophisticated sensor that collects higher resolution data. The first thing that runs through my mind is how much more thinning am I gonna have to do to get the file size down to where I can get it to where it needs to be in time. And that's where I think we're gonna see the mix of AI, not in the prediction side but in the preprocessing side of the satellite data. And ultimately, I think the holy grail would be on edge where you're actually processing a lot of this stuff on orbit and down linking the really useful important stuff. And that may vary day to day, hour to hour, minute by minute, depending on what's going on in the atmosphere. But to me, that's the biggest challenge. And I think a lot of the manufacturers of these instruments are thinking about this. How much data can I produce high resolution? They're not necessarily always talking to the people who use the data and where they're using the data. So doing a lot of the preprocessing on edge to me is the ultimate. I really think that you're gonna see a mix of edge computing along with AI on orbit to solve this challenge. Jim, if I can build upon the comments that were just made and first off, I wanna say thank you. Thank you for having me here. Thanks for Booz Allen for hosting it. This is a terrific event. So I just want to add a little bit of context on the remarks about the size of data. And you started off by mentioning the six petabytes of data coming off the land sat satellite. And I went and looked at some of the numbers to try and put context into when we're seeing this massive amounts of data, at least even if you just look at earth observation satellite. So there's about 8000 little over 8000 active satellites in orbit today. About 1200 of them are earth observation satellites. Now, those numbers are projected to grow anywhere between 10 and 14 times over the next 10 years. So these numbers are going up rapidly from what they are today. If you translate the amount of data coming from these earth observation satellites in into HD movies like on Netflix, it's 100 billion movies. And if you think about that and just the sheer amount of information in 100 billion movies and being able to extract information out of that, it's just this tremendous challenge as well as this great opportunity for us. Yeah, and, and I wanna maybe I, so again, thank you for having us at this event. Right, Bartlett from NVIDIA. I kind of wanted to actually build on that even a little bit more. So in my background in remote sensing, we, we've talked about raw data but we, I think if you actually look at the holdings of each of these satellite platforms, it's even kind of compound, it's worse, right? You have raw data that comes off the satellite, but then we do a lot of processing, we call it sort of like level zero cleaning, level one, level two, et cetera. Um And so these derived products that, that get pulled out of the raw data, all get storage as well, right? So we have this kind of compounding storage debt that grows over time. So I think it's, it's good in that it was it's being made available. But I think it does, it's not sustainable going forward perhaps to store kind of like that much data that's just dried from this one source. So one of the things that I think is an opportunity for us is to look at ways to uh more efficiently store those raw pixels And then rethink the architecture around using tools like accelerated compute to create those products on demand versus sort of like storing everything forever. So, we've sort of heard two things coming out here. One is just the sheer volume of data and that's gonna expand almost exponentially as the private sector gets more and more involved. And we're already seeing that second is how do you manage that so that you deliver actionable information to your customer? Obviously, there are some challenges uh and limitations but also capabilities. Would you guys like to discuss potentially how AI/ML applications computing at the edge which Brent you brought up? What are those, what are those capabilities? What are our advantages? But also when we talk about computing at the edge, what are some of the limitations that we're going to have to address? that's open to anybody. So I can start because it, it does come down to, you know, we're, we're starting this kind of conversation on this panel about starting with the data and then, and then how you use artificial intelligence, machine learning generative AI these different tools to then understand the data. The data management is a foundational piece of it. So at Microsoft, one of the areas that we're working on is something called the planetary computer, which is the ability to bring in massive geospatial data sets. So you had mentioned, you know, five petabytes earlier, we're currently hosting in the cloud over 70 petabytes of uh geospatial data across different space systems to then serve as the platform to bring artificial intelligence, machine learning tools on that. So that's in the hyperscale cloud. That's something that you know, there is tremendous amount of compute and storage resources which really does then provide this great foundation for these other tools on top of it. So, yeah, so I guess I'll jump in again. One of the things that I think we're also getting excited, like very familiar with planetary computer, it's an awesome resource. You know, I think one of the things that has gotten folks excited about the opportunity of things like generative AI is the ability of the technology to some extent today is still very new to sort of democratize computing, right? So you can now access a computer in a way that you've never been able to before without necessarily needing to know low level like assembly language or Python at a higher level. So I think it'll be really interesting to see where that type of technology goes relative to vision, right. So we're starting to see attention mechanisms start to work well with foundation model building with imagery and remote sense geospatial data. So, yeah, I think big opportunity to sort of democratize insights in a way that hasn't, you know, been available before and more easily access the information there in those, what you say 70 I did mind. Yeah, so that would be exciting. So you bring up the concept of democratization of data, right? What to you? And I'll just start with Winston and come all the way this way on the panel. What does that mean to you? I think at ALBEDO. So we're focused on one, capturing the imagery itself. So, you know, we'll have the imagery and next year once we launch our satellite, but getting that imagery out in front of users in a timely manner. So, you know, we have a lot of dod IC folks in the building. you know, that time value, the value of that imagery decreases over time and at a certain time it falls off a cliff, right? So getting that imagery in front of users when they need to and them accessing it in the ways that they need to, I think you mentioned formats So, I mentioned formats earlier but being able for being putting controls in front of the user so that they can specify what kind of data, how fast they need it and then immediately be able to access that. And that helps with the latency problem that helps with the US usability problem. So I think that nutshell for in terms of democratization is making everyone has a sort of level and fair playing field where they understand how to get access to that data. Yeah, so I love that term democratization of data. And, when I think about it, to me, it really does come down to the accessibility of these tools and technologies. And you know, we're here at this space plus AI summit and those are two major, you know, one of course a domain, another a technology, but both going through what I would say this democratization moment. Whereas, you know, space you know, historically was the domain of governments and in fact, superpowers if you go back to, you know, the history of the start of the space age, but now you have all this commercial activity with companies like Albedo and others that are now bringing these capabilities to not only governments but organizations in different sectors and across the entire globe. Additionally, AI is at this moment where lots of people now have access to leveraging the technology. And I think, you know, chatGPT which has been, you know, talked about some today can really be credited for that, for having this moment of now putting A I into the fingertips of everybody. And I guess, you know, one of the things I would close is, you know, something that, our CEO Satya Nadella said, I believe he said it this week or last week at the World Economic Forum is that computers provided information at your fingertips. AI is now providing expertise at your fingertips. Yeah, that, that's really interesting way to frame it. There's a lot of good quotes. So I think maybe the part I'll kind of layer on to that is going circling back a little bit to the edge comments, right? So, I think an interesting thing that that will kind of arise out of this, we've talked about the huge amount of maybe you can call it like store and forward, right? You're storing up data and then you're down linking it and then storing it back on earth. And there's a certain value that you can drive from those historical holdings. There's another I think interesting way to use AI in conjunction with these large number of proliferated low earth orbit constellations go up where you have sort of IND individualized tasking that is AI driven, right? So instead of having to have a lot of expertise on like how to fly constellation and you need to be a government to do it. I think the tooling has the opportunity to bring us to a point where people can at a very cost-effective price point sort of get real near term data collected in a particular area, maybe it was a farmer with their field or something. And using AI tuning and, and enable them to do this type of hyperlocal analysis that they haven't been able to before. You know, when I was at NOAA, this is a pretty interesting use case on democratization of data because a lot of times if you ask them what they think about that, they'll say, oh, it means everyone gets equal access to it. There's a congressional mandate that requires Noah to make all their data, all their code, everything freely available to the public. And I asked for the global modeling code about 20 years ago and they said it's free. Here's a tarred up version of all our FORTRAN code. It's 9 million lines and it's hard coded for IB MA IX. So you're gonna have to go buy $100 million supercomputer if you actually want to run it. So just having access to it is not enough. And so for the last probably 10 years, all of their code has been refactored, it's all on github and refactored. So it can be run on almost any architecture which really opens a lot of doors because the problem was, yeah, this code is freely available and all these university researchers and postdocs and pis and grad students who had all these great ideas on how to improve the weather and climate model forecast couldn't test their ideas because they didn't have a security clearance. And the only people that actually had the machine, which the code would run on was a government agency. And yeah, the code's freely available, but where were they gonna run it? And so now all the codes out there, everyone can access it. I can run it on anywhere from, you know, commercial cloud to my laptop and it's a force multiplier because now we're seeing the science and research community, the universities coming up with cool new ideas on ways to use the code, ways to improve the code and they're sharing that back. The other thing is just having equal access to the observations. Well, that's great. But you still need the code, the data simulation systems, whether you want to run the model or you wanna do your training now, all that's on cloud too. So a lot of the cloud service providers are hosting the data, you can run everything there. There's almost no reason to this is probably something we should touch on at some point to move the data around. You know, that's what I've been telling people just stop moving the data, you know, move your processing to the data. So that brings up a topic we've sort of touched on a couple of times now and in the panel previous, they were also touching on it. And that is getting back to computing at the edge and doing your processing at the edge. In other words, on board the spacecraft, which does two things. One is that one allows faster dissemination of actionable intelligence, but also frees up the bandwidth restrictions that we've always had on the, the way I think of it as an si pipes going down, right? So the information that's coming back as Jimmy Spivey mentioned coming back in mission control. So all of that information has to come back from what I'm hearing in one form, if nothing else, at least in an archive and access capability. But then there's the actionable intelligence piece which seems to be the domain more of the on resident or on spacecraft processing capability. How do you guys see that as as requirements if you want to think of it that way, it's almost like two separate sets of requirements. And what does that mean in terms of how we would apply generative AI as we were talking about in the back room and I'll kick that open to whoever wants to jump on that one. I can, I can go first again. So I, I think one of the areas I'm kind of tracking right now is sort of how, you know, we, we've talked a lot about large language models, but some of the tooling that have been is being built around large language models. So using a large language model almost as a new form of operating system. So and one that runs very efficiently, right? So autonomous agents are kind of coming out where you have the large language model, the almost acting like a decision maker on which different tools it might run to, to do an action. So in terms of applying that kind of methodology into to space processing, I think there's some interesting opportunities to kind of rethink what it means to be an edge in in space. So, you know, you can try to place a processor onto a satellite that also is running coms and it's also running mission control and the payloads. But you can also think about putting specialized satellites that are almost like floating little data centers, right? And using cross communication links to get data to those processing nodes. So I think as like our architecture is mature, there'll be more opportunity to do that kind of compute in orbit. And, and like you said, provide those insights back down. Yeah. I'll build on the generative AI piece and I think that it's really amazing what this technology is doing in so far as empowering others. And I remember when it first kind of kicked off out there, you know, chatGPT and open AI you know, people are saying, OK, well, it can create haikus and jokes and things like that. But then pretty soon what you saw, you know, Stanford was using it for instance and they showed how it could pass the bar exam and it wasn't just passing the bar exam with the multiple choice, it was the written section as well and it wasn't just doing average, it was doing at the top 90%. So you start thinking about what these capabilities can then provide for space. And I think that it really goes across all elements of the life cycle, whether you're building and there's aspects of these capabilities, you know, for instance, on github, there's something called copilot where you can actually create code with natural language, then there's training and testing elements where you're able to look into large amounts of information and then ask questions of it and have that information at your fingertip or in operations where it may be helping you. You know, we did something for instance, where we worked with the Defense Innovation Unit to show that intersection of large language models and geospatial information. And if you ask questions and then bring that together to be able to get responses with what's happening on the space data. And I, I think that that's really where the future lies is this multi multi modal aspect where you're gonna have these different technologies that are coming together. It's not necessarily just the large language piece. You're gonna have the visual piece, you're gonna have an audio piece and it's really each of these coming together. Yeah. No, that's great. I at Albedo, we, we aren't necessarily using edge processing on our satellite. We kinda believe in the adage space is hard, so we don't try to overcomplicate things up there. At least for now. but on the ground really looking forward to be able to using, you know, all the things we've seen come about in the past year, chatGPT. maybe as like a silly example. I love playing basketball outside. So I once we start being able to image the earth, wanna find all of the undiscovered basketball courts and play on them. So I think, you know, just as Steve just pointed out, like you're using it not only as an existing search function where you're not searching PO I data and Google maps, you're searching, you know, real features that are from fresh imagery. But also using that in another part of the chain. So using this chatGPY like function to you know, develop more training data, right? Like that's an easier process instead of having to go through and, you know, one by one label things starting out with some sort of prediction. So I think the integration of these tools that we're seeing and we will continue to see in parts of the chain will really make the big difference. Like I don't really think it's a you know, silver bullet or Yeah, I think it's change comes in parts. So from that standpoint, we think about what do I want to know? And when do I want to know? It is really kind of what we're getting to and democratization to me almost means, you know, what is the information I want to know versus what's the information Neil wants to know and they're gonna be different. And so having that ability to anticipate what's required for the operator or the customer. in the case that, that Jimmy brought up in the last panel is what does the crew need to know? Right. So it's an ops guy. That's what I want to know. What do I need to know now and what do I need to do about it? What do the folks in mission control in an integrated center? What do they need to know? And those two things are completely different, but the more we move towards this this adaptive capability, we will now be able to deliver actionable intelligence in different flavors, right? What it gets to what are some of the challenges that you see in being able to deliver that actionable intelligence? And you can take it either from ground processing or processing at the edge. However, you want to approach it, what are the challenges that we see in front of us? And how could we use generative or adaptive? AI to be able to help solve some of those problems? So, I can start on that. But one of the things we did and you know, a challenge sometimes comes down to, like you're saying, there's a timeliness piece of it, of there's certain information that's it's perishable, it needs to be known right away. And, we did, you know when talking about edge processing. One of the examples we did is we worked with NASA to bring artificial intelligence on board the International Space Station. And that was specifically a project that we did with Hewlett Packard Enterprise was something that they have called the Space Born Computer. And, but we worked with the NASA scientist to use computer vision technologies to be able to rapidly identify if there's damage to an astronaut's gloves after they go on a spacewalk. Because when they're out on the spacewalk, they're using their hands, they're doing different experiments or maintenance on the space station and any sort of damage to their gloves could be a serious issue to the safety and health of that astronaut. So when they come in, they use the imagery, they take images and, and can even use video to be able to rapidly identify and assist them in space on the edge to see if there's uh anything that they should be um concerned about. So that's a great example. Having done a few of those myself, there's um and gloves are the worst part because the mass of the spacesuit, most people don't necessarily know that you put me the suit and the equipment on me. It's about £750 mass and you're working in a weightless environment, you're putting a lot of force on those gloves, right? The ability to, to basically what I understand is that you can now scan, just do an image analysis of the gloves and have on board an assessment. What are the weaknesses in this glove that I need to worry about? And, and then that information of course, gets transferred down as more or less. I don't want to say the archive set, but the set that now goes down to Jimmy's mission control team that says this is the decision that they're making based on this information and then they have the ability to vote. Yay or nay, I would guess on something like that. Yeah, that's actually really interesting that I, I think the interplay of. So we talked a lot of, you know, this is a, a space panel but we, we haven't talked a lot about sort of like auxiliary sensors that are in a particular environment, how they can enhance data that's coming from space. So as an example that my dad's actually a farmer and grew up doing throwing hay bells around and it was always a challenge like to go out and be like, ok, the hay is ready or we're gonna cut it today and do we have enough time for it to dry before the rainstorm comes in going back to the weather prediction challenge earlier? And it's also kind of like when is it dry enough? Right. So it, you know, I was brainstorming the other day, my dad about this and he's like, why can't I just, you know, you guys do AI, right? Like, why can't you tell me, you know, when, when the, when to do this? And he's like, I'll put a sensor on my tractor, right? Like, you know, and, and it's not that far of a leap going from precision agriculture use case to say like tactical environment, right? So I think that is an opportunity that we have to have these AI systems inform kind of like which sensing that we can place in situ in an environment that will make that insight that we try to drag out of the satellite imagery a lot easier to obtain. So, yeah, I've been thinking about this actionable information cause in, in my world, it always comes down to probabilities. And if I'll give you an example, like dealing with the weather forecast, if I was to tell you, it's gonna be 30% chance of rain, you don't know if that means there's slightly less than one in three chances you're gonna get rained on or is it 30% of the day and it's 100% of rain for 30% of the day or what that, you know, actionable information when you provide someone with probabilities is really tricky. I mean, just look for example, at hurricane landfall or tornado warnings and people are inherently irrational. You know, if you, if there's a, there's a study out there and everyone thought, oh, if, if we could extend the lead times in tornado warnings will save more lives. Well, it's, it's actually not true. There's, it actually, if someone thinks they have a lot of time they may think. Oh, it's, it's, it's, yeah, I got 20 minutes, I'll go get my kids out of school or, or something and they'll get in their car and get caught on the road. And so translating information, particularly probabilities into actionable information by that. I mean, a, a decision that we hope someone would make to protect themselves to me is a big challenge. And even though we've been talking about edge as a, as it is applied to space based capabilities that initialize these models. I think there's another use case that we'll probably see coming here soon. And that's once we have the capability to replicate these forecasts, using AI that runs on such a tiny fraction of compute, you could literally do it on your phone and then you have the opportunity for a phone that knows your habits and your location uh to have a custom tailored solution with you at all times and that's where you're gonna get your information from. Yeah, I love that example. I think as you know, I see the value chain as you know, we start with the sensor which is where albedo is at. And at the end there is the value provided to the end user or the customer, right? So it's a big value chain. And as much as a seller of satellite image, you would like to say we're the only piece of information you need to drive that actionable, you know, insight decision. we aren't right. You need to combine all these multimodal sources of data and to put it into context, you know, if there's a hurricane coming, that's not the same. And the decision you want to make is when should I leave? That's not the same decision for everybody, right? Like if I'm a single person and I would live alone in an apartment, I can just pack up and go. But if I have a family and six kids and a small car and a dog. How am I gonna fit everything in the, in the car? Right. And I have to leave and plan and pack earlier. So I think it's you need people in the middle of the chain to provide that value, that additional translation to make it specific to the end user. and there's a degree of generalization that you can do, but in the end, it has to, it has to make sense for that customer. So really that gets down to when you look at the whole chain, as you were just mentioning, Winston, and you want to democratize the information to each of us individually on this panel, for example, you know, it depends on what is it that I want to know and when do I want to know it? And so there's an in between step that seems to have to exist, right? That's now looking at the patterns of your behaviors versus my behaviors and then tailoring that information so that it becomes ingestible by the end user of the customers if if I can build upon that because it does come to down to your data as, as you're saying, and you know, we're here in the nation's capital, you know, a couple of miles from the, the White House and from Capitol Hill, and there's a certain amount of public data that, you know, we've talked about before certain satellites. But then there's data that is not going to be public. For instance, we had, you know, panelists up here earlier from the Department of Defense, from the National Cons office in the intelligence community. And they're going to have data that they are going to need to you know, keep at the right security classification level and and keep only to those that have access. And what's really interesting is these capabilities. So for instance, you know, Azure Open AI, which is based upon the GPT AI technologies enables enterprises or government to be able to protect their data. So it's not just using the large model that's out there but being able to ground it with their own data. So they're getting insights from these models based on what they're putting into it to include those models. And we announced late last year are going, those capabilities are going to the classified clouds. So governments are now able to bring their data or the US government in particular is able to bring their data and specifically be able to have the benefit of these technologies with it. So we've, we, one thing we haven't really touched on yet Today is the term digital twins. And so what I'm becoming aware of is that when we talk Digital twin, we might have five different opinions on what a Digital twin is just on this panel alone. So in terms of the application of a digital twin, first, what is a digital twin mean to you? And then second, what does it mean to your customer? How do you see the digital twin being an application of the generative AI support that allows you to drive to a true digital twin. Where do you see some of the challenges and some of the risks associated with those uh that I could go first. We happen to have a whole, a whole suite of products around Digital Twin, right? So, I sort of decoupled them in my head from generative AI and I think as the months kind of tick by, they start to kind of merge together more and more. But we do have a tool called Omniverse NVIDIA that was kind of came into fruition over quite a bit of work internally in quite a few years before I was an NVIDIA. Um But to use the term we use. But yeah, the Omniverse Metaverse concept kind of was out just a little bit before I would say the big splash of generative AI and it, at least from my perspective, I support the Dod and the federal sector and NVIDIA and I see it as a really enabling tool to to take a lot of things that we do physically in the world today, things like testing, you know, testing email, and, and bring that into the digital world and there's a, a lot of advantages to that. So, you know, we can iterate more quickly. as we kind of quote, we, we call it the, the real to sim gap as we kind of like close the gap between reality and the physical world. We're able to kind of iterate and create our systems and test them a lot more efficiently and safely. And then, you know, as, as AI starts to permeate that ecosystem, we're starting to see some really compelling things happen. So, NVIDIA actually created this tool called Chat us D. I won't get into the weeds of what universal scene description is, but it's essentially a way of creating this general file format that can encapsulate the 3D virtual world. And so, uh it's a language like any other. And it's possible to train generative AI to understand that language. So, yeah, there, there's a lot of a lot of kind of fascinating tech capabilities that are coming out of that where you can do things like describe like if you're in the dod and you're trying to do a mission planning to go rescue a sailor that's stranded. You know, you can actually leverage these these AI tools within this virtual world that's grounded in and the physics of our world. and come to a rapid answer and what, what type of solution you might wanna approach, like which, which helicopter platforms are most suitable and if that doesn't work backup plans. So yeah, that's, that's kind of what it means to me is just at, at just the beginning here, but like being able to do things a lot more efficiently and rapidly and safely. If you wanna touch on that, I, I love where you went with that thinking of digital twins plus artificial intelligence, I would expand that to then plus cloud computing plus augmented reality. And it's really the intersection of these different technologies coming together and the hardest use cases is where things really start getting interesting. And you know, I would say that that's the type of stuff that we are working on. In fact, we the Space Force recently announced a contract that it has with Microsoft called the I three E integrated Immersive Intelligent Environment. And that was a follow on contract to the initial six months where we demonstrated some capabilities to include things like how do you have a digital twin of the International Space Station? You bring that telemetry down, you're then being able to visualize and understand that in an augmented reality environment, you're then able to bring artificial intelligence tools to understand when anomalies or situations are happening to it, you're doing it in a cloud environment that expands beyond a physical location, but to multiple sites. And you can start seeing the integration of these different technologies addressing use cases in ways that really never have before. That's an interesting one. Let me follow that for just a second. So when you mentioned bringing the environment of the space station down to the ground, for example, I could conceivably see how mission control architecture could now adapt to that, particularly as Jimmy mentioned, the farther out we go in space, the longer the latency is gonna get. And when we talk about distances to Mars, we're looking at 40 minutes round trip potentially when we're at 250 million miles and two weeks of occultation where unless you have relays, you're really out of communication. So having that capability, say in mission control would allow with enough fidelity a mission control architecture to operate near real time capability even though they are 40 minutes away, so to speak. And Jr I would expand it into, it's not only mission control, but it's those people training and learning before they're going to mission control to be in those environments and then it's people building and it's people building the next generation of the system to actually be figuring out what the next requirements are, what the features, what the new things they're gonna add to it. So it really ends up, these technologies that we're talking about do end up transcending across them. And you know, the probably the most important thing I think that I would say today to encourage this audience and anybody listening is, how are you getting involved? How are you leveraging these technologies? How are you experimenting with them? How are you getting your hands on them? You know, there was a fortune article in the last week or so. or maybe a couple of weeks that the headline caught my eye, which was essentially AI might not replace your job, but it will definitely become your coworker. And the thing I ask you all is it becoming your coworker yet? And if it's not, why not? I think that there's a use case with the digital twin that really applies to satellite observations in space and, and I'll give you some background on how we used to do it. It would be design a satellite, launch it into space, pull down the data, let's say, for example, there's a 10 year usable lifespan of this instrument. Well, it may take the scientist five years to figure out what to do with the data, how to quality control it, how to optimize it and use it in the model. By the time it actually is providing a positive impact, the lifespan of the instrument on orbit is half over, you could eliminate that process by just having a digital twin of the earth. And you can synthetically extract data on the ground using your digital twin and simulate what the satellite would produce, feed that back into the models and optimize everything before you even launch. To me. That is one of the most exciting use cases. And we've, we've done these experiments in the past, we call them observing system simulation experiments, But they've always only been as good as what we call the nature run, which is sort of our best guess of the atmosphere, which is orders of magnitude more crude than a very sophisticated digital twin. And so that's where I think at least from my world and, and earth observations. an exciting use case of digital twin. Actually, I just need to jump in real quick. I, I actually just got an email before I got on the panel that they were releasing access to Earth 20 So jump on Earth 20 where we, we're actually pursuing that actively. That's something that our CEO is really passionate about of kind of building a supercomputer that would be able to simulate the entire earth at a reasonable useful resolution. And I know that watching the engineering teams iterate over the past six months to a year, a a lot of that was figuring out how to pull all these feeds into that. So, yeah, that's uh something that jumps out. Well, unfortunately, we could go for another hour,, on this, but we're running out of time so we wanna make sure we do have an opportunity for questions. So if you have a question, raise your hand and come to you. Sure, this year. Thank you. Good afternoon. I'm Sandra Irwin with space news. I have a question for Steve Kay, or if the other panelists would like to weigh in as well. It's about the Department of Defense and their initiatives with AI. Uh we, we heard that they're doing a lot with trying to promote AI, trying to increase use of AI. even the Secretary of the Air Force said we need more tools to analyze all the data. Like all the things you guys just talked about. So Steve, you mentioned this Azure GPT tool that you know, the government is using but is that the solution to the dod problems or why is this continuing to be a problem? Given all the technology that's available? Why are they not using it at the scale? Maybe that they should be using it? Thanks. So it's, it's a great question and speed and adoption of these new technologies can often be a challenge, not only I would say with the Department of Defense, but in the United States government writ large, but we are seeing um great people, great leaders that are trying to promote the speed of adoption and recognizing how much these new tools and technologies, for instance, Azure Open AI in the Air Gap Cloud can help empower their workforce to achieve more. I would say, you know, we're, we're getting there. But I think there is a lot more work to go to get that speed of adoption and then it's moving from demonstrations to scale global capability. Because at the end of the day, if we're talking about the Department of Defense, you need to get the capability into the hands of the operators who are trying to deter conflict, maintain peace. And if peace fails that they're prepared to win. So it's making sure that technologies get all the way out to them and it gets beyond that demonstration. So I would say that there's pockets where we see it happening, which is exciting. But I think, you know, overall we are looking for more adoption and faster speed and, you know, I think that it's not just Microsoft, but there is an ecosystem of innovation, you know, Albedo NVIDIA Booz Allen here. In fact, actually, they've got some really exciting stuff I saw in the back. And it's helping the department leverage these across the board. We'll get two of them. Do you have any examples of Digital Twin gone wrong or like what are some common mistakes that uh maybe to look out for, for that sort of thing. I, I could do a quick, I see lots of signs flashing in so I'll try to be brief. Um So I, I think one of the I'll just talk to some of the challenges of Digital Twin. We talked about opportunities. You know, the things that I see come up over and over again is, you know, we, we get very excited about making this perfect replica of a facility or um the whole earth. the oldest state of this building. The reality is you have to have a pretty broad skill set on your development team to make that happen. People who are experts at 3D design, getting all these like these lights with these swishes in here. The buildings, the material attributes that, that are real closing that like sim to real gap is a real challenge. making sure you get the physics right? And then typically, like as soon as you move out of like R GB like visible that's pretty well described into like infrared or different sensing modalities like RF or radar, the challenges become compounded. So I think that those are some of the things that, that are starting to be addressed. But those, those are like, hopefully gonna gonna kinda like mature over time. OK? I wanna thank our panel. It was a great opportunity to have this discussion with you today. Um And thank you very much for joining us here at Booz Allen. We really appreciate your insights today. It was very helpful. Thanks very much. 

And to wrap up our day. Now, it's my great pleasure to introduce to you, Booz Allen space lead, retired Lieutenant General Chris Bogdan. OK. We're supposed to end at three. So I have about 20 minutes of comments for you. I'm only kidding. First, I wanna thank everyone uh for being here today. And thank those who have tuned in virtually. At one point in time, we had nearly 400 folks listening to, to the panels. So that was, that was pretty impressive. Thank you. Folks out there in, in virtual land. I hope that you found today's space AI summit somewhat insightful and fascinating. And our purpose here was to make you think to make you think about what the future could be when it comes to space. And AI, I know it truly has had me thinking about some things. Throughout the day, we heard a variety of topics and insights from, from our colleagues, Greg Robinson from NASA kicked us off with what it means to be a space pioneer and the challenges he and his team faced putting up the webb telescope and then let us to understand that many of the technologies that they developed on the web telescope were one of a kind. And 1st, 1st of a kind and they put them together and it actually worked and when it worked, it created tremendous advances and will create tremendous advances in what we know about the universe. AI holds the same exact kind of promise. We just have to take that challenge and meet it. We've had some incredibly insightful panels today too on Domain awareness and ground systems and space data and how AI can really improve space missions like ISR and like Earth observation and climate and domain awareness and traffic management and including modernizing ground systems um for future complex constellations. And finally, we were very grateful for AC to come by and talk to us about how AI is being used at NASA today and in the future and, and there's some tremendous promise there. I thought your charts were about the best I've ever seen. I'd like to get a copy of those. They were really pretty awesome. So what lies ahead for AI and space? What I heard today was lots and lots of challenges and we know they're out there. But I also heard a lot of potential for AI and digital technologies and, and data technologies across many, many different organizations and agencies including our military partners, including our intel community, including a civil and and commercial space operators. with respect to really figuring out how to harness all the power of the data that space is providing us. And I think AI is a huge part of that solution. So leave here today thinking about how AI can help your mission and help you harness the power of your space data. We here at Booz Allen, we're positioning ourselves to being a leading provider of space data as well as creating modern ground systems and an integrator of complex space constellations. And we know for sure that AI is an important part of the tool set that we need to bring our clients. So we're thinking long and hard about those challenges and long and hard about how AI can bring that to our clients. We talked a little bit about it today, probably the biggest hindrance I feel today to using AI in this domain and other domains is culture and trust. OK? And we heard some really good stories about how professors thought calculators were going to ruin math for, for Children. And it, and it didn't turn out that way. II I think we're on the precipice of something really, really significant when it comes to AI. And so we have to overcome the barriers of, of culture and overcome the barriers of trust, but it's incumbent upon us the those folks who understand AI and use AI to help our clients understand what we're doing and how we're doing it to build that trust and also to help them to adopt these capabilities within the culture of their organizations. We can't forget that you can't just give folks technology and expect it to do wonderful things. You gotta take the human dimension into, into account. And I think that's an important part of moving forward with ethical AI and building that trust and having people understand and change the culture to use it properly. So, with that, on behalf of myself and Judy Dotson, our boss in global defense here at Booz Allen and all of my great booz Allen teammates who made this happen and I'm gonna give you a round of applause for that, my booz Allen team. We wanna thank each and every one of you for coming. I would know I was supposed to say I was supposed to look straight into the camera and tell all 400 people that are out there virtually. This concludes our event for you. So you can click leave in the little box up on your web page. But for those of you who are here, please, before you leave, take a stop and take a look at our demo on Space Domain orders and traffic management. But even more important. There's snacks out there to include brownies and cookies. And on the way out you can get some Booz Allen Oreo cookies. And if you're really, really risk taker, you can eat some freeze dried astronaut ice cream. Hm. I tried the cookies. I have yet to try the ice cream, but I will. If you want me to do it first, I will. But it's out there too. So thank all of you for coming wonderful venue. I hope you're gonna leave here thinking long and hard about how AI can help your mission and, and how we can help bring this tremendous power to our space clients. Thanks very much.