Takeaways from DE24: Enabling Data for Mission Outcomes

Good morning. I'm Rich Crowe, I’m the President of Booz Allen’s Civil Sector. Thank you. Today, we're excited to have you with us at the Helix, our Center for Innovation. And we're excited today to be joined by Fed Gov Today and Francis Rose hosting, taping of one of the future segments that will be aired on April 7th on ABC7. We will talk about three segments today. The first is contemporary patterns for unlocking the power of data, the present and future role of AI, and then how do we maximize and securely deploy these solutions to really affect greater change and outcome for mission sets that we support? The topic is data. It is probably the most valuable commodity the government has today in solving missions that ubiquitous. It's everywhere, but it's corralling it and wrangling it and getting it into a better place that we can kind of take on these missions sets more effectively. We call this data enabled missions because each agency has their own unique missions, they have their own problems that they're trying to solve. But when we look at these missions, there's some underlying tools and techniques and technologies available that allow agencies to effect the change they need to be more productive and deliver better services to citizens. We call this data enabled missions. Now, in order to get to that data enabled mission, there is a data enablement journey, and there's three phases to this. The first is phase one on this roadmap, which is basically identifying the data that you need for the mission outcomes you're trying to achieve. Identify that data, collect the data, validate the data, get it into one place, kind of like doing the grooming. And then once you have that, then agencies, as you all well know, needs to then move forward and start migrating stuff to the cloud, getting it into better locations, and then being able to take advantage with modern I.T. modernization and devsecops solutions to enable those cloud technologies to deliver better services. And then the third phase really is in how do you start thinking about bringing tools like AI to bear. And you can start bringing AI and other data technologies into the stack that you can really start creating mission stacks essentially on top of your tech stacks that’ll allow us, allow the agencies to give better results, faster, more efficient, and make them more effective and efficient government outcome. If you look at the civilian agencies across, it gets really exciting to think about what the potential is to unleash this power. I mean, you can imagine the VA is already doing extraordinary stuff with the with the data that they've unlocked. If you look forward into the future, how many more services, faster, better, more effective can be provided for veterans and better outcomes by the Department of Veterans Affairs. If you think about FEMA and its role in response to natural disasters, how much better can they be at predicting and pre deploying equipment based and having better outcomes in terms of resilience for the United States and critical infrastructure in the face of natural disasters? And then you start thinking about the CDC and the recent experiences learned from the pandemic about being able to use data across multiple state jurisdictions to proactively predict different health crises before they get out of control, to alert public health officials to impending problems and deploy resources appropriately within the health system. These are tough challenges. They're complex. But the exciting thing is they're surmountable. It's a lot of work. A lot of things have to happen. But we can get there. And we're at the cusp of I would say, a revolution that's really going to be spurred by better and more access to data. Today we're going to talk about, we'll talked with the several federal mission leaders and Booz Allen experts to talk about this data enablement journey where they're at with all this. So it doesn't really matter where you're at in the journey, whether you're in the very beginning of just trying to identify where you're at or you're in the middle of modernizing your I.T. stack or starting to implement AI. What we're going to talk about here today is how you can quickly get up to speed wherever you are in the journey. And it's really important to kind of define where you are in that roadmap and really where the end of the journey is and the outcomes you're trying to achieve. Sagar, thanks very much for your time. Thanks for joining me today, where are you, inside FAST in particular, but broadly across GSA in your data enablement journey. Welcome. So thank you for having me. So we started this journey around 5 to 6 years ago, and our primary objective or our strategy was to continuously innovate, build and ultimately commoditize, the technology layers that form the ecosystem where our security data and cloud are converging. And the idea was to use this as a force multiplier or accelerator for the business transformation. So in a sense it was a data centric security baked in ecosystem that can enable the composable architecture, which helps in thumbs up product mindset. So on the onset, we, at first had this federal marketplace I know many of you know, first is, is a more of a middle person in terms up the customer personas is the customer such as suppliers, the agencies that use the services of fast and contracting officer. So when we started this journey, we wanted to make sure that we how this product mindset idea was as we are building these products, we wanted to decompose that into a business capabilities. And the first layer was making sure we automate the CICD pipeline in terms of developing and deploying software. So that was the first layer from a compute and storage perspective. The second layer was as we worked to, we had this product needs to talk to each other. So we needed this idea about API framework that was a second layer. The third layer, which is more important, most important. No business transformation can be done without, without the data. Data is where the actual transformation happens. So we wanted to create that layer about these two layers, which is a data layer. And the idea about the data layer was we wanted to make sure that any data assets, or processes are discoverable, identifiable and is presented to the user in a format they could make their own decision. It's about the data driven decisions. And that part of the data layer was built, all through that time. Just like the way you have a CICD pipeline, it has a lifecycle in order for developer to build the code, develop and deploy, there's a lifecycle for that, in a similarly, data has also a lifecycle. You got to find, identify the data source, then you got to curate the data and present it in a way that the business users can make those data driven decisions. So the notion of our strategy was always then as you build the one layer, which is a CICD layer, you build it, standardize it, commoditize it, the next layer was the API layer, a layer aboard that security layer. Now we are in the journey of where that data layer is has, at a given time, that we could bring any data source into the lifecycle. We could bring any tool when used in a seamless fashion as if the demand, if there is demand for it and the users would be able to go in their data in a more self-service manner and make those data driven decision making. And and that's where we are on the journey. But what I want to point out was the layer approach, nobody can start at the data layer in the beginning, you got to start with the cloud layer and then you go to the security layer and then to the API, at the data because you need to, the data layer needs to inherit the layers below that. And so where we are as part of the journey is right now our users at the given time can have their self-services capabilities without worrying or without depending on relying on the technology people in my organization. A couple of concepts there I want to ask you about, because you use terms that I don't hear very often when it comes to this journey. One is commoditizing the data and the other is curation. Curation is one that I expect to hear a lot more than I actually do. Why are those two concepts so important to the journey that you're taking and getting you to whatever the destination is for that journey, Sagar? And so I guess the whole concept about is with that we hear the democratization of AI/ML. At the end of the day, when you say artificial intelligence or machine learning, it's all about the data. And idea is if data is your biggest asset, we want to make sure we give our technology in the hands of tools, in the hands of the users who are expert in the data, which are the business people. And the commoditization is they don't have to worry about where the data comes from or how to curate the data, how to manipulate the data along the way or extract and manipulate. And so those things are done from an engineering perspective. So the business, the users can focus on that high value where they could look at the efficiency, they could look from a mission involvement perspective. So I think those things how done because of the strategy that we put in place like 60 years ago or five years ago, is about making those layer as a commodity. And the curation of our data, which is I talked about, is data lifecycle. Everything has a lifecycle. So those lifecycle of the layer, it is also commoditized. So at a given time you could be bringing new data source or you could bring new tools. The reason I'm talking about the new tool being AI/ML the ecosystem allows any news and technology to be inserted a seamlessly and that goes back to this standardization commodity from an IT perspective. We have less than a minute left, Sagar, how do you map this journey so that you know where you are and where you want to go and how you're doing on getting to where you want to go? So, so, so from a journey perspective, I would say from a data enabled mission outcome, where we are in this journey is we are pretty much, we pretty much know how to develop and deploy a software. We pretty much know how to develop and deploy a product. Data is also a product. So from that journey prospective where we see ourselves that we are open for empowering our business users to use the tools that they like, access, identify or access the data wherever that resides, and make sure the data is secured, the data is clean, and at a given time you could use it, and entrust it. So those things are, from a journey perspective, that's where we are. And I believe that from moving forward from a mission, mission perspective, any time at a given time, any user who want to bring and you leverage from AI/ML, that can happen. Sagar, it's great to have you. Thanks very much for your time. Thank you 

John and Catherine, thanks very much for joining me. It's great to see both of you. John, we were talking before we started the cameras rolling and you used a term regarding artificial intelligence that I want to start with, and that is there is a lot of low hanging good remaining still for not just the broader sectors, but federal government in particular. What do you mean by that? What what's the definition of that term and why do you apply it that way? Yeah, I think what I was trying to get at is that we have a tendency, I think, to look at AI immediately go to the existential threat that it poses. And there's so many things that we can overcome in terms of sort of challenges as a nation, and as a world that AI is going to be so well-suited to help address. And so I think about things like bending the cost curve in health care, something we've tried for many, many years to do, and I think relatively unsuccessfully. And so when we think about something like that, the ability to look at patterns in data and see how we can address these types of health challenges, look at some of the models that can identify things like cancer, emergence of patterns that lead to or predict cancer. Think about things like climate change. You know, really a critical issue for us. Discovering the patterns within that data to unlock some of the solutions to those challenges. So I think it's important to look at this technology and frame it in the context of the amount of good it can do before we get to sort of the substantial crisis that we tend to talk about, I think because we're constantly fearful of emerging technology. Catherine, you've told me about a project that you're working on with NIH that John mentioned cancer, and there's a cancer pain application for artificial intelligence. Give me a thumbnail of that, because it's, I think it's pretty remarkable. Yeah. So I've had the great fortune to work with the National Cancer Institute and their scientists on detecting chronic cancer pain. And this is not something that is socially good for AI, this is socially necessary for AI. And in that work we're building really robust multimodal AI models, not just warm and fuzzies, very socially good type of exercises for really helping cancer patients, but real cancer data. What that means is that from even a single smartphone selfie that the cancer patient is filming in the comfort and safety of their own home, we're able to extract frames the faces from three dimensional landmarks, the information of their voice, what they're saying and feel of all of these, what we call modalities together to be able to help and assist the oncologist or the research nurse to figure out exactly what level of pain is this cancer patient in. It's very difficult and challenging, but as you can see, the long term results is really enabling a data mission for health. How much of that work revolves around understanding one particular patient and using that patient's data as predictive vehicle for the future for that patient? How much of it is combining all of the data for all of the patients to try to predict what somebody who may today might not even be in the system yet, might experience six months from now or a year from now or some future? Yes, that's an excellent question, because this is a humongous volume of data. The end point for this clinical trial is 112 patients. I've been lucky to work with the Booz Allen team at NIH for about three years, and we've done analysis on 29 patients, which we published in a scientific journal as well as now we've got 80 patients. So as you can tell, we have terabytes of data for all of these cancer patients and they're from various walks of life. We talk about ethical and explainable and responsible AI. They are different demographics, different ages, different stages of cancer and different levels of pain. And when we train AI models in such a broad swath and diversity, what we like to say is that the ending kind of final prediction model is generalizable, meaning even if we don't see that cancer patient, can the AI model learn to help distinguish pain. A couple of things there I want to come back to you later in the conversation, John, you wrote recently, “We're at a tipping point where AI is permeating everything,” and you talked a moment ago about, it's kind of human nature to be at least apprehensive about new technologies like AI. What do you advise people who are federal government leaders who are starting to roll out use cases, see successes, as Catherine talked about it at NCI, but have people that they're leading who are maybe experiencing that apprehension or just outright fear, how do you help them manage through that? Yeah. So I think it's important to realize that there is a shift that is taking place, right? So I think we've lived through the hype cycles in the past. This is fundamentally different. I think Generative AI would liken it to the advent of the graphical user interface for the PC. This transformed the way we were going to interact with AI. It has given us a natural way to program AI and a natural way to consume the outputs from AI. And so I think the first thing I would say is recognize that we are a shift. This is a paradigm shift in AI, you think about all the troughs and valleys of, the hype cycle in the past. This is a shift in the line up. We are no longer on that same plane. It has altered. And so I would encourage you to embrace it. This is a technology that is truly going to transform, I think, our ability to bring analytics and data driven insights to missions. I think it's important to also lean in and educate. This is something that achieving innovation through this technology requires a sort of think about three things predominantly. First is start with the problem first and work backwards. The tools are amazing. You know, we're practitioners, Catherine and I, but don't become enamored with the tools, become enamored with the problem and work backwards from the problem. The second is embrace sort of a culture of innovation, which means that you have to be willing to take risks. And that also means by definition, risk involves sometimes failing and being willing to fail. Understand that failure, celebrate that failure, but move on from that failure. Right. And so I think that's the second thing you have to do. And the third thing is, is innovation is, I think, everywhere and always everyone's role. But there probably is going to need to be an organization who champions it and drives it. And so like within Booz Allen, we have a CTO organization that we sit with them and our goal is to look over the horizon, to think about those emerging technologies and how those technologies are going to transform. We made investments around eight years ago in anticipation of where we be today, and I think the government should be thinking about those types of activities as well. Catherine, we have about 2 minutes left and you talked about ethical AI, and that's a great subject of conversation in the federal community for sure. And broadly, I think across the technology community regarding AI. Are the ethical implications of using AI in a setting like you described at NCI, different or additive compared to, in some other sector because of the nature of the health care mission? Or are the concepts pretty similar? I think that when it comes to people in particular, the ethical standards are much higher. This is embedded even in non AI routine biomedical types of work. So while it's incredible to be able to get the chance to work on the clinical trials from an AI perspective, as you might know, there are institutional review boards that are required just to get the people signed up. So there's an added response ability that whatever AI, whether we're handling the data, the types of models are evolving. Even our interpretation and the way that we explain it needs to fall in line with some responsible standards. Catherine. John, it's great to have you on the program. Both of you. Thanks for joining me today. Thank you. 

Joe, welcome. It's great to have you here today. Thanks for joining me. What is OIT’s role inside VA to help the organizations, VPA, VHA and cemeteries harness the data that they have, coordinate the exchange if necessary, inside the organization, so on. It's a question we get a lot, right? Where, does a service provider that is really tackling very different challenges focus on customer outcomes, right? So what do we do from a data perspective is that we're providing a federated enterprise platform and summit data platform. We're providing the experience and knowledge and skills to tell people both within city and in our business partners, if this is how you solve for data at enterprise scale, this is how we go out and identify authoritative systems. This is how we catalog them. This is how we build curation pipelines so that people understand when they're using a particular enterprise data asset that it really does represent what they think it does. And that it ties back to a transactional system that was very much involved in the business relationship with a veteran, their family member, a caregiver, etc. Because at the end of the day, we're using data as an accelerator across the department to build seamless veteran interactions. That's what we want to do as a service provider. We want to build great IT products that make outcomes better for veterans. That seamless idea is an interesting one to me too, because my understanding from talking to some of your colleagues at VA is that you want that to be seamless inside the organization, too. It's not just for folks that are coming to VA from outside where there's caregivers or, folks that you're, the vets you're supporting. Absolutely. When you look at the products that we're building across the department, right? Master data is important to that. Ensuring that a veteran only has to update their address information once with the department is huge. That boasts accuracy. It's respectful of the veteran’s time. It makes sense in the digital age that we live in. And that means that you're integrating with other service providers internally and then that's from a transactional perspective. But then if you look at the analytics side of the house, there are pockets of interest across each of the administrations. You mentioned the staff offices that support them that simply just need to know what's going on in the rest of the department to plan their workforce capacities, building facilities, think about the impact of policy changes and laws on the rest of the department, and that becomes very much an internal play, right? For, using those data assets. So, absolutely right? We're enabler not just for those external transactions, but to drive efficiencies inside the department. How much and I don't I'm not asking for specific percentages, but how much of the work that you and your colleagues do on data in OIT is strategic and how much of it is nitty gritty really tactical stuff about data standards and the infrastructure, the pipes that move the data and so on. So a healthy balance, right? If you look at the VA strategic priorities, we want to treat data as a strategic asset and that means you have to have high reliability in the data that we curate, which means you spend a lot of time in the nitty gritty which some of those transactional systems ensuring that they represent those business processes for the Department VA profile is a great example of that, where that is executed from my area, we're very transactional, very much master data centric, but it's a key feeder into our analytics ecosystem because we're talking about all of the core data that you would think about, about a veteran. Where they're, where they live, where they receive care, what their mailing address is, what their phone number is, etc. all of those kind of contact data demographics in one place that served up across department. So that I think is a reference implementation, right, of how you do that, which then leads us to having great patterns, great architectural overviews, and say “Hey other teams, this is what's been successful” and you can integrate with this because obviously it solves a need that we have across the department, but it also serves as a bit of a beacon for how do you do this from a transactional practice and how do you integrate it into the analytics ecosystem? You mentioned the strategic plan and I've talked to a number of your colleagues about it, is that your primary roadmap to determine that you're at the place in the journey that you want to be and how tightly is that woven to a timeline? So yes and no. I mean, obviously we published these plans for a reason, right? It’s where our leadership wants us to go, and that's informed by the administration. It’s informed by public policy. It's informed by what Congress has to say. Right? And we do that as public servants to tell us, hey, this is how we're going to do that. The progress that I think that is even more interesting for us as an organization is the fact that we do so much measurement around the experience for veterans and for employees. And I think that's a even better barometer. You know, in a day to day micro level, how are we really executing towards that strategic plan? Is, are we moving the needle on those experiences? So, there's a balance there. Of course, you have to have a long term strategy. You have to have a roadmap, you have to know where you're going if you're going to get anywhere of value. But measuring the incrementally, right, in terms of not only what we're delivering from an IT product perspective, but how satisfied were people with those products? What are those feedback loops look like? Are they positive, are they negative or are they indifferent? Right? All of those are valuable and all of them are reintegrated into what we do. And of course, that's a life cycle of informing, the next iteration of the strategic plan. Is measuring those or responding to those indicators as simple as if the scores are going up, you keep doing that, more of that. And if they're going down, you do less of that or do it different or is it more complicated? I think you have to take that into the context of the service that's being provided, the external factors that are at play, whether that's different changing, you know, the aging veteran population, veterans coming from different walks of life, things like the Pact Act that have changed the priorities for what we're tackling for the veteran population. So I think it's there's a lot more calculus involved than just say we swept the floor and it's clean, so we're going to keep doing that. It's more of a measuring the structural viability of the building that holds that floor and then how do we learn from that, to build great structures in the future, I think would be probably a little bit more nuanced version of that. What are you laying the groundwork for now that may not mature for a year or two or whatever the next phase is on your data journey? So I think if you look at the VA in that journey, it really began in earnest when the electronic health care record was pioneered by the department years ago. And if you look at that journey of, hey, how do we build enterprise level reporting from across a very large health care system, and then how you do that for a benefits organization, how do you do that for a place that's managing cemetery space, providing life insurance? VA home loans. Think about all these things that we do that are singular companies, right? And other places and a full industry vertical. And so when we we look at where we're going long term, obviously we want to build, we don't want to lose capability that we have today from an operational reporting perspective. But when I get ready for the ML/AI generation and you can only do that with great data management, so if you don't know where your data comes from, if you don't know what it represents, you're wasting your time with AI. So in the next couple of years, I think that we're going to mature those practices across the department and get them into more quarters to unlock more data that we maybe didn't necessarily have great eyes on, because in a singular reporting, descriptive analytics world probably wasn't that interesting taken as a whole with everything else that we know about the veterans and the departments, it becomes very interesting. And those become additive to our training set for those AI solutions down the road. Do you think that you have as an agency, your arms around everything that you have now? Because that's taken a lot of agencies a long time to just even know what else is out there to be able to consider. So we've made a lot of strides with our enterprise data catalog program. And what that's really doing for us is ensuring that you're working with the Data Governance Council, we've got good eyes on the transactional systems that are out there, the various systems that are either powering websites or powering internal applications, but mapping them to the business processes that are identified across the department. How do we ensure that we're making great decisions about those being authoritative for that process? How do we catalog that? How do we make that broadly accessible to the VA employee population contracting population for that matter, so that we understand the coverage of that? And I would think that in the last two years we've made significant strides on that, for two reasons. One, we're modernizing a lot. We've got broad modernization programs in place at VA. Whenever you start modernizing, you inherently have to do a study of where you are today, how it's working, what datas flowing through, what systems, how they're interconnected, which really informs a lot of that, right, in terms of a data asset inventory. And then the second part is really just doubling down on being a high reliability organization. If you look at all of the architectural pieces that we track in various systems at VA, we've got good eyes on what IT systems are out there. We've made great inroads in partnering with the business to understand what they were doing from more of a business-directed IT over the years and integrating that into the same governance framework that OIT uses for our own products. So that's been very important to that. So I would say that is the journey ever complete? Of course not. It's always going to evolve. There's always going to be something new that we have to figure out how to put into our, how we track architecture, how we track data assets. But as an organization and, you know, in my time here, I think over the last couple of years, we've made significant progress and I believe we will continue to. And as we modernize and build new things and integrate new SAS products from the outside, we're going to start at, hey, we need to know what this represents, we need to know how this integrates with the rest of our data ecosystem. So we get that force multiplier as well. Thank you very much for the insight. Thanks for taking time to come to do this on a Monday morning. We really appreciate it, Joe. Thanks for having me. Happy to do it. 

Well, thank you all for being here. You've gone through this journey and you've told me about it. Your CIO, Ann Dunkin, is really hands on when it comes to day. She's told me about the day, the journey that you've undertaken at the Energy Department. What has that looked like from your perspective, where you started versus where you've gotten and how does your experience at other agencies inform, the way that you're doing what you're doing with Energy, Bridget? Like you said, Ann’s taken it very serious, she brought in a Chief Data officer. So I started at Energy last May. He came last July. And in the time since then, or most of our labs and headquarters have started adding chief data officers. So they're seeing the importance, they’re understanding that AI is only going to be as good or any of our data based on what we're putting into the systems. And so working with Robert, Chief Data Officer, we're looking at how we can standardize it, because today everyone has different plans, everyone has different methods, but we need to share the data. And so now being able to replace it is one of the areas that we're working on. And from other agencies that I came from, they are looking at it a little ways to where they've started to bring on chief data officers, but actually collaborating. To learning from ours and seeing how they're doing things over there. You have the wrinkle of exchanging data, a lot of data with non-governmental organizations, the labs and so on. What does that introduce into your modernization journey and also your optimization journey try to make sure you get the most out of the data that you have. It's definitely increases our risk because we are sharing not just internal within DOE, within federal government. We're doing universities, we're also doing international. So the scientific community is very excited, ready to share data. But that means we're adding that risk of bringing data into our environment. And so those are some of the areas we're working with our CISO and making sure that the security is in place and aligned for the data we bring in, but also those standards. So that someone is accountable for the data. It's not just, we're bringing it in, we're using it, it's great. And so that's definitely made it a wrinkle. However, it's also open us up to be able to collaborate more and see and learn from other people. Trang, welcome. It's great to have you here. You are another government leader with a CIO who is really hands on when it comes to data. Sonny Bhagowalia is, talked about that with me on a couple of occasions, what are the challenges that you and your team see at CBP regarding some of the issues that Bridget talked about and that we've talked about here already today about data? Good morning. Well, I think everybody stole all my talking points this morning, but you can do it again. Honestly, it's it's the same theme across all agencies. Right? We have a lot of the same problems that we face today. And that is first discovering where our data is. What data do we have out there? What data assets, where do they reside and how we inventory that data? And then the next thing that once you inventory that data, how do we ensure the quality of the data? How do we ensure that, you know, the data that we have is really good data so that it can be used for these new technologies, things such as models or anything like that? Right? Because for even for AI, if the data is garbage, it's going to give you a garbage model. So that's the second challenge that I face. And then the third challenge that we face also is data governance is we have data, right? But how do you use that data? What are the frameworks around it? What are the guardrails around it? What can people do with that data? What are some of the things that we, the policies we put around those databases where can be stored for how long? So those are some of the things that we need to consider as an agency. And then lastly, most importantly to me actually, is the people aspect. And really that means educating our workforce to let them know the importance of data. Data is an asset now. We're trying to get our organization to be a more data driven organization, and in order to do that, we need everyone in the workforce to understand their role and how data affects them. Not just the executives, not just the technical people from OIT, but our mission partners as well. How will they be using that data and how important is that data to them? How can it help them accomplish the mission that they are looking towards? You talked about governance and you used the term guardrails. A lot of people use that term. Why do you think that's an apropos term to use when you're thinking about governance around something like data? Well, because, because we are mission focused agency. I mean, our first thing is to protect the land, facilitate lawful trade. But the data can be not only our strong point, but also our weak points. If they're, if we're not managing it correctly, if we're not using it correctly, if we're not securing it correctly, that's why we need these guardrails. They’re called guardrails, but really guidelines in terms of what can be do to ensure that we are using it for the purpose that it's meant to be used and that we're using it in such a way that we're not losing public faith. Kathryn, welcome. It's great to see you. You were nodding, pretty enthusiast, actually, when Trang was talking about discovery. Why is that something that resonated with you about the work that you do in it? Noah So Noah is a data organization that likes to think it does science in all the best ways possible. Of course, science is our core. Soon discovery and the potential that our data creates on a broader scale when we're solving climate crises, when we're looking to actually really deliver more value to the public. If you all have ever seen, I don't know, your phone's gone off because you had a tornado warning or severe thunderstorm. Yeah, that's NOAH. It's the National Weather Service. So how can our data come to not only protect more lives, but in the long term aspects of the climate. So discoverability, we tend to create pipelines of data, hyper manage for what we expect in the operational lineages. And I really think there's opportunities for enhanced discoverability as well as innovation. And that's really where my team is focused. Yeah, What does innovation look like in the data, the data discovery, the data curation and so on, space look like, what does that look like? Yeah. So data modernization starts with your IT. We're looking to cloud. Most prominently not gov cloud. We're actually going to go commercial cloud from the standpoint that we have open data. We want to share our data. So can we tailor up to high integrity as well as high availability, meet our requirements, but also go in that direction. For discoverability, we also have the unique challenge of NOAH having a lot of different types of self describing data formats. Some very well richly described and not. So it's actually at a metadata level, which is a fun term that you get into when you talk about data, but it's the key enabler. You can either discover it or not because of how rich that metadata is. You can write your APIs in the most viable format. So there's an actual transformation that we're looking to do to, again, bring it out of this like very, I would say, low level data flow that we have for operations to really enable more innovation at a data level just based on formats and the access through discovery. Kathryn thanks for that insight. Osama, welcome. What do you hear among your government peers and colleagues that resonates with you as far as what you see when you talk really broadly across the enterprise, the federal government? Yeah, I mean, I think to me and it's really cool to have such different missions represented up here today because the data that you are going to try to harness for the problems that you're trying to solve, you're going to approach that differently based on the mission, the type of the data and the way that data gets used. So, for example, you know, you think about NOAH and weather data that might be coming from, you know, satellites in space. And so the way you collect that and then have to transmit to a ground station, the size and the speed and performance of that data is going to be inherently really different from, for example, CBP, where it might be, you know, maybe law enforcement related data, the way you're tracking your sensors on, you know, on an officer who's in the field and and that maybe doesn't have connectivity versus, you know, in the energy world where you might be talking about a lot of really fast transactions and the way you need to think about that differently. So I think starting with a mission and understanding what are the problems and we've heard kind of that theme, what are the problems that we're trying to solve, but then saying, okay, well, what does that mean from a data perspective and how do we need to architect and optimize that not just for executing the mission, but then also on the other end of the spectrum, creating environments for us to do advanced analytics and be able to get insights from that data. Because one of the things that I think is really exciting about where we're going with kind of this data evolution and revolution that's happening right now is that it's not just about solving the problems that we're already solving today with our data better, faster, cheaper, safer, but it's also enabling mission and business partners with new insights and new ways to do their job with data that maybe they wouldn't have thought of using before. I think that's really exciting. Mission transformation. I have one more question for you, but, well, I have a lot, but one I'm going to ask now, but I want to tell you all I do these a little it differently than I think most people do. I would like you guys to be really involved from the beginning instead of me going on and on for 45 minutes and then giving you this little bit of time at the end. So if you have questions starting like as soon as I'm done with my next one for Osama, I encourage you just raise your hand and get involved in the conversation. You'll be first. Thank you. Because that's the hardest one is to get somebody to go first. So kudos to that. So if there's a microphone or anything, we'll have it come up. Here it comes. So just if you have a question, raise your hand and the microphone will come to you and when you ask your question, if you can tell us first who you are and who you're with so that they have a context for the question and then just go right to the question so we can get as many people in as we can. That's great. My question for you is, you wrote recently about three phases that agencies should walk through shifting to a product mindset, cloud based data platforms. And the third one is starting with what's the business problem that somebody is trying to solve? Is there one of those that has proven to be more difficult for government organizations than the other of those three, Osama? Yeah, I mean, I think the cloud platform one tends to be a little bit a little bit further along right in the in the government overall journey going to cloud. And I think using that as a parallel, as you think about your data journey I think is working well for folks right now. The big difference I'm finding between the cloud infrastructure, cloud services and the data side is the fact that more people need to have their hands on that with data, right? You've got to have the business and the mission more involved, right? Security. More involved. IT, more involved. Policy, more involved. And all those stakeholders I think adds a little bit more complexity. So it's not a 1 to 1 parallel of the cloud journey that a lot of our our clients have been going on. To me, the one that I think is a little bit trickier is probably the product mindset thinking about applications and services as a product. I think it's something that a lot of our government clients are starting to be able to wrap their heads around because that's how they interact with Apple and Google products in their personal lives. They understand, okay, you know, the email service is really a product. They keep dropping you for social media, they keep dropping new features and they're always responding and self innovating. And they kind of understand that when you think about data as a product, that's, it's a little bit different. And I think that a lot of times, you know, it's hard for people to maybe feel like they can really own data like that, especially in the government context, because you have a lot of baton passing that happens depending on what part of the data value chain that you're on. And so for folks to really kind of culturally feel like, yeah, now I can take ownership of this and I can start listening to my customers and stakeholders about what do they need, what changes should I make or how can I start unsolicited, bring new value to them. I think that just culturally is a little bit of a mind shift for folks. That seems to be a tough one. Kathryn, Trang, Bridget, anything you want to add to what Osama said before we, start the questions? Everybody good? I can add one thing to that. I definitely second that we're seeing that in Energy where, yes, we have a lot of things in the cloud, but it's part of how we're doing that change management because we have people that are like, it's in the cloud, it's fine. It's like, no, we need to change that data. We need to move it, or we have ones that are like, we are definitely in this cloud only and we don't want to interact with them. Like you need the data from another cloud and some of those processes definitely still, and we call it the eighties like they're still in the mainframe aspects versus the real time that you can get if you're able to leverage the cloud to the native capabilities. Cool insight. All right. Name who you're with. And the other thing is, if your questions directed to one panelist in particular, let us know who that is and go for it. All right. Eric Kavanagh with DM Radio. The longest running, showing the world about data. As I like to say, year 17. We’re doing it a long time. You talked about guardrails, but I'd love any of you to comment on this. It seems to me, especially with respect to generative AI, those guardrails are going to be in your rag models, your retrieval augmented generation, which can be extremely sophisticated because you can design workflows around how you're hitting the all of them. So you can hit a you can hit a set of policies on the way out. You can hit a set of policies on the way back. And I think that's probably going to be the answer. But can any of you comment on what you see in terms of sophisticated rag models for enforcing data governance and data quality and data use. Go ahead. Thanks for starting with a non-technical question. Thank you very much. Anybody want to take that? I'll take a first stab at it. So with AI and GenAI, I think having what, putting in air quotes, those guardrails or the let's see policies around responsible usage and ethical usage, of AI especially when it comes to GenAI, is really critical for any federal agency. And that's probably why there's that executive order that came down from the president on the responsible usage of AI is because it's such a new, it’s not a new technology, but it's a technology that's taking full force right now. And in order to ensure that we are not using it, I think a lot of times when we develop AI we need to ensure that the AI that we're developing is actually being used for what we had intended it to be used for, right? So developing and ensuring that we have the appropriate processes and procedures in place to take a look and ensure that we don't have the biases in the model, that the model is not impeding on any civil rights and liberties or any safety issues when it comes to the public. So those are things that as an agency we need to be looking at and we are looking at. And even before we develop any type of models, we're looking into developing like what you said, the rights, the process, the procedures and the responsible usage of have. Maybe just one quick thing. I love the term guardrails, because to me the intent of that is to allow organizations to move as fast as possible without going off a cliff. Right? Like just like you think about on the road. And so you want those policies to be in place to provide a spectrum, where, people and dev teams and data product owners can operate with some freedom and be able to experiment and do things within, a certain set of confines. And so I think, I think that's a great term when it comes to that. And then I think the other key thing we think about, AI, especially with the missions we have represented here, there's a component of this which is executing a mission and using data and AI to support executing that mission specifically. And then, there’s also, I think, DOE is a great example of like providing data and as a platform for industry and for society to be able to innovate on top of those. Another great example where you're doing some of that and then providing data sets. And so then when you think about AI in that context, who is using it and for what purpose, I think comes into play as well. So it becomes quite a tricky problem to solve. Kathryn, Bridget, anything you want to add? Not particularly. He talked about the platform for artificial intelligence. When you got to Energy, what what did that platform look like and what have you had to do in the meantime to build out the vision that the agency has for what you want to do with AI? When I arrived, we had and our labs, a lot of different platforms, but they were more like in the non production, just discovering. So we actually created a discovery zone that is a DOE environment where we have Google, we've added Microsoft. It's interesting that you're mentioning commercial because we've done commercial Microsoft. And now that they have federal working, we started adding that because again, our data is in all of the platforms and we know that we need to be able to share across. We also know that we also are using AWS and Oracle. So we have all the clouds, but looking to make sure that we can incorporate where our data is with where our AI is going. Excellent. Eric Great question. Thanks for starting us off. Who's next? Anybody in the audience have another one right back here. Welcome. You're making my job easy, by the way. I appreciate you. My name's Dave Anderson. I'm with Prestructure and we're an ethical AI strategy company. And the low hanging good concept really resonated with me. So thank you to the early panelists to bring that up. Inside your agencies, are there any new projects that highlight low hanging good that can build trust with citizens and residents that you're serving? That's a great question. I do. Strikes me there's probably a lot of examples like that and you're challenged might be Kathryn, to tell people that that's what's happening more so than anything else, because you've been doing these kinds of things for so long and now just have these amazing new tools to be able to do those things. Yep, I'm going to answer that one. So, yes, we are moving forward with some of our AI workflows really focused on blending of different data sets that were impossible to blend in the past. I can think of this one, TC PRIMED, may not mean much to you, but in terms of blending satellite historical data, plus model data, plus outcomes can result in better forecasts. So we are working in that world. I think why it's just a challenge for us is our data is in silos. Silos of beautiful excellence, but things that were hasn't come together meaningfully. So we have to actually because we don't have our IT modernization fully in place and searchable, discoverable to be able to do AI, we're having to cherry pick a little bit to pick some great examples of what could be and use those to catalyze the future. We're also doing a lot of AI-based weather modeling. You may have heard that there's some industry working in that world. So NOAH, as the predominant organization, building our nation's weather models, is moving to an AI ready approach to that, as well as an AI model where we could do faster, less expensive in terms of compute forecasts. The challenge with it is you bake in the errors, right? You're baking in the errors of whatever model you're using to train that data. And so that is going to be our challenge is how do we incorporate AI into our operational workflows to reduce that compute, but also to alleviate the errors? And maybe a quick add to that is, I think some of the best social good kind of happens at the intersection of multiple domains and, an agency's missions. So you think about, you know, food and nutrition data overlaid with a map and, you know, zeitgeist-type information and socioeconomic information. And then you add weather patterns and you can look at who's going to be potentially most affected over a certain time, right? Time horizon based on weather patterns and things like that. That's sort of where that the intersection of agencies meets, I think is, there's some really great opportunities. So I'm hopeful, and Booz Allen has sponsored a few hackathons over the last year, just even engaging with universities and some of our staff on the weekend with their free time, because they’re passionate about it, but bringing some of those kinds of different datasets together and giving them opportunities to ideate and come up with prototypes of the applications and things like that, that can help. But I think that's something I'm excited about kind of as we, as the agencies get this base level of platform and maturity, then, you know, and again, we talked a lot about like interorganization sharing and interoperability. Yes, we start to solve that more on a broader scale. I think there's some really exciting things we'll be able to do, and it'll come through government innovation and industry innovation and kind of that partnership together. I want to come back to that, but I want to see if Bridget or Trang, if anything, you want to add response to that question? I would add quickly that again, we have the silos and data and one of the areas we're looking is for the customers to know where our grants are. And so we are layering it with the states because we do have if you go to our website, there's grants, but you would have to know like, is it in this group or specific office? So working with our public affairs is how we can make that more accessible because the average person doesn't have like 30 minutes to go through our site or is it EPA or is it IRS? Like, so even working with some of our federal counterparts is to make it easier to get some of the help that there is out there. Anything you want to add? I just want to add on, they are absolutely correct, data silos is a challenge that all federal agencies and most family agencies are facing, including CBP. And I think that's why it's important to have the data discovery, try to figure out where those data resides, who we're sharing that data with, what, and pretty much inventory, what we have so that we can get rid of duplications and being able to manage that data better so that we can better meet the mission, because if there's lots of data out there that aren't speaking to each other, when they should be, we should try to resolve that. But first that comes with trying to discover where that data is. And I imagine inside an agency, DHS, that becomes even more complex because you're not just thinking about CBP, you're thinking about the other 20 some components, too. And there's value, I imagine, in some of the data that you have for the Coast Guard and the Coast Guard has data that's interesting to you. And so we could go throughout all those permutations if we can. Absolutely. We do share data with amongst the components. And that's the thing is, is that we need to know exactly where our data is in order to appropriate share the appropriate data with upgrades, to make it more useful for them to help them with their mission space. Okay. Next question. Raise your hands and the microphone will come. And while you're deciding that you're going to ask your question, I want to ask you, Kathryn, you've talked a couple of times now about IT modernization. What's your sense of where you are in that journey to get to where you want to be, to eliminate those of excellence? Which is a term I love, by the way. So good. Nice. A graphic of science. It’s funny because it's true. It's made me happy. We're well along the way. NOAH is a band of multiple organizations, just as most federal agencies are. And I think we're getting to the point where we realize the interoperability and necessity, the dependencies of our data across all of our workflows, and using that as actually a catalyst to think through what's possible. What can we do? My organization is leading an enterprise IT development, so not only did we look at, all right, let's just set aside some of our legacy systems, let's bring them all together, but let's also bring the data together, but let's also do it in the cloud. So we just decided to go in a step function because we know we can't sustain business as it is today and continuing to invest in bigger and bigger pipes between our systems isn't going to work. So you put data at the center of your architecture and you think about people accessing the data and the patterns for it, that can actually enable a different view of IT modernization, but I think it's fundamental in a positive way for catalyzing change. All right. Next question is from me, it looks like Osama, you were talking about the desire to exchange data. And one of the things that has gotten management experts completely separated from IT experts excited is I think you all know how much time we've spent in government, in the government community over the last, however many years thinking about reorganizations of different agencies, of moving some bureau from one agency to another and all of that. And one of the things that is at least on the mind strategically of management experts is when we get to a state that all of you, four of you, have talked about, that might not be necessary at some future point because the data that needs to be shared among each of those organizations, that would cause someone to say, well, this box should be on this tree instead of on this one, that won't matter anymore because I'll be able to send that stuff to Bridget. Doesn't matter. She can be at Energy and I could be at some complete different organization. Is that, how far are we technologically from that vision? Do you think, Osama? I mean, from a data technology perspective, I think we're on the path towards that for sure. I think, you know, especially with kind of the rise of a sort of data mesh-based architectures that allow you to kind of keep data where it kind of most naturally comes into an organization and kind of optimize there, but then have access to that and be able to extract it, ask for it and have kind of a controller to say, hey, is this makes sense, is the rules, okay, this person have the right access and security roles? What are they trying to do with it? And then even on like, for example, like the defense and intel side, there's some pretty, you know, leading technology capabilities that are even allow you to follow that through so that even if something changes, and you're like, you know what, we need to pull that back. They're starting to be able to do that as well, you know, through watermarking and things like that. So technologically, I think we're for sure on the way there. I think it gets back to kind of the people and the culture side of it. I think that's going to be and really for all IT modernization we've seen over the last decade. Right? That's sort of the long pole in the tent is getting the people and the culture to kind of be comfortable with new ways of working and doing things. But I'm excited that having some folks pave the way and show the pattern and seeing the outcomes of what you can do and how much faster and how much more mission transformation you can have with taking advantage of some of these technological innovations, I think will hopefully pave the way for that transformation to go faster. Anyone? Go. You might as well go again. I mean, you got to gig. Thank you so much. So you probably know these large language models. What they do is they convert text into numeric values, right? Actually vectors. That's why they call it. They talk about embeddings being vectors. It's a point and a line basically that you do these similarity searches and that's how it actually fills out the text to predict where you both, several of you've talked about silos, and the challenges of silos. We know all about this, data warehousing was invented to be able to hack through the silos and get some clean view of what's happening across information systems. And it seems to me these large language models can kind of fill that gap, but again, with the proper guardrails, with the proper embeddings, etc.. But you talk about the different silos, if you've vectorized these silos, then you have a sort of overlay of the large language model and now you've enabled text conversations which all this data. So what, I'm kind of going with this is that we've spent so much time and money over the past 40 years moving data around, all this ETL stuff, moving huge amounts of data, from this system of that system, this system to that system, I think that's going to go away with this new model of being able to query massive numbers of datasets. You think about what these like ChatGPT has been trained on. I just want to get your thoughts on that. And then the other side is the transparency ChatGPT is now a black box or as Llama 2 to is open source. Are there concerns about leveraging black box technologies like ChatGPT, because we don't know how it works. And even the ones that are open source, the people I know who really know this stuff, they'll tell it like Usama Fayyad. He runs the Institute for Experiential AI at Northeastern. He says these models are too big, they're not supposed to work. Nobody knows why they're working. He actually says this it's kind of funny. So what do you think about large language models kind of solving with guardrails in rag this whole data silo problem that everyone is talked about and what do you think about transparency? Thank you for that. Anybody want to take that? And I have a follow-on to that too, actually. Sure. Great question. We're talking silos. So I think at least for our workflows, our challenge, the silos don't come because we liked building silos. Where is dollar? Where is Congress appropriating money and the culture of “I own it” end to end has permeated into specific systems. So this is a challenge that I don't think we'll be able to fundamentally overcome with natural language processing. However, access we talked about data meshes. Thank you, Osama. I think when it comes to the accessibility of data, where to find it, how to bring it, what does it have in it that is fundamental. I can think of a great example, rain. What are other words we use for rain, precipitation, drizzle, moisture could all mean the same thing. The advent of natural language processing has actually disrupted that in a positive way. So long as we have the self-description within the data to understand that they're connected, that vectorization is so key. Now, if we want to actually change because, I go back to that operational mindset. Ours is about huge amounts of processing, right? It's a lot of array and 4E transforms and we can go into all the mathematics with smarter people. But I think the language processing to bring it together is valuable. So I see a really public access facing side of it as key. And then internally I would think we could actually use, not natural language processing but AI and ML to modularize and actually abstract smaller parts of the data that would have been mathematical models internally. If either of you want to add anything to that, that's great. And I'm also curious where you are both infrastructure-wise. And the other piece of this that everybody's talked about is culture, where you think you are as far as being able to bust whatever silos still exist in your organization. So either of you want to follow on to that question. So to answer your question about infrastructure and silos, and culture, I think we're still at our, we just hired a CDO last year, so, you know, the organization started to realize that data is important. So with that being said, we, to me, we're still at our infancy phase, meaning we're still educating our workforce in terms of how they can use data, how they can leverage data, how they can use data for AI/ML and large language models, how those can help out, right? So having to educate the workforce and I don't want to say it's a culture shift almost because the biggest, the most important thing to CBP right now is meeting our mission, being able to protect the borders, facilitate trade, and then, but in order to do that, I think what I, what we find out with an organization are we're going to have to do is remind folks of you can use data, you can use technology such as AI/ML to help make it more efficient and effective for you. So imagine being an officer or a nation out in the field and having in the palm of your hand, something that can tell you and detect to say, hey, that's not an animal crossing the border. That actually is the first thing you should be looking at that corner of the border to see if it's a foe or a friend or whatever it is instead of having them use it waiting for that data to come back and forth. So with that, we're bringing more of the technology to the edge or officers and agents in order for them to be able to use it and, but that can help the understanding, the importance of it, understanding the usage of it, understanding the data, understanding that the technology can help them along the way. Okay, there's more there. I want to dig into. Do either of you want add anything on? I would say with us, it is definitely everyone's admitted that we do need to share the data. And part of the pieces is that we are, we don't have today an enterprise wide data structure. And so we are working on that data catalog, the data structure, of how we're going to govern our data across the enterprise. But because we have a lot of different components that have already started the journey, so it's more of meshing together how and where they are to meet, where we can go from it. So we are looking at ways to use AI to help speed that process up. Right now we're starting with our resources, so a lot of areas humans have to come into the agency, how we can standardize that because today we have like four different paths and to owning okay, let's have a path. It's simpler, but then we also can share the data across. So we own that and we're starting to look, and then also working with a few other agencies that say, wait, your HR modernization has started. We need to do that as well. How we can learn from you. How big of a hill was it to climb to get everybody to admit that the data should be shareable because that not every agency has had an easy time with that. Luckily, Ann had already started that before I got there, and it was more of we can't continue the old processes because we were still using even though we are in the cloud, it was a lift and shift, it was an improvement. So it's still doing the mainframe batches, but just in the cloud. So you’re saying we're going to modernize, not just move it to the cloud was the first step. And then once they're noticing that, hey, we can make decisions quicker, it is more accessible and shared versus still doing the manual uploads for data that we can do the APIs and have some of the real time. So it was a shift, I would say probably at least a couple of years. Luckily for me, I came in where they've already accepted the reality that we can just implement it. Well, and it sounds to me like part of the issue there is leadership because you used a phrase that I think is is interesting. You said at some point whether it was Ann or somebody else, somebody had to say, this is just how we're going to do it, there's not we don't have time to survey everybody anymore and decide what people want to do. This is how we're going to move forward. You're nodding your head pretty enthusiastically over there, Osama, I'm when I say that. Yeah. I mean, I think leadership top down fine, is critical for any big transformative culture change like this. I think the part that really kind of hit me in the heart was when you're talking about kind of the trust, and one of the things that I'm finding that's a challenge culturally for a lot of the organizations that I work with is the idea that when we go to this new paradigm where we're using enterprise data instead of me, you know, using just this pocket of data, yeah, I get the benefit of these other insights that maybe I didn't have yet, but then that means that, like, I've got to let of this and give this to like the mothership, right? And I have to start, I have to know that what I, I really need a piece of data for my mission which is critical, that I'm going to get that data in a timely manner and it's going to be accurate and it's going to be managed well and I can trust it to make my decision. That's the part that I think, and it’s coming from a good place, right? Like, you know, our government partners, they want to do their mission really well. And so that's why it just gets, it's harder to let go of something because then you're like, I have to trust you that you're going to take care of that and get it to me when I need it in the way that I need it. And I think that's the kind of cultural piece that that we're going to be probably working with, you know, a lot of our clients by the next 12 to 18 months. That's going to be the big cultural mountain we're going to be climbing. Have you seen government, whether it's federal or others, have you seen government organizations make that cultural shift? And if so, what have they done? Where have they started to lay the groundwork to then be able to build on to get to that state that you just described? Yeah, I mean, actually it was up here earlier, but I think soccer's organization, the GSA, is making some great progress towards that right now, and it's really they're riding on the back of the credibility that the IT organization has made in terms of the IT modernization work that's been done and say, look, we're delivering your applications, we're delivering these services. We've built a credibility you trust us with that. Now, if you give us data, we're going to continue to build on that trust that we've, we've done and we're going to give you these new insights, this new value. We're going to reduce the cost for you to manage this data, things like that. So I think that's a great example of where we're seeing that that trust being built and being able it's got to be on the back of other building that's happening, right, That model at GSA in general. But fast in particular is interesting to me too, because of the context there. So GSA CIO is very highly regarded. Dave Shive, been there for a long time, knows the organization inside and out. But the last past commissioner was Sonny Hashmi, who had been CIO there before. So he had a lot of background also when it comes to the technology side and then was able to go into that organization and say on day one, as we've talked about, about leadership, this is how we're going to do it from now on. And Julie Dunn, who was commissioner before Sonny and Allen Thomas before her, had kind of already started that process, too. It's a date is going to be valuable to us. And then Sonny was able to go in and say, yeah, this isn't stopping. This is going to continue. Is that maybe issue one for people like the other folks that are up here with us today that they, and their colleagues who are in leadership are able to say we're moving forward in this way and you can come along, and join us, but not really leave a lot of of space, a lot of margin. Yeah, for sure. I mean, I think look, it's, it doesn't need to be more stick than carrot. You're right. But I think the example that you just uses is perfect because when you have somebody comes and says, I understand the value I want bring IT and give them a seat at the table with the business, with security, with the mission, and we're going to work together and trust this path forward. I think that's where it really kind of paves the highway for increasing the speed and the velocity of transformation versus, sort of like, hey, IT works for the business and it's a, you know, subservient relationship or, you know, or like, I will tell you what I need, but I'm not looking you to tell me how to do the mission right when there's that partnership and exchange. And I think even in one of the earlier sessions, right, they were talking about innovation. I was John Larson saying innovation is the job of everybody. And so I think when you kind of really bring everybody together like that, that's a big catalyst for change. And I think what you're getting at, there's something I don't know firsthand. I don't know personally about CBP and NOAH but I know personally firsthand at Energy that and has that kind of trust for you to be able to just go and do those things and knows that what you execute is going to fit with the vision that she has more broadly for the organization. In the same way that we just talked about GSA, where Sonny said, Folks, you go do this and I know you're going to do what we have in mind. Yes, that is accurate. And she has support from our deputy secretary. And as we do those reports up, it's oh, we’re just working with science, who's working with our nuclear regulatory on different elements because it's the business and IT versus, okay it's the business and if they would like to, they can work with IT and that it's partnership for us to be successful. But part of the pieces is that delivery and it's almost like our business proposition, like we won't mess it up if you let us be your system owner because everyone's had their own systems. But we're like, if we manage it, we're going to do your security, we're energy, your privacy, we're going to have your data and you can focus on your mission. And that is that trust aspect. And we're constantly keep reinvigorating and keep giving those touch points so that they see that it is something that we can partnership with. Excellent. Anybody out here? Have I missed one? Yes, ma'am. mic is coming. And name and who you're with and who your questions directed to, if anyone in particular. Hi. Morning. My name is Prachi Sukhatankar, I’m one of the leaders within Booz Allen’s Climate and Infrastructure Practice. Thank you for our great conversation today. I really appreciate it you know, I've heard about the whole data lifecycle from acquisition to dissemination. You all started talking about some of the key barriers and I started hearing a touch on some of the transformational techniques or technologies that might really be game changing here. So my question is, you know, there is no silver bullet to some of this, right? I mean, it's that portable work, it's that hard work. You can get all of those different barriers and and techniques. Having said that, to really accelerate the transformation within your own mission, do you feel like maybe there is a single lever or maybe a couple of levers that you wish you had that might really accelerate the transformation for the mission outcomes that you are looking for? So, for example, it could be technology whether that's GenAI, data mesh, whatever, or it could be something completely non-technological that's perhaps outside of your agency or your charter. So just curious about that. Excellent question. Thanks for that. Wish lists. The wish list? Yeah, that's right. Well, I think we're fortunate enough to have top down support in terms of technology and what is needed for the mission. We do have mission partner support. We in the past, you know, a lot of agencies refer to our business component as customers, right? It's now we are no longer OIT technology and is no longer looking at our mission partners or mission individuals as customers. But we're really, the theme here is mission partners, right? And then what Sonny Bhagowalia, our CIO, has done is he's established what is called trusted partner initiative, where we meet with each of the mission partners to kind of figure out, you know, what do you need, what can OIT do to help push your mission forward, to make your mission a little bit more effective. So one of the things you know, I'm going to say it, but this is coming from training, probably not any other government agency. So please don't quote me on this one. But what we need, not only support from our trust partners and, you know, mission top down, but the resources in order to accomplish our mission. There never can be enough resources. When I'm talking about funding, I'm talking about individuals to help out because there's lots that we can do or we can only prioritize so much based upon the resources that we have. So first, on my is probably going to be the necessary resources we need in order to accomplish what we need. And then the second thing is I'm just helping everyone to understand comes back to that culture and the data shift and the paradigm shift of the technology. Having everyone understand and change the cultural movement here. And I know you talked about the culture in terms of making sure that we are no longer siloed and we're able to share data and, you know, data, it's no longer than mean monster. And that started for us with developing a data strategy and that data strategy is kind of corralling everybody together to say, and we have common understanding. And now, you know, I just want everybody to understand and pretty much work towards the common goal of what's in the data strategy in order to make data more of an asset to them than the liability, make data more useful to them in order for them to accomplish their mission a little bit more efficient, effective so that they're not out there and let them know that they're not alone and they should not be afraid of technology and they should not be afraid of the change that's out there. So I think those are my two biggest things is, you know, not only the resources that's needed in order for us to accomplish our mission, but for everyone to understand and realize, you know, we get a little bit more educated in terms of the usage data, the usage of the technology, and how we can be a leverage for them versus a liability how we can be in asset. I want to give you a chance to talk about your wish list, too, in answer to the question. But you used that word liability twice in I think an interesting way. I don't know that I've ever I don't know that I've ever heard anybody describe the possibility that somebody, some rank and file person in a agency might think of data as a liability. I would think it would just be either an asset or neutral. You don't even think about it or care about it that much. Is that a thing or is it just something that you're aware of that you're trying to avoid? This is something that we're trying to avoid. I don't think it's a thing per say, I just don't want people to think that, you know, the technology, not necessarily the data, but the technology has the hindrance on their current ways of doing things right. More of that technology is not something that's going to stop them from doing something. The technology is not something that's going to take away their job, but rather the technology is more something that's going to help them out. It's going to spring them forward, and it's going to be more of an asset to them because I think one of the common themes that I've heard a lot of my peers talk about is when they want to use AI or even not just the professional in the private sector, is that AI can replace the job. AI is not going to replace jobs. AI is going to make things better, more efficient, so you can concentrate on doing something that your core job ought to be. Like in our case, it's actually law enforcement, right? So if we're doing mundane things like looking at spreadsheets instead of having our PA take a look and process things for us, then we're not, we're not concentrating both of our time on something we should be doing, such as law enforcement activities. So more, not serving as a as a liability, but more considered technology as an asset you can just bring forward. Yeah. Lower valued work to higher value work, I think is the language that was in the president's management agenda. What's on your wish list, Bridget? Mine’s resources as well and the government, our budget process is two years in advance. So two years ago we may have asked for say, $1,000,000 for AI because we knew it was always around. But now with the executive order and some of those items, it's like, okay, we are de-prioritizing other items to comply with that mandate. The other piece for us, liability, from a data is how the bad actors, the malicious data aspects of, we already are have satellites, we have things that or assets that we do need to protect. But patching together if we are using more the open source models, that you’re able to get more information than you should about some of our areas. So we're looking at it from a cyber lens to make sure the data that are put in models that are not controlled, even on the low side, is unable to be put together, used for malicious aspects. Anything you want to give us for your wish list, Kath? I mean, of course money and people are fundamental here. I would say perhaps a lessons learned and also something that is key to me is this idea of, kind of hit on it with the idea of that the product enablement, the product mindset, it's really an enterprise mindset. It's what can done. And what that's doing is addressing control. Control I think is a fundamental challenge in your organization. It's why people are here. It's they show up. I'm in charge of this, this is mine. I deliver this. And as a civil servant, you don't physically own it, right? So you're here to represent the mission. So we now have a control about mission and delivery and your value to the nation. So by getting out of that mindset of I have to do this and thinking too, how can I do this better, where is the value of this perhaps intersecting in different areas. That mentality shift, and the more that we can imbue that with our people will enable such better solutions to come. I'm not suggesting Osama that they should cut corners, but are there any techniques that you've seen organizations use to kind of compensate for what they wish they had or wish they had more of? No, I mean, I think they, that the things that they are saying that was on their wish list, you know, I would say my list of clients probably matched them up pretty nicely. Maybe one thing to throw out there that wasn't mentioned explicitly, that I think it kind of enables all of those things is probably around training and education. I think the more that we can get people more literate around modern data paradigms and kind of what these terms are what they mean, so that people aren’t, you know, coming up with their own definitions of things and making, you know, taking stances, whether they're for or against something based on that versus really kind of being educated. I think that that could play a huge role kind of more broadly. And I mean that in all parts, right? The management, the business, the mission, IT, even within data organizations, the experience and customers, right? Like I really think education for everybody kind of helps you propel everything forward as well. Trang, you talked a couple of minutes ago about how your CDO is educating people and you and your team are educating people about data. The benefits inside the organization. How are you doing that and how receptive are people to that education process? Good question. That’s kind of my job. So I think we're starting out we've had several things, such as lunch and learns, actually had an AI day looking to do like a day to day. But we're also having things not just internally within or we're looking to talk with vendors as well is in trying to find out what's out there, how can they help us out and vice versa. But when it comes back to our staff and educating them, it's really going out, to Sonny’s credit, having these trusted partner initiatives and being able to talk to them and try to figure out, you know, what is it that you need and how can I as within the Chief Data Officer, be able to help you out? We also have a, I think it's a data working group within CBP that has representation within all of our mission partners. And we talk about things such as, you know, data quality, data consistency, sharing of data and all the things that we as accomplished within our data strategy and trying to figure out from them how we help out and vice versa. And they've come and asked us multiple times, Hey, can you just come talk to my staff here about this topic, help them understand where they fit in. That's the key. Helping them understand where they fit in. Lots of time people do not understand where they fit in in the paradigm of data and how data, they just know that it's there. But because their core mission is just, you know, helping to attack the border or facilitating trade as it ought to be, and then my responsibility is helping them, helping them understand, you know, how that data can help them better achieve what they need to do faster and quicker and more efficient, reliable and secure at the same time. Bridget, that idea of where they fit in sounds almost like an architecture. If you have your IT architecture. But is there or should there be, if there's not, a people architecture where you have very clear not roles and responsibilities, not like job descriptions exactly, but where everybody understands what their level of interaction with their role in that interaction is when it comes to data, is present, or am I thinking about it too hard? No, I think that is definitely where we're headed. Part of the piece now and is that foundational of that the data is here and how we're going to incorporate that or it's being used in the systems. The good part about us from an IT standpoint is that our data governance is aligned in the same caveats that we talk about our systems, we talk about our business and we talk about like our data, those governance boards roll up into one. So we do have those aspects. But today that is not officially a thing yet and we know that that is in the pipeline, ideally in fiscal year 25. We're looking to have more of that role, but this year it's mainly working on foundational pieces and everyone understanding data should be something that we work on. Yeah, Kathryn, I imagine because of the number of scientists that you have and the amount of maybe it wasn’t even called data, maybe it was over the years. I imagine you probably have a lot of people with a very high level of understanding of what that value is. Is that a fair observation on my part. Their data for sure. Yeah. So has it been that as much of an education process as it sounds like it's been for the other agencies that are represented here and more broadly across the government? Or do you think you've had maybe an easier shot at that? In some ways I think it's easier. We have an archive, right? So we are archiving data, there's stewardship, there's the interconnectivity of the data curation. However, I would say that when you look at the scientists, when they interact with their user community to deliver products, right, that's fundamentally our mission. The data, the readiness is so hyper tailored to who they perceive their users are. So the potential usage of data outside of that is not something that they carry because they think they're delivering the mission. So it's not necessarily data literacy, it's an expansive, it's more broader look to what's possible and the idea of innovation and that when you have operations sitting next to innovation, they're pushes and pulls, right? Because you have to deliver. So, you know, you need to get things out the door, but could we do it better? Could we do it in a more meaningful way? That I think is a little bit more of our challenge is how do I use all this different data? How do I use the diversity to deliver more and better. In those kinds of situations like you just talked about, is and I don't mean to pick on operations, but are they getting better at being able to say to the technologists, I wonder if we could or would we be able to fill in the blank, whatever that is, to then have the technologist talk about, use the term the art of the possible. Is that happening more? Is it your sense? I think it is. I mean, I think our IT operations are still rather constrained on premises without a lot of because we have high performance computing without a lot of room there. So yes, it's innovation, but it's within the bounds. So if we can look to leverage more cloud and reduce that, perhaps bounding box as much, I think there's more opportunity for those types of conversations. All right. Is John still here? John? Right here. Can you give him the microphone? Because I want to get you in on this, too, with Osama. What facilitates that from your perspective? What facilitates that combination of the tactical and strategic, the operations and the IT, to drive that innovation? The reason I'm asking you is because you've made that point on the TV show earlier. But Osama, have you go first. What drives that both in quantity and quality? I appreciate you asking me to go first. I can't disagree. John. I agree with that. And probably not. I would say I think it's a couple of things. I think one, having a shared north star, we talked a lot about strategy up here today. I think that's critical. So being able to, you know, if you got pockets of folks are kind of moving out in their own directions for good reasons and they're trying to do the right thing. But if you can plant something that nobody's at yet that's further out and see, this is where we want to go, organization very quickly, everybody pointed in the same direction. They kind of have to converge. So I think that's a critical way. And then the other thing I would say is maybe, this idea and again, another can maybe shout out to the GSA crew, is thinking about things more as an ecosystem versus like an architecture. And, you know, enterprise architecture has evolved a lot over the past decade. And, but I still think kind of, we tend to think of that as more rigid and ecosystem is a little bit more fluid. And when you think of it like that, you can have a portion of that ecosystem responding to the customer needs or providing new experiences and changing things and it having an impact, a ripple effect on the rest of the ecosystem, but still being able to do that kind of based on what the closest trigger or the business need is for that. I think those two things together kind of help enable kind of that operations and this tactical strategy to come together. And I hope you have something besides I agree with him. I was going to say I agree with him. No, that that's a great starting. Yeah, that's a great start. I was always fine with Osama, if you agree with him, that's a good start here. There's a couple of things I'll say. First of all, Kathryn, I love what you said, which is like you have to be able to understand the power of these tools. I talk about the education element of this as understanding the the aperture, the art of the possible. And if you can open that aperture, you now have a way to drive innovation. I talked about everyone's responsibilities, innovation, but it's constrained when you don't understand the power of these tools. And so I think the education element of this and really creating what we've done internally is we've watched this large AI program, we call it AI Aware, because everyone in this institution that we're part of has an obligation to bring these solutions to our clients and understand the mission challenges and how AI is a component of that solution set. And when you make everyone aware of those capabilities, they can see the art of the possible. They can figure out how to plug in. So that's the first thing I'll say. The second thing I'll say is I do think there is an ecosystem at play here. You have to bring it. These problems are often, I think, too complex for one single solution. You have to bring together and integrate technology. That's kind of where I feel like we are really good at bringing together a lot of different technology players and stitching them together and engineering what I think of as at the seams of these technologies. These technologies don't work flawlessly. They need to be integrated together. You have to engineer at the seams and the last mile, whereas you're bringing that technology to that mission application, that last mile of AI to accomplish that. And so we have investment initiatives. We have a venture capital fund that we've stood up with $100 million where we're going out and we are investing in these technology companies to bring these capabilities to bridge gaps, because I think that's the last thing I would say is, is that like there's enormous technology out there, but you need to understand where the gaps between that technology exist and where the mission application fits. And ask yourself, do we need to engineer that solution or can we go partner by right, whatever that may be. And that's what we try to do is think about that in terms of that's how you get to innovation quickly and that's how I think you accelerate that time the mission impact with AI. Thanks for the insight. I appreciate you jumping in unanticipated. I really appreciate all of you being here. And I'm going to wrap up in just a second. Turn it back over to Katie to to wrap up. But I want to give each of you a chance to say what you didn't have a chance to say yet, a closing thought, whatever. Doesn't have to be long. But Bridget, I'll start with you. Anything you want to add just to throw in there before we wrap up? Yes, I think it is a very exciting time right now. And you had to unlike the other hats I wear, so, I am enterprise architecture and innovation. So I have a very unique role in AI and then I have spectrum management, I have all the type-- I don't, no, but they all are correlating and mixing today because with AI, we're able to see new tools. We're needing to be innovative, but then also protecting our spectrum pieces. So I think it's exciting for me. I never have like a dull day, but it's the matter of picking the right tools for Department of Energy and so that’s part of the pieces that we mainly need help with is narrowing down, yes, there's technologies, but what's best for our clients and how we can make those enterprise capabilities. Excellent, Trang? Well, I think I'm going to second what she says. Also wearing multiple hats. You think that's a theme federal government. Yes, that's. But you know, where is the artificial intelligence? I lead the artificial challenge, the Center of Innovation at CBP. So wearing that side of the hat, I think one of the things that it's an exciting time for sure. And, you know, I think more and more our mission partners are realizing that the technology such as AI, such as edge computing, are going to be a game changer for them. It's going to be able to help them better meet their mission needs. And so I think having everybody understand that and having everybody corral together to be able to assist in that realm world is really critical for what we do. And having been here during the same time when we have this AI-tsunami, come and attack us, is it's really fun but scary at the same time. In a good way. Yes. All right, cool. Kathryn? I am excited about this. I think data and putting data first and thinking data in terms of sharing is the only way we're going to really be able to solve some of the major problems that have moved forward and in our country at a global level. And I'm really interested in how data and the idea of social science come to pair because you can just give people data, but if they don't do something with it, that's the problem. So I think there is I'm really excited about using all of your data. I don't know how CBP is going to fit in, but we’ll figure it out. But bringing it together in a meaningful way to put forward solutions because that's what we need. That's why we have so many brilliant people working on this. And then if we can bring in the social scientists, bring in the people elements, that's how real change gets affected. But it starts with data. So everything I've heard here today, with everyone, I’ve had all the great conversations with. I’ll just leave here with a lot of hope. Excellent. That's wonderful. Osama, final thoughts? I mean, I think, first off, thank you to our government partners and all the work that you all are doing to help transform and take us into the future. Super appreciative of that. And thanks for you all in person today, Francis, for facilitating us. I think for me, I echoed the sentiment of excitement. I would say. I think as climate shifts and changes and our borders potentially shift and change for us specifically like the natural borders. And so that becomes there's your intersection right there. You guys can have coffee over afterwards. But I think, you know, to me just the themes about, you know, taking a strategic view and enterprise view, partnerships, I think are really going to be critical and in making sure that we're we're looking at the intersections of seams where innovation can occur. I think those are the things that I think are kind of good thematic takeaways from today. And yes, definitely exciting times going forward. Thank you all very much for participating. And I know you are very busy, so to take your time to come and do this morning, I appreciate. I want to thank again the Booz Allen team for having Fed Gov Today here. April 7th is when the TV segments will be on TV. Sunday morning at 10:30 on ABC7 in Washington and YouTube all around the world. And you should subscribe, by the way, to the Fed Gov Today YouTube channel. There's good stuff on there and I mean that not just in the self-serving way, I do kind of mean it that way, but not only in that way. And it's my pleasure to throw things back over to the Booz team. Katie is supposed to take things from here, and I thank you all for your attention. And I don't feel like I need to do much closing because you guys took all my data points, which you can read on the screen that they're not going to turn on. But thank you again, panelists. We heard some really great stories across GSA, VA, Energy, CBP, NOAH, today. And this is just the tip of the iceberg. We know that this is a really this is going to be the journey that we collectively are on for years to come and it's sort of exciting to start to see the progress that we're making, new design patterns, best practices, things that can be leveraged across. So I don't want to repeat what was just said because you all said it so well, but I'm tuning in on April 7th. I know I'm going to be watching and that's must see TV in my household. So thank you all for attending. There is lunch. You always need food. There is lunch and time to sort of meet and greet and network with one another. So please stay and enjoy some more time and camaraderie around these subjects. Thank you. Thanks everyone.