1 00:00:00,240 --> 00:00:04,150 Yeah . Yeah . Yes . 2 00:00:05,740 --> 00:00:07,962 Hey , good morning ladies and gentlemen 3 00:00:07,962 --> 00:00:09,962 welcome today's press conference on 4 00:00:09,962 --> 00:00:12,129 artificial intelligence . Someone sent 5 00:00:12,129 --> 00:00:14,296 a Commander , Arlo Abrahamson and I'll 6 00:00:14,296 --> 00:00:16,573 be moderating today's briefing with us . 7 00:00:16,573 --> 00:00:18,796 Today is the honourable robert , a work 8 00:00:18,796 --> 00:00:18,730 Vice chair of the National Security 9 00:00:18,730 --> 00:00:20,841 Commission on Artificial Intelligence 10 00:00:20,970 --> 00:00:23,026 and Lieutenant General Michael grown 11 00:00:23,026 --> 00:00:25,137 the director of the D . O . D . Joint 12 00:00:25,137 --> 00:00:27,303 Artificial Intelligence center . We'll 13 00:00:27,303 --> 00:00:29,414 begin this morning's briefing with an 14 00:00:29,414 --> 00:00:29,350 opening statement from both principles . 15 00:00:29,350 --> 00:00:31,550 Then we'll go to questions . I'll plan 16 00:00:31,550 --> 00:00:34,270 to go out to people out in the phones . 17 00:00:34,280 --> 00:00:36,224 We have a few people in the room . 18 00:00:36,224 --> 00:00:38,270 Please just identify your name and 19 00:00:38,270 --> 00:00:40,492 outlet to the principles before you ask 20 00:00:40,492 --> 00:00:42,659 your question . And with that I'll now 21 00:00:42,659 --> 00:00:44,659 turn it over to mr work and general 22 00:00:44,659 --> 00:00:46,381 grown to deliver their opening 23 00:00:46,381 --> 00:00:48,437 statements . Well thank you and good 24 00:00:48,437 --> 00:00:50,590 morning everybody those here and also 25 00:00:50,600 --> 00:00:52,890 who are following online . I'd like to 26 00:00:52,890 --> 00:00:55,660 start by just two over arching comments 27 00:00:56,440 --> 00:00:58,680 first for the first time since World 28 00:00:58,680 --> 00:01:01,660 War two , the United States Technical 29 00:01:01,660 --> 00:01:05,100 Predominance which undergirds both our 30 00:01:05,110 --> 00:01:06,560 economic and our military 31 00:01:06,560 --> 00:01:09,460 competitiveness is under severe threat 32 00:01:10,040 --> 00:01:12,050 by the people's Republic of china 33 00:01:14,140 --> 00:01:17,880 Nick Burns who is in his confirmation 34 00:01:17,880 --> 00:01:20,440 hearing or Bill Byrne's . Bill Burns . 35 00:01:20,440 --> 00:01:22,890 I'm sorry Bill Burns in his 36 00:01:22,890 --> 00:01:25,112 confirmation hearing is the director of 37 00:01:25,112 --> 00:01:27,279 C . I . A . Said that in the strategic 38 00:01:27,279 --> 00:01:29,390 competition with china technology 39 00:01:29,390 --> 00:01:32,340 competition is the central pillar and 40 00:01:32,340 --> 00:01:35,560 the AI commission agrees totally with 41 00:01:35,560 --> 00:01:38,620 that . The second broad thought is 42 00:01:38,630 --> 00:01:40,852 within this technological competition . 43 00:01:41,540 --> 00:01:44,260 The single most important technology 44 00:01:44,740 --> 00:01:48,130 that the United States must master is 45 00:01:48,140 --> 00:01:50,307 artificial intelligence and all of its 46 00:01:50,307 --> 00:01:53,780 associated technologies . Now we 47 00:01:53,780 --> 00:01:57,670 believe uh we view a I 48 00:01:57,670 --> 00:02:00,370 much like thomas Edison viewed 49 00:02:00,380 --> 00:02:03,840 electricity , he said it is a field of 50 00:02:03,840 --> 00:02:06,790 fields , it holds the secrets which 51 00:02:06,790 --> 00:02:09,660 will reorganize the life of the world . 52 00:02:10,640 --> 00:02:12,862 Now it sounds like a little hyperbole , 53 00:02:12,862 --> 00:02:15,800 but we actually believe that it is a 54 00:02:15,800 --> 00:02:18,190 new way of learning which will change 55 00:02:18,200 --> 00:02:21,350 everything . It will help us in uh , 56 00:02:21,360 --> 00:02:23,900 utilize quantum computing better . It 57 00:02:23,900 --> 00:02:26,011 will help us in health , it will help 58 00:02:26,011 --> 00:02:27,956 us in finance , it will help us in 59 00:02:27,960 --> 00:02:31,450 military competition . It is truly a 60 00:02:31,450 --> 00:02:33,420 field of feels . So if that is 61 00:02:33,420 --> 00:02:36,550 background , we said , look , we are 62 00:02:36,550 --> 00:02:39,250 not organized to win this competition . 63 00:02:39,940 --> 00:02:43,110 We just are not , we say we're in a 64 00:02:43,110 --> 00:02:45,221 competition , which is a good thing . 65 00:02:45,221 --> 00:02:47,443 The first thing you have to do is admit 66 00:02:47,443 --> 00:02:49,499 you have a problem . So Houston , we 67 00:02:49,499 --> 00:02:51,332 have a problem , but we have not 68 00:02:51,332 --> 00:02:53,054 organized ourselves to win the 69 00:02:53,054 --> 00:02:55,610 competition . We do not have a strategy 70 00:02:55,620 --> 00:02:57,980 to win the competition . We do not have 71 00:02:57,980 --> 00:03:01,490 the resources to implement a strategy , 72 00:03:01,500 --> 00:03:04,760 even if we had one . So the first thing 73 00:03:04,760 --> 00:03:07,210 is we have got to take this competition 74 00:03:07,210 --> 00:03:10,760 seriously and we need to win it . We 75 00:03:10,760 --> 00:03:13,560 need to enter it with the one single 76 00:03:13,940 --> 00:03:16,140 goal . We will win this technological 77 00:03:16,140 --> 00:03:19,650 competition . Um , now , 78 00:03:19,650 --> 00:03:21,950 what we decided the best way to think 79 00:03:21,950 --> 00:03:25,280 about this is We are not organized now . 80 00:03:25,290 --> 00:03:27,500 We need to get organized . We said by 81 00:03:27,500 --> 00:03:30,970 2025 , we should the department and the 82 00:03:30,970 --> 00:03:32,914 federal government should have the 83 00:03:32,914 --> 00:03:36,050 foundations in peace for widespread 84 00:03:37,140 --> 00:03:40,810 uh integration of AI across the federal 85 00:03:40,810 --> 00:03:43,070 government and particularly in D . O . 86 00:03:43,070 --> 00:03:45,760 D . Now there are three main building 87 00:03:45,760 --> 00:03:48,290 blocks to achieve this vision . First , 88 00:03:48,290 --> 00:03:50,457 you have to have top down leadership . 89 00:03:50,457 --> 00:03:53,070 You cannot say AI is important and then 90 00:03:53,080 --> 00:03:56,080 let all of the agencies and subordinate 91 00:03:56,090 --> 00:03:58,420 departments figure out what that means . 92 00:03:58,430 --> 00:04:00,550 You have to have someone from the top 93 00:04:00,550 --> 00:04:02,606 saying this is the vector . You will 94 00:04:02,606 --> 00:04:04,494 follow the vector . If you do not 95 00:04:04,494 --> 00:04:06,272 follow the vector , you will be 96 00:04:06,272 --> 00:04:08,606 penalized . If you do follow the vector , 97 00:04:08,606 --> 00:04:10,840 you will gain extra resources so you 98 00:04:10,840 --> 00:04:13,760 have to have top down leadership . Now , 99 00:04:14,440 --> 00:04:16,551 one of the first recommendations that 100 00:04:16,551 --> 00:04:19,660 we made is jake was underneath the C . 101 00:04:19,660 --> 00:04:22,040 I . O . And it was actually underneath 102 00:04:22,050 --> 00:04:25,440 disa in many ways administratively we 103 00:04:25,440 --> 00:04:27,480 said if you want to make a I your 104 00:04:27,480 --> 00:04:31,140 central technological thrust it needs 105 00:04:31,140 --> 00:04:33,160 to be elevated . And we recommended 106 00:04:33,160 --> 00:04:35,104 that the jake report either to the 107 00:04:35,104 --> 00:04:37,280 secretary or the deputy secretary that 108 00:04:37,280 --> 00:04:39,340 was actually included in the India . 109 00:04:39,340 --> 00:04:41,620 And now Jake reports to the Deputy 110 00:04:41,620 --> 00:04:43,750 secretary defense . And that's a very 111 00:04:43,750 --> 00:04:45,861 good first step . Well , we think the 112 00:04:45,861 --> 00:04:48,250 next step is to establish a steering 113 00:04:48,250 --> 00:04:50,700 committee on emerging technology . This 114 00:04:50,700 --> 00:04:53,480 would be a try chaired organization , 115 00:04:53,480 --> 00:04:55,258 the deputy secretary , the vice 116 00:04:55,258 --> 00:04:57,369 chairman of the Joint Chiefs of Staff 117 00:04:57,369 --> 00:04:59,424 and the principal Deputy Director of 118 00:04:59,424 --> 00:05:01,780 National Intelligence . They would sit 119 00:05:01,840 --> 00:05:03,729 and they would look at all of the 120 00:05:03,729 --> 00:05:05,729 technologies . They would drive the 121 00:05:05,729 --> 00:05:08,770 thrust towards an Ai future and they 122 00:05:08,770 --> 00:05:11,470 would coordinate all activities between 123 00:05:11,470 --> 00:05:13,637 the intelligence community and D . O . 124 00:05:13,637 --> 00:05:16,210 D . Which is a righteous thing . They 125 00:05:16,210 --> 00:05:18,377 would be the ones who identify lack of 126 00:05:18,377 --> 00:05:21,790 resources , address that problem . Uh 127 00:05:21,800 --> 00:05:24,350 and also remove any bureaucratic 128 00:05:24,350 --> 00:05:26,490 obstacles . The steering committee 129 00:05:26,490 --> 00:05:28,434 would oversee the development of a 130 00:05:28,434 --> 00:05:30,560 technology annex of the National 131 00:05:30,560 --> 00:05:33,710 Defense Strategy . The last time we had 132 00:05:33,720 --> 00:05:37,010 a list of technologies , There were 10 133 00:05:37,010 --> 00:05:40,260 on the list . All 10 of those were very , 134 00:05:40,260 --> 00:05:42,371 very important . But when you have 10 135 00:05:42,371 --> 00:05:44,593 things as your priorities , you have no 136 00:05:44,593 --> 00:05:47,830 priorities . You have to establish some 137 00:05:47,830 --> 00:05:50,160 type of prioritization and enforce it . 138 00:05:50,740 --> 00:05:52,962 So the technology annex to the National 139 00:05:52,962 --> 00:05:55,073 Defense Strategy would do just that . 140 00:05:55,840 --> 00:05:58,150 Also , the department should set a I 141 00:05:58,150 --> 00:06:00,320 readiness performance goals by the end 142 00:06:00,320 --> 00:06:04,090 of this fiscal year , 2021 With an eye 143 00:06:04,090 --> 00:06:07,210 towards 2025 when we need to be a I 144 00:06:07,210 --> 00:06:10,820 ready . So top down Leadership is the 145 00:06:10,820 --> 00:06:13,300 first big pillar . The second is to 146 00:06:13,300 --> 00:06:15,133 ensure that we have in place the 147 00:06:15,133 --> 00:06:16,830 resources , processes and the 148 00:06:16,830 --> 00:06:19,690 organizations to enable aI integration 149 00:06:19,700 --> 00:06:22,840 into the force . Now , Yeah , 150 00:06:23,640 --> 00:06:25,780 the commission said you need to 151 00:06:25,780 --> 00:06:28,160 establish a common digital ecosystem . 152 00:06:28,740 --> 00:06:30,750 The jake has established the joint 153 00:06:30,750 --> 00:06:33,070 Common Foundation . There are a lot of 154 00:06:33,070 --> 00:06:36,200 similarities between the two , although 155 00:06:36,210 --> 00:06:39,070 the commission's view is a little bit 156 00:06:39,080 --> 00:06:41,210 broader than the joint Common 157 00:06:41,210 --> 00:06:43,480 Foundation at the point , but the point 158 00:06:43,480 --> 00:06:47,430 is that everyone sees the necessity uh 159 00:06:47,440 --> 00:06:50,120 that provides access to all users in 160 00:06:50,120 --> 00:06:52,350 the department to software , train 161 00:06:52,350 --> 00:06:54,930 models , data computing and a 162 00:06:54,930 --> 00:06:57,070 developmental environment for def sec 163 00:06:57,070 --> 00:07:00,730 ops , that is secure . We 164 00:07:00,730 --> 00:07:02,952 recommended that you designate the jake 165 00:07:02,952 --> 00:07:05,630 is the Ai accelerator . We actually 166 00:07:05,640 --> 00:07:08,350 assess that china is a little bit ahead 167 00:07:08,350 --> 00:07:12,150 of the United States and fielding 168 00:07:12,160 --> 00:07:16,000 applications at scale . We can catch up 169 00:07:16,000 --> 00:07:18,480 with them and we believe that jake is 170 00:07:18,480 --> 00:07:21,270 the logical place in the department to 171 00:07:21,270 --> 00:07:23,200 really be the accelerator for AI 172 00:07:23,200 --> 00:07:26,990 applications at scale . The 173 00:07:26,990 --> 00:07:29,268 department has to increase its S and T . 174 00:07:29,268 --> 00:07:33,260 Spending on uh A . I . And all of R and 175 00:07:33,260 --> 00:07:35,427 D . We think it should be a minimum of 176 00:07:35,427 --> 00:07:38,570 3.4% of the budget . And we recommend 177 00:07:38,570 --> 00:07:40,514 that the department spend about $8 178 00:07:40,514 --> 00:07:42,950 billion on A . I . R . And D . Annually . 179 00:07:43,440 --> 00:07:45,607 That will allow us , we think to cover 180 00:07:45,607 --> 00:07:47,662 down on all the key research areas , 181 00:07:47,940 --> 00:07:49,773 there's all sorts of specialized 182 00:07:49,773 --> 00:07:51,829 acquisition pathways and contracting 183 00:07:51,829 --> 00:07:54,000 authorities out there . We still 184 00:07:54,000 --> 00:07:56,590 continually need to refine them because 185 00:07:56,600 --> 00:07:58,730 many of them are not perfectly 186 00:07:58,740 --> 00:08:01,750 applicable to software type things . 187 00:08:02,140 --> 00:08:05,230 And I know jake is working on this but 188 00:08:05,230 --> 00:08:07,397 we have to have an updated approach to 189 00:08:07,397 --> 00:08:09,452 the budget and oversight process for 190 00:08:09,452 --> 00:08:12,030 these things . So the second big pillar 191 00:08:12,050 --> 00:08:14,272 is ensure you have the resources in the 192 00:08:14,272 --> 00:08:17,420 processes and the organizations and 193 00:08:17,420 --> 00:08:19,587 third you have to accelerate and scale 194 00:08:19,587 --> 00:08:21,809 tech adoption . You really have to push 195 00:08:21,809 --> 00:08:24,110 this . So we recommend stand up and AI 196 00:08:24,110 --> 00:08:26,460 development team at every single cocom 197 00:08:26,840 --> 00:08:29,210 with foreign deployable elements and 198 00:08:29,210 --> 00:08:31,440 they leverage technological uh 199 00:08:31,450 --> 00:08:33,260 knowledge to develop innovative 200 00:08:33,260 --> 00:08:35,760 operational concepts and essentially 201 00:08:35,760 --> 00:08:38,840 establish a poll for AI enabled 202 00:08:38,850 --> 00:08:40,760 applications that will help them 203 00:08:41,140 --> 00:08:44,590 accomplish their missions . The 204 00:08:44,590 --> 00:08:46,701 department should prioritize adoption 205 00:08:46,701 --> 00:08:49,040 of commercial AI solutions , especially 206 00:08:49,040 --> 00:08:51,360 for all of the back office stuff . 207 00:08:51,740 --> 00:08:53,962 There's really no reason to do a lot of 208 00:08:53,962 --> 00:08:56,073 research on those type applications . 209 00:08:56,240 --> 00:08:58,351 The commercial industry has plenty of 210 00:08:58,351 --> 00:09:00,300 them . You just have to prioritize 211 00:09:00,350 --> 00:09:02,183 identifying the ones that can be 212 00:09:02,183 --> 00:09:04,380 modified for our use and bring them in 213 00:09:04,380 --> 00:09:06,820 as quickly as possible . We think the 214 00:09:06,820 --> 00:09:09,042 department should establish a dedicated 215 00:09:09,042 --> 00:09:11,209 AI fund under the control of the under 216 00:09:11,209 --> 00:09:13,320 Secretary of Defense for Research and 217 00:09:13,320 --> 00:09:15,610 Engineering . And that fund would allow 218 00:09:15,620 --> 00:09:19,580 undersecretary to get small innovative 219 00:09:19,590 --> 00:09:21,960 AI companies across the Valley of Death . 220 00:09:22,340 --> 00:09:24,660 And this would be up to the Under 221 00:09:24,660 --> 00:09:27,400 secretary Defense for Arnie , who is 222 00:09:27,400 --> 00:09:29,580 the chief technology officer of the 223 00:09:29,580 --> 00:09:33,220 department . Now the things that under 224 00:09:33,220 --> 00:09:35,250 cross all of these are talent , 225 00:09:36,640 --> 00:09:39,660 ethics and international partnerships . 226 00:09:39,810 --> 00:09:43,080 We talk about talent first , we think 227 00:09:43,090 --> 00:09:45,312 we have to have a . D . O . D . Digital 228 00:09:45,312 --> 00:09:47,479 core modeled after the Medical Corps . 229 00:09:47,940 --> 00:09:50,350 These are digitally savvy warriors , 230 00:09:50,590 --> 00:09:54,290 administrators and leaders . We just 231 00:09:54,290 --> 00:09:56,457 need to know who they are . We need to 232 00:09:56,457 --> 00:09:58,512 code them in some way and we need to 233 00:09:58,512 --> 00:10:00,568 make sure they're in the places that 234 00:10:00,568 --> 00:10:02,790 have the highest return on investment . 235 00:10:03,140 --> 00:10:04,751 We need to train and educate 236 00:10:04,751 --> 00:10:06,307 warfighters to develop core 237 00:10:06,307 --> 00:10:10,050 competencies and using and responsibly 238 00:10:10,540 --> 00:10:12,790 teeming with machine systems , 239 00:10:12,800 --> 00:10:14,760 understanding their limitations , 240 00:10:15,140 --> 00:10:17,251 understanding what they should not be 241 00:10:17,251 --> 00:10:20,220 asked to do , et cetera And equally . 242 00:10:20,220 --> 00:10:22,442 And other emerging technologies need to 243 00:10:22,442 --> 00:10:24,664 feature prominently . And senior leader 244 00:10:24,664 --> 00:10:27,440 education and training with a key focus 245 00:10:27,440 --> 00:10:30,260 on ethics . The ethical use of AI . 246 00:10:31,740 --> 00:10:34,210 Now go right into that . We're in a 247 00:10:34,210 --> 00:10:36,960 competition with authoritarian regimes . 248 00:10:36,970 --> 00:10:39,550 Authoritarian regimes will use 249 00:10:39,550 --> 00:10:42,430 technology that reflect their own 250 00:10:42,440 --> 00:10:45,150 governing principles . We already know 251 00:10:45,150 --> 00:10:47,450 how china wants to use Ai . They want 252 00:10:47,450 --> 00:10:49,672 to use it for population surveillance . 253 00:10:49,710 --> 00:10:51,488 They want to use it to suppress 254 00:10:51,488 --> 00:10:54,710 minorities . They want to use it to cut 255 00:10:54,720 --> 00:10:57,690 individual privacy and trample on civil 256 00:10:57,690 --> 00:11:00,000 liberties . That's not going to work 257 00:11:00,000 --> 00:11:02,222 for a democratic nation like the United 258 00:11:02,222 --> 00:11:05,210 States . And so this is as much a 259 00:11:05,210 --> 00:11:07,550 values competition as it is a 260 00:11:07,550 --> 00:11:09,880 technological competition . The way 261 00:11:09,880 --> 00:11:12,102 eric Schmidt , our chairman talks about 262 00:11:12,102 --> 00:11:15,640 this Is we're going to employ platforms 263 00:11:15,660 --> 00:11:18,360 which bring these technologies . So 264 00:11:18,360 --> 00:11:20,490 let's just think about how 5G worked 265 00:11:20,820 --> 00:11:24,260 Huawei's 5G technologies 266 00:11:24,640 --> 00:11:28,640 allowed a country who do this 267 00:11:28,650 --> 00:11:32,370 uses it to essentially surveil their 268 00:11:32,370 --> 00:11:36,050 population . So these values are 269 00:11:36,050 --> 00:11:38,110 very very critical and an important 270 00:11:38,110 --> 00:11:41,660 part of the competition . And finally 271 00:11:41,670 --> 00:11:43,837 we're not going to succeed if we do it 272 00:11:43,837 --> 00:11:45,726 alone . This is a kind of central 273 00:11:45,726 --> 00:11:48,280 thinking and U . S . Defense um 274 00:11:48,290 --> 00:11:51,300 strategies . So we have to promote a i 275 00:11:51,300 --> 00:11:53,300 interoperability in the adoption of 276 00:11:53,300 --> 00:11:55,760 emerging technologies across among our 277 00:11:55,760 --> 00:11:58,910 allies and our partners . We are 278 00:11:58,920 --> 00:12:01,087 absolutely confident as a commission , 279 00:12:01,087 --> 00:12:03,650 we can win this competition but we will 280 00:12:03,650 --> 00:12:06,400 not win it if we do not organize 281 00:12:06,400 --> 00:12:09,310 ourselves and have a strategy and have 282 00:12:09,310 --> 00:12:12,000 resources for the strategy and a means 283 00:12:12,000 --> 00:12:15,300 by which to implement the strategy and 284 00:12:15,300 --> 00:12:18,900 make sure that everyone is doing their 285 00:12:18,900 --> 00:12:22,060 part . Thank you . Mhm . Good . 286 00:12:25,340 --> 00:12:27,507 All right . Good morning everybody and 287 00:12:27,507 --> 00:12:29,618 thank you very much for participating 288 00:12:29,618 --> 00:12:31,840 in this important session . And first I 289 00:12:31,840 --> 00:12:34,062 want to say thank you to Secretary work 290 00:12:34,062 --> 00:12:36,284 and the National Security Commission AI 291 00:12:36,284 --> 00:12:38,396 on AI team . Just incredible work . I 292 00:12:38,396 --> 00:12:40,451 mean , what you see if you if you've 293 00:12:40,451 --> 00:12:42,507 read the report , if you haven't , I 294 00:12:42,507 --> 00:12:44,673 encourage you to go to the website and 295 00:12:44,673 --> 00:12:46,729 look at the NSC AI final report . Um 296 00:12:46,729 --> 00:12:48,284 What you see is like a deep 297 00:12:48,284 --> 00:12:50,340 understanding and a deep analysis of 298 00:12:50,340 --> 00:12:52,562 down to first principles bare metal for 299 00:12:52,562 --> 00:12:55,360 what it takes for a i integration and 300 00:12:55,360 --> 00:12:57,582 preserving our military effectiveness . 301 00:12:57,582 --> 00:12:59,582 Um What they produced is critically 302 00:12:59,582 --> 00:13:01,749 important and critically important for 303 00:13:01,749 --> 00:13:03,804 us in the department , but it's also 304 00:13:03,804 --> 00:13:03,740 critically important for our national 305 00:13:03,740 --> 00:13:06,070 competitiveness . In the same breath , 306 00:13:06,080 --> 00:13:08,191 I'd like to say thank you to Congress 307 00:13:08,191 --> 00:13:10,413 and uh and department leadership , both 308 00:13:10,413 --> 00:13:12,191 of which clearly understand the 309 00:13:12,191 --> 00:13:14,530 importance and the need to innovate and 310 00:13:14,530 --> 00:13:16,700 modernize the way we fight and the way 311 00:13:16,700 --> 00:13:18,690 we do business . And I'm happy to 312 00:13:18,690 --> 00:13:20,700 report as the director of the jake 313 00:13:20,710 --> 00:13:22,932 positive momentum toward implementation 314 00:13:22,932 --> 00:13:25,430 of an AI uh implementation of AI at 315 00:13:25,430 --> 00:13:27,541 scale . We certainly have a long ways 316 00:13:27,541 --> 00:13:29,486 to go . But you can see the needle 317 00:13:29,486 --> 00:13:31,870 trending positive uh with bipartisan 318 00:13:31,870 --> 00:13:33,814 support from Congress , with great 319 00:13:33,814 --> 00:13:36,148 support from the D . O . D . Leadership , 320 00:13:36,148 --> 00:13:38,259 the services are beginning to develop 321 00:13:38,259 --> 00:13:40,370 aI initiatives and expand operational 322 00:13:40,370 --> 00:13:42,426 experimentation that is taking those 323 00:13:42,426 --> 00:13:44,592 first steps . The defense agencies are 324 00:13:44,592 --> 00:13:46,759 reaching out daily to share their best 325 00:13:46,759 --> 00:13:48,981 practices with us and with each other . 326 00:13:48,981 --> 00:13:51,203 The combatant commands , especially the 327 00:13:51,203 --> 00:13:53,426 combatant commanders , have have caught 328 00:13:53,426 --> 00:13:55,648 a glimpse of what the future might look 329 00:13:55,648 --> 00:13:57,703 like through a series of integrative 330 00:13:57,703 --> 00:13:59,814 exercises . They like it and they are 331 00:13:59,814 --> 00:13:59,360 they're eager to gain these 332 00:13:59,360 --> 00:14:01,940 capabilities with the jake now aligned 333 00:14:01,940 --> 00:14:03,900 under the deputy secretary , which 334 00:14:03,900 --> 00:14:05,567 gives her and the rest of the 335 00:14:05,567 --> 00:14:07,567 department leadership access to the 336 00:14:07,567 --> 00:14:09,733 tools and processes to reinforce their 337 00:14:09,733 --> 00:14:11,678 priorities . Underline our ethical 338 00:14:11,678 --> 00:14:13,900 foundations , integrate our enterprises 339 00:14:13,900 --> 00:14:16,067 and transform our business processes . 340 00:14:16,067 --> 00:14:18,233 And we were eagerly looking forward to 341 00:14:18,233 --> 00:14:21,880 that work . Um like the CIA I we see 342 00:14:21,880 --> 00:14:24,250 a as a core tenet of defense 343 00:14:24,250 --> 00:14:26,990 modernization and when I say I want to 344 00:14:26,990 --> 00:14:29,101 be clear , I'm not just talking about 345 00:14:29,101 --> 00:14:31,740 the Jake all ai the efforts of the 346 00:14:31,740 --> 00:14:33,407 services , the efforts of the 347 00:14:33,407 --> 00:14:35,880 department's uh and the agency's rides 348 00:14:35,880 --> 00:14:37,991 on the foundations of good networks , 349 00:14:37,991 --> 00:14:40,920 good data services , good security and 350 00:14:40,920 --> 00:14:43,300 good partnerships . An important part 351 00:14:43,300 --> 00:14:45,580 of the jake's business model is to 352 00:14:45,590 --> 00:14:47,290 build those as part of our AI 353 00:14:47,290 --> 00:14:49,920 infrastructure and with lots of budget 354 00:14:49,920 --> 00:14:51,976 work ahead I think , you know , well 355 00:14:51,976 --> 00:14:54,530 here , you know , as Fy 22 is relooked 356 00:14:54,530 --> 00:14:57,550 and and the Palm 23 to 27 is developed . 357 00:14:57,560 --> 00:14:59,449 You know , we'll hear a lot about 358 00:14:59,449 --> 00:15:02,110 modern weapons systems and concepts and 359 00:15:02,120 --> 00:15:03,953 it's important that we should we 360 00:15:03,953 --> 00:15:06,250 understand that their potential those 361 00:15:06,250 --> 00:15:08,194 weapons systems , those concepts , 362 00:15:08,194 --> 00:15:10,306 their potential to modernize our ward 363 00:15:10,306 --> 00:15:12,610 fighting rides on those trends , on the 364 00:15:12,610 --> 00:15:15,140 foundational data , the networks , the 365 00:15:15,150 --> 00:15:17,261 algorithms that we build to integrate 366 00:15:17,261 --> 00:15:19,610 and inform them . Uh We'll have to talk 367 00:15:19,610 --> 00:15:22,290 about these technical foundation and 368 00:15:22,290 --> 00:15:24,640 architecture architecture in the same 369 00:15:24,640 --> 00:15:26,418 conversation that we talk about 370 00:15:26,418 --> 00:15:29,210 platforms getting A I right and are 371 00:15:29,210 --> 00:15:31,377 secure data fabric environment , right 372 00:15:31,377 --> 00:15:33,266 will be central to our ability to 373 00:15:33,266 --> 00:15:35,321 compete effectively with the chinese 374 00:15:35,321 --> 00:15:37,488 and the Russians as well or any modern 375 00:15:37,488 --> 00:15:39,730 threat for that matter . Um And and 376 00:15:39,730 --> 00:15:41,900 there's more actually so in an era of 377 00:15:41,900 --> 00:15:45,640 tightening budgets and and a focus on 378 00:15:45,650 --> 00:15:48,950 on on squeezing out things that our 379 00:15:48,960 --> 00:15:51,016 that our legacy are not important in 380 00:15:51,016 --> 00:15:54,010 the budget . Um uh the productivity 381 00:15:54,010 --> 00:15:56,330 gains and the efficiency gains that ai 382 00:15:56,330 --> 00:15:57,997 can bring to the department , 383 00:15:57,997 --> 00:16:00,330 especially through the business process , 384 00:16:00,330 --> 00:16:02,720 transformation actually becomes an 385 00:16:02,730 --> 00:16:05,440 economic necessity . So in a squeeze 386 00:16:05,440 --> 00:16:08,570 play between modernizing our warfare 387 00:16:08,580 --> 00:16:10,990 that moves at machine speed and tighter 388 00:16:10,990 --> 00:16:14,520 budgets , AI is doubly doubly necessary . 389 00:16:14,530 --> 00:16:16,474 Uh So what I'm , what am I talking 390 00:16:16,474 --> 00:16:18,197 about when I talk about a I as 391 00:16:18,197 --> 00:16:20,430 secretary works comments uh convey the 392 00:16:20,430 --> 00:16:22,652 integration of AI across the across the 393 00:16:22,652 --> 00:16:24,486 government and the Department of 394 00:16:24,486 --> 00:16:27,720 Defense is much more than just a just a 395 00:16:27,730 --> 00:16:30,590 facile layer of technology applied . 396 00:16:30,600 --> 00:16:33,470 It's not about shiny objects . You've 397 00:16:33,470 --> 00:16:36,510 heard the phrase amateur study tactics 398 00:16:37,240 --> 00:16:39,740 and professional study logistics . Well , 399 00:16:39,740 --> 00:16:41,840 in this environment , amateurs talk 400 00:16:41,850 --> 00:16:44,060 about applications and professionals 401 00:16:44,060 --> 00:16:46,140 talk about architecture and networks 402 00:16:46,150 --> 00:16:48,330 and elevating the Ai dialogue in the 403 00:16:48,330 --> 00:16:50,552 department so that we are talking about 404 00:16:50,552 --> 00:16:52,608 the foundations of all of our modern 405 00:16:52,608 --> 00:16:54,941 capabilities is a really important task . 406 00:16:54,941 --> 00:16:57,108 One that we're that we're working hard 407 00:16:57,108 --> 00:16:59,640 on the core business model . That is 408 00:16:59,640 --> 00:17:01,830 what the department uh gives to the 409 00:17:01,830 --> 00:17:04,052 american people . What our mission is . 410 00:17:04,052 --> 00:17:06,150 Doesn't change . But a modernized , 411 00:17:06,160 --> 00:17:09,200 data driven software heavy organization 412 00:17:09,210 --> 00:17:12,890 will do things in a different way . It 413 00:17:12,890 --> 00:17:15,001 really represents a transformation of 414 00:17:15,001 --> 00:17:17,460 our operating model . How do we do the 415 00:17:17,460 --> 00:17:19,516 things that we do as a Department of 416 00:17:19,516 --> 00:17:21,738 Defense ? And that operating model will 417 00:17:21,738 --> 00:17:23,349 have to create a common data 418 00:17:23,349 --> 00:17:25,293 environment where data is shared , 419 00:17:25,293 --> 00:17:27,571 data's authoritative data is available . 420 00:17:27,571 --> 00:17:29,627 The data feeds and algorithms across 421 00:17:29,627 --> 00:17:31,849 the department will create productivity 422 00:17:31,849 --> 00:17:34,480 gains , accelerate processes , uh 423 00:17:34,490 --> 00:17:36,657 provide management visibility insights 424 00:17:36,657 --> 00:17:38,840 into markets . And if all of that 425 00:17:38,840 --> 00:17:41,500 sounds like a modern software driven 426 00:17:41,500 --> 00:17:43,910 company , uh you know , you can think 427 00:17:43,920 --> 00:17:46,390 think of all of our tech giants uh and 428 00:17:46,390 --> 00:17:48,860 and uh and smaller innovative companies 429 00:17:48,860 --> 00:17:51,570 across the U . S . Economy . Um It's 430 00:17:51,570 --> 00:17:53,792 because it is it's the same challenge , 431 00:17:53,792 --> 00:17:55,959 It's the same problem . And so we have 432 00:17:55,959 --> 00:17:58,014 examples right ? There's very little 433 00:17:58,014 --> 00:18:00,370 magic here . It's about making our 434 00:18:00,370 --> 00:18:02,537 organization the Department of Defense 435 00:18:02,537 --> 00:18:04,259 in this case as productive and 436 00:18:04,259 --> 00:18:06,092 efficient as any of these modern 437 00:18:06,092 --> 00:18:08,148 successful data driven enterprises . 438 00:18:08,240 --> 00:18:11,140 But there's so much more because all of 439 00:18:11,140 --> 00:18:13,307 this technology applies equally to our 440 00:18:13,307 --> 00:18:15,029 warfighting capabilities , our 441 00:18:15,029 --> 00:18:16,862 capabilities in a broad range of 442 00:18:16,862 --> 00:18:18,807 supporting activities from all the 443 00:18:18,807 --> 00:18:20,973 defense agencies and other places that 444 00:18:20,973 --> 00:18:23,251 make up the business of the department . 445 00:18:23,251 --> 00:18:25,362 We've created a positive momentum for 446 00:18:25,362 --> 00:18:27,696 AI and we continue to build on that now . 447 00:18:27,696 --> 00:18:29,862 But now comes the real critical test . 448 00:18:29,862 --> 00:18:31,890 Um As in any transformation . The 449 00:18:31,900 --> 00:18:34,100 hardest part is institutional change 450 00:18:34,110 --> 00:18:36,430 and change management of the workforce 451 00:18:36,430 --> 00:18:38,870 and practices and processes that drive 452 00:18:38,870 --> 00:18:41,037 that drive a business . This step will 453 00:18:41,037 --> 00:18:43,203 not be easy even within the Department 454 00:18:43,203 --> 00:18:45,259 of Defense , but its foundational to 455 00:18:45,259 --> 00:18:47,110 our competitive success , our 456 00:18:47,110 --> 00:18:49,277 accountability and our affordability . 457 00:18:49,740 --> 00:18:51,796 Yeah , as the N . S . C . A . I work 458 00:18:51,796 --> 00:18:53,407 reveals , I mean , we have a 459 00:18:53,407 --> 00:18:55,518 generational opportunity here for a I 460 00:18:55,518 --> 00:18:58,310 to be our future . We must act now . We 461 00:18:58,310 --> 00:19:00,532 need to start putting these places into 462 00:19:00,532 --> 00:19:02,930 place now . Um So I want to quickly 463 00:19:02,930 --> 00:19:04,819 describe our position through two 464 00:19:04,819 --> 00:19:06,930 different lenses . One is competition 465 00:19:06,930 --> 00:19:09,097 and the others opportunity uh first of 466 00:19:09,097 --> 00:19:10,819 all , with with respect to A i 467 00:19:10,819 --> 00:19:12,763 competition . Uh I think it's it's 468 00:19:12,763 --> 00:19:14,986 illustrative to talk about the economic 469 00:19:14,986 --> 00:19:17,152 impact of artificial intelligence as a 470 00:19:17,152 --> 00:19:19,152 first order . Um economic forecasts 471 00:19:19,152 --> 00:19:22,900 predict an Ai economy of 16 472 00:19:22,900 --> 00:19:26,660 trillion a $16 trillion ai economy 473 00:19:26,740 --> 00:19:29,510 In the next 10 years . And and this 474 00:19:29,510 --> 00:19:32,260 will this will this could amount to 475 00:19:32,270 --> 00:19:36,240 massive GDP increases 26 as high 476 00:19:36,240 --> 00:19:39,210 as 26 for China , as high as 15 for the 477 00:19:39,210 --> 00:19:43,100 United States . That to be participate 478 00:19:43,110 --> 00:19:45,450 in this competitive ai marketplace . 479 00:19:45,840 --> 00:19:49,300 And if we do that , uh this is core , 480 00:19:49,300 --> 00:19:51,411 this core economic competitiveness of 481 00:19:51,411 --> 00:19:53,356 the United States then needs to be 482 00:19:53,356 --> 00:19:55,080 reflected in a core military 483 00:19:55,090 --> 00:19:57,201 competitiveness in this in this space 484 00:19:57,201 --> 00:19:59,423 as well . It's important to note that , 485 00:19:59,423 --> 00:20:01,479 you know , while we talk about a $16 486 00:20:01,479 --> 00:20:03,423 trillion dollar market in the next 487 00:20:03,423 --> 00:20:05,570 decade , um this happens to coincide 488 00:20:05,570 --> 00:20:07,750 pretty closely with china's declared 489 00:20:07,750 --> 00:20:10,410 and often repeated intent To be 490 00:20:10,420 --> 00:20:14,410 globally dominant in Ai by 2030 . So we 491 00:20:14,410 --> 00:20:16,330 look at the transformation of our 492 00:20:16,330 --> 00:20:19,390 economy has to be accompanied with a 493 00:20:19,400 --> 00:20:21,622 close attention to the emerging threats 494 00:20:21,622 --> 00:20:23,456 that are taken that are that are 495 00:20:23,456 --> 00:20:25,567 declaring their intention to use this 496 00:20:25,567 --> 00:20:27,830 as a point of competition between 497 00:20:27,840 --> 00:20:30,370 autocracies and democracies . Our 498 00:20:30,370 --> 00:20:32,560 forces must operate with tempo with 499 00:20:32,560 --> 00:20:34,393 data driven decisions with human 500 00:20:34,393 --> 00:20:36,900 machine teaming , our forces must have 501 00:20:36,900 --> 00:20:38,900 broad situational awareness , multi 502 00:20:38,900 --> 00:20:42,810 domain integration . Uh The PRC has a 503 00:20:42,810 --> 00:20:45,230 robust entrepreneurial ai environment . 504 00:20:45,240 --> 00:20:47,407 We're all familiar with , you know and 505 00:20:47,407 --> 00:20:49,629 financial or alibaba . Tencent . I mean 506 00:20:49,629 --> 00:20:51,629 these are global companies . Um But 507 00:20:51,629 --> 00:20:53,518 we're also very familiar with the 508 00:20:53,518 --> 00:20:56,290 artifacts of population surveillance , 509 00:20:56,300 --> 00:20:58,750 minority oppression . The things that 510 00:20:58,750 --> 00:21:00,861 the secretary work talked about under 511 00:21:00,861 --> 00:21:03,660 the chinese communist Party's rule . We 512 00:21:03,660 --> 00:21:05,660 we read about Beijing's large scale 513 00:21:05,660 --> 00:21:07,604 campuses , their tech campuses and 514 00:21:07,604 --> 00:21:09,827 they're they're state owned enterprises 515 00:21:09,827 --> 00:21:11,382 that create a pipeline from 516 00:21:11,382 --> 00:21:13,760 entrepreneurs and innovators in china 517 00:21:13,770 --> 00:21:16,550 to through the civil military fusion . 518 00:21:16,940 --> 00:21:19,230 Take those capabilities directly into 519 00:21:19,230 --> 00:21:20,841 the P . L . A . And military 520 00:21:20,841 --> 00:21:23,140 capabilities without intervening 521 00:21:23,150 --> 00:21:25,850 accountability or transparency . Um 522 00:21:26,540 --> 00:21:29,340 They're organizational efficiency that 523 00:21:29,350 --> 00:21:31,690 that autocratic rule . They count that 524 00:21:31,690 --> 00:21:34,430 as an advantage Is being applied 525 00:21:34,430 --> 00:21:36,710 directly to their Ai development and 526 00:21:36,710 --> 00:21:38,766 they are surging forward in their in 527 00:21:38,766 --> 00:21:40,988 their in their capability . This has to 528 00:21:40,988 --> 00:21:43,490 give us pause to contemplate what does , 529 00:21:43,500 --> 00:21:46,420 what does China's dominance in a I mean 530 00:21:46,420 --> 00:21:48,950 for us , if they intend that dominance 531 00:21:48,960 --> 00:21:51,850 by 2030 , what does that imply for us ? 532 00:21:52,440 --> 00:21:54,551 But we also can look through the lens 533 00:21:54,551 --> 00:21:56,773 of opportunity . Our best opportunities 534 00:21:56,773 --> 00:21:58,970 lie in american innovation , academia 535 00:21:58,970 --> 00:22:01,081 and small companies are brimming with 536 00:22:01,081 --> 00:22:03,490 good ideas . In the ai space , the 537 00:22:03,500 --> 00:22:06,090 number of Ai companies is proliferating 538 00:22:06,090 --> 00:22:08,950 rapidly rapidly . Um We have war 539 00:22:08,950 --> 00:22:10,783 fighters across the department , 540 00:22:10,840 --> 00:22:12,562 especially young ones that can 541 00:22:12,562 --> 00:22:14,580 visualize their use cases in their 542 00:22:14,580 --> 00:22:16,636 operating environment and the things 543 00:22:16,636 --> 00:22:18,691 that they need to do from a military 544 00:22:18,691 --> 00:22:20,802 capability perspective , they're good 545 00:22:20,802 --> 00:22:23,024 at this . They know how to operate in a 546 00:22:23,024 --> 00:22:25,080 data driven an app based environment 547 00:22:25,080 --> 00:22:27,247 because they grew up that way and they 548 00:22:27,247 --> 00:22:29,191 expect the same from their defense 549 00:22:29,191 --> 00:22:31,358 systems . We have the best science and 550 00:22:31,358 --> 00:22:33,880 the best Ai research available uh in 551 00:22:33,880 --> 00:22:36,102 academia , inside the United States and 552 00:22:36,102 --> 00:22:37,936 in small companies . And we also 553 00:22:37,936 --> 00:22:39,991 benefit from the fact that we have a 554 00:22:39,991 --> 00:22:42,047 tech inversion in place where the Ai 555 00:22:42,047 --> 00:22:43,991 technology that we need to run our 556 00:22:43,991 --> 00:22:45,991 department and change our operating 557 00:22:45,991 --> 00:22:47,936 model exists right literally right 558 00:22:47,936 --> 00:22:49,936 across the street . And many of the 559 00:22:49,936 --> 00:22:52,560 companies , the modern ai driven , data 560 00:22:52,560 --> 00:22:54,727 driven companies that have survived in 561 00:22:54,727 --> 00:22:56,727 a very competitive market . We have 562 00:22:56,727 --> 00:22:59,040 lots of good examples to look at . We 563 00:22:59,040 --> 00:23:01,470 also have a rock solid ethical baseline 564 00:23:01,500 --> 00:23:03,667 that drives a principled approach that 565 00:23:03,667 --> 00:23:05,722 drives our test and evaluation , our 566 00:23:05,722 --> 00:23:08,070 verification or validation , our policy , 567 00:23:08,080 --> 00:23:10,302 and in the end of the in the end of the 568 00:23:10,302 --> 00:23:12,910 of the analysis , our trust in our AI 569 00:23:12,910 --> 00:23:14,966 systems and I welcome your questions 570 00:23:14,966 --> 00:23:17,021 about that . The good news . We have 571 00:23:17,021 --> 00:23:18,854 1000 flowers blooming inside the 572 00:23:18,854 --> 00:23:20,910 department through the initiative of 573 00:23:20,910 --> 00:23:22,910 the services , the agencies and the 574 00:23:22,910 --> 00:23:24,854 activities of the department . And 575 00:23:24,854 --> 00:23:26,854 we're doing better to integrate our 576 00:23:26,854 --> 00:23:28,743 industry technical expertise with 577 00:23:28,743 --> 00:23:30,743 warfighting functional expertise so 578 00:23:30,743 --> 00:23:32,799 that we can actually responsibly and 579 00:23:32,799 --> 00:23:35,170 responsibly build implement technology 580 00:23:35,170 --> 00:23:37,500 in the places that matter most . We 581 00:23:37,500 --> 00:23:39,167 have the opportunity to drive 582 00:23:39,167 --> 00:23:40,722 productivity , efficiency , 583 00:23:40,722 --> 00:23:43,000 effectiveness uh , of the department to 584 00:23:43,000 --> 00:23:45,111 new heights . And uh , the performers 585 00:23:45,111 --> 00:23:47,167 across the department in the jake in 586 00:23:47,167 --> 00:23:49,333 the services and other places are very 587 00:23:49,333 --> 00:23:51,444 excited and count themselves lucky to 588 00:23:51,444 --> 00:23:54,000 be part of this work . And uh with that 589 00:23:54,000 --> 00:23:55,889 we very much look forward to your 590 00:23:55,889 --> 00:23:58,222 questions and appreciate your attention . 591 00:23:58,240 --> 00:24:00,296 All right , everybody , as we've got 592 00:24:00,296 --> 00:24:02,462 about 16 or 17 reporters on the line . 593 00:24:02,462 --> 00:24:04,573 So if we could dash just one question 594 00:24:04,573 --> 00:24:06,796 at a time and then I promise I will get 595 00:24:06,796 --> 00:24:08,907 to you for a second if we have time . 596 00:24:08,907 --> 00:24:10,962 So the first question is going to go 597 00:24:10,962 --> 00:24:12,629 out to Mr Aaron Greg from the 598 00:24:12,629 --> 00:24:14,851 Washington Post . Aaron , I know you're 599 00:24:14,851 --> 00:24:14,490 on the line . I believe you're on the 600 00:24:14,490 --> 00:24:17,320 line . Go ahead . Thank you guys for 601 00:24:17,320 --> 00:24:20,030 doing this . Um how does the enterprise 602 00:24:20,030 --> 00:24:22,330 cloud strategy play into all of this ? 603 00:24:22,340 --> 00:24:25,020 Um Is this hodgepodge that you're 604 00:24:25,020 --> 00:24:27,076 currently working with , working for 605 00:24:27,076 --> 00:24:29,020 the department ? And what does the 606 00:24:29,020 --> 00:24:30,909 strategy look like under this new 607 00:24:30,909 --> 00:24:34,060 administration and the new sect of ? So , 608 00:24:34,060 --> 00:24:37,460 so I'll take that one first . Um So so 609 00:24:37,460 --> 00:24:39,627 what we have today , you're right , we 610 00:24:39,627 --> 00:24:41,516 have development environments and 611 00:24:41,516 --> 00:24:43,682 pretty mature development environments 612 00:24:43,682 --> 00:24:45,793 in each of the services , some of the 613 00:24:45,793 --> 00:24:47,738 services have multiple development 614 00:24:47,738 --> 00:24:49,960 environments . And so one of the things 615 00:24:49,960 --> 00:24:52,430 that we that we have to look at is um 616 00:24:52,440 --> 00:24:55,550 you know , what degree of resilience do 617 00:24:55,550 --> 00:24:57,383 we gain from having multiple deV 618 00:24:57,383 --> 00:25:00,540 environments , but also what advantages 619 00:25:00,540 --> 00:25:02,484 do we gain by , by stitching those 620 00:25:02,484 --> 00:25:04,540 development environment environments 621 00:25:04,540 --> 00:25:06,762 together into a into a fabric . So that 622 00:25:06,762 --> 00:25:08,818 is our intent and that is what we're 623 00:25:08,818 --> 00:25:11,040 looking for now , mapping that out . So 624 00:25:11,040 --> 00:25:13,096 what you know , what we need is is a 625 00:25:13,096 --> 00:25:15,096 network of development environments 626 00:25:15,096 --> 00:25:17,030 that shares through containerized 627 00:25:17,030 --> 00:25:19,500 process shares , you know , authority 628 00:25:19,500 --> 00:25:21,540 to operate on networks that shares 629 00:25:21,540 --> 00:25:24,030 access to data sources that shares 630 00:25:24,030 --> 00:25:26,180 algorithms and that shares even 631 00:25:26,180 --> 00:25:28,291 developmental tools and developmental 632 00:25:28,291 --> 00:25:30,236 environments . And so this is what 633 00:25:30,236 --> 00:25:32,458 we're trying to construct today so that 634 00:25:32,458 --> 00:25:33,958 we can broaden the base of 635 00:25:33,958 --> 00:25:36,180 developmental work . But on top of that 636 00:25:36,180 --> 00:25:37,902 we need an operating layer and 637 00:25:37,902 --> 00:25:40,124 operating network . And this is kind of 638 00:25:40,124 --> 00:25:42,347 the next step because if you take those 639 00:25:42,347 --> 00:25:44,347 developmental algorithms and you're 640 00:25:44,347 --> 00:25:46,513 going to employ them in a steady state 641 00:25:46,513 --> 00:25:48,680 basis , in a combatant command and war 642 00:25:48,680 --> 00:25:50,847 fighting situation wherever , then you 643 00:25:50,847 --> 00:25:52,958 need a network of operating platforms 644 00:25:52,958 --> 00:25:55,180 that you can do the same thing . And so 645 00:25:55,180 --> 00:25:57,236 this is the next step . As we evolve 646 00:25:57,236 --> 00:25:59,458 developmental platforms into a fabric . 647 00:25:59,458 --> 00:26:01,402 We move that up to the operational 648 00:26:01,402 --> 00:26:03,458 level and integrate service networks 649 00:26:03,458 --> 00:26:07,230 into into into a global network . This 650 00:26:07,230 --> 00:26:09,230 will give us the capability to have 651 00:26:09,230 --> 00:26:11,420 global situational awareness and then 652 00:26:11,420 --> 00:26:13,142 to achieve the goals of what's 653 00:26:13,142 --> 00:26:15,476 described in jsc to which is , you know , 654 00:26:15,476 --> 00:26:17,642 any sensor , any shooter or any sensor 655 00:26:17,642 --> 00:26:19,830 in any decision maker . Uh we're gonna 656 00:26:19,830 --> 00:26:21,940 build that network , the data stores 657 00:26:21,940 --> 00:26:23,773 and the processes that make that 658 00:26:23,773 --> 00:26:25,940 possible . We're going to do that as a 659 00:26:25,940 --> 00:26:27,996 team across the department , but the 660 00:26:27,996 --> 00:26:29,884 jake hopes to help coordinate the 661 00:26:29,884 --> 00:26:32,570 alliance that brings that together . I 662 00:26:32,570 --> 00:26:36,030 can't add to that . Okay , we'll go to 663 00:26:36,030 --> 00:26:38,600 the next question sir . Go ahead . I'm 664 00:26:38,600 --> 00:26:40,656 living Martinez with abc news . Um , 665 00:26:40,656 --> 00:26:42,933 it's a question for both of you please . 666 00:26:42,933 --> 00:26:46,190 Um General um Secretary work talked 667 00:26:46,190 --> 00:26:49,200 about how china is way ahead on this . 668 00:26:49,210 --> 00:26:52,540 Um , in terms of what you just spoke 669 00:26:52,550 --> 00:26:55,990 about worldwide awareness , china right 670 00:26:55,990 --> 00:26:58,157 now is really still more of a regional 671 00:26:58,157 --> 00:27:00,820 player trying to become a worldwide 672 00:27:00,830 --> 00:27:04,620 player . Um Does a I make that leap for 673 00:27:04,620 --> 00:27:07,100 them or is the Ai advantage that they 674 00:27:07,100 --> 00:27:09,990 have still strictly only regional and 675 00:27:09,990 --> 00:27:12,120 Mr work if I could ask you about ? I 676 00:27:12,120 --> 00:27:14,342 think the final report talked about the 677 00:27:14,342 --> 00:27:16,810 importance of the human element in A . 678 00:27:16,810 --> 00:27:19,090 I can you talk about that especially as 679 00:27:19,100 --> 00:27:21,100 some people may have concerns about 680 00:27:21,100 --> 00:27:22,933 since we're here at the pentagon 681 00:27:22,933 --> 00:27:24,933 talking about how AI relates to the 682 00:27:24,933 --> 00:27:27,340 weaponization of that technology . Yeah , 683 00:27:28,540 --> 00:27:31,380 so thank you for the question . The the 684 00:27:31,390 --> 00:27:33,446 uh I think it's important to kind of 685 00:27:33,446 --> 00:27:35,557 pay attention to what china and their 686 00:27:35,557 --> 00:27:37,779 relationship with AI and the technology 687 00:27:37,779 --> 00:27:39,779 is you know for example the chinese 688 00:27:39,779 --> 00:27:42,910 export autonomous systems to nations 689 00:27:42,920 --> 00:27:44,976 around the world in you know in some 690 00:27:44,976 --> 00:27:47,810 places that that have some pretty some 691 00:27:47,810 --> 00:27:50,170 pretty ugly conflicts that are underway . 692 00:27:50,440 --> 00:27:52,390 Um and you know , lots of human 693 00:27:52,390 --> 00:27:55,560 suffering and uh not a lot of world 694 00:27:55,560 --> 00:27:57,671 attention in some cases . But so here 695 00:27:57,671 --> 00:27:59,504 you are you have a nation that's 696 00:27:59,504 --> 00:28:01,616 proliferating autonomous systems with 697 00:28:01,616 --> 00:28:03,450 no ethical baseline , no sets of 698 00:28:03,450 --> 00:28:06,380 controls , no transparency into those 699 00:28:06,380 --> 00:28:10,130 very dangerous small brush fire wars 700 00:28:10,130 --> 00:28:11,963 that are going on in in a lot of 701 00:28:11,963 --> 00:28:13,463 different places . So that 702 00:28:13,463 --> 00:28:15,186 proliferation of technology is 703 00:28:15,186 --> 00:28:17,408 something that we need to pay attention 704 00:28:17,408 --> 00:28:19,519 to . Similarly , if you look at , you 705 00:28:19,519 --> 00:28:18,740 know , for example , just you know , 706 00:28:18,750 --> 00:28:21,170 right now chinese ships under way , you 707 00:28:21,170 --> 00:28:23,337 know , you know , moving moving east , 708 00:28:23,337 --> 00:28:25,700 you know to you know , as a as a as a 709 00:28:25,710 --> 00:28:28,370 as a demonstration capability shows you 710 00:28:28,370 --> 00:28:30,037 their willingness to push the 711 00:28:30,037 --> 00:28:31,926 boundaries , you know , and to be 712 00:28:31,926 --> 00:28:33,759 considered something more than a 713 00:28:33,759 --> 00:28:35,648 regional power . So that ambition 714 00:28:35,648 --> 00:28:37,537 drives I think is linked to their 715 00:28:37,537 --> 00:28:39,880 technological ambition of AI dominance . 716 00:28:39,880 --> 00:28:42,570 And so we have to look at If these are 717 00:28:42,580 --> 00:28:44,580 if these things are coupled today , 718 00:28:44,580 --> 00:28:46,636 what does that hold for the future ? 719 00:28:46,636 --> 00:28:49,340 You know , in 25 2025 or 2030 , we have 720 00:28:49,340 --> 00:28:51,507 to be prepared for that and we have to 721 00:28:51,507 --> 00:28:54,610 be as agile and as competitive in this 722 00:28:54,610 --> 00:28:56,960 space as the Chinese intend to be . 723 00:28:58,040 --> 00:29:00,262 It's a great question . I would like to 724 00:29:00,940 --> 00:29:04,840 clarify if something I said , we do 725 00:29:04,840 --> 00:29:07,900 not believe china is ahead right now in 726 00:29:07,910 --> 00:29:10,230 a I the way we went about it as a 727 00:29:10,230 --> 00:29:13,290 commission as we said , look a i is not 728 00:29:13,290 --> 00:29:15,550 a single technology , it is a bundle of 729 00:29:15,550 --> 00:29:18,230 technologies and we refer to it as the 730 00:29:18,230 --> 00:29:21,350 Ai stack and the Ai stack has 731 00:29:21,740 --> 00:29:23,851 talent , the people that are going to 732 00:29:23,851 --> 00:29:26,980 use this has data , has the hardware 733 00:29:26,980 --> 00:29:29,330 that actually runs the algorithms , 734 00:29:29,740 --> 00:29:32,110 algorithms , applications and 735 00:29:32,120 --> 00:29:34,640 integration . And so what we tried to 736 00:29:34,640 --> 00:29:36,696 do is we look at each of the six and 737 00:29:36,696 --> 00:29:38,529 said , where does the US have an 738 00:29:38,529 --> 00:29:40,640 advantage ? And where does china have 739 00:29:40,640 --> 00:29:42,751 an advantage ? We believe the U . S . 740 00:29:42,751 --> 00:29:44,862 Has an advantage and talent right now 741 00:29:44,862 --> 00:29:46,918 we definitely are the global kind of 742 00:29:46,918 --> 00:29:49,750 magnet for best talent . There's a lot 743 00:29:49,750 --> 00:29:51,972 of things changing in that . And unless 744 00:29:51,972 --> 00:29:53,861 we're smart about our immigration 745 00:29:53,861 --> 00:29:55,917 policies , et cetera , we could lose 746 00:29:55,917 --> 00:29:58,450 that . But right now we judge that we 747 00:29:58,460 --> 00:30:01,800 have better talent . Second , we know 748 00:30:01,800 --> 00:30:03,967 we have an advantage in hardware , the 749 00:30:03,967 --> 00:30:06,220 United States and the West most Mueller 750 00:30:06,220 --> 00:30:08,750 broadly . And we think we have an 751 00:30:08,750 --> 00:30:11,210 advantage in our algorithms . Although 752 00:30:11,210 --> 00:30:13,321 the Chinese are really pushing hard . 753 00:30:13,321 --> 00:30:15,488 We think that they could catch up with 754 00:30:15,488 --> 00:30:18,120 us within 5 to 10 years now they have 755 00:30:18,120 --> 00:30:21,800 an advantage in our view , uh in data , 756 00:30:21,810 --> 00:30:23,977 they have a lot of data and they don't 757 00:30:23,977 --> 00:30:27,250 have the restrictions on privacy 758 00:30:27,250 --> 00:30:30,650 etcetera that we do . They have an 759 00:30:30,650 --> 00:30:33,250 advantage in applications . They're 760 00:30:33,250 --> 00:30:35,600 very good at that . And we think they 761 00:30:35,600 --> 00:30:37,390 have an apple to an advantage 762 00:30:37,400 --> 00:30:39,370 integration because they have a 763 00:30:39,370 --> 00:30:43,200 coherent strategy to get all of the Ai 764 00:30:43,200 --> 00:30:45,920 stack together to give them a national 765 00:30:45,930 --> 00:30:49,350 advantage . Now . We judge because 766 00:30:49,360 --> 00:30:53,350 talent hardware and algorithms are so 767 00:30:53,350 --> 00:30:56,320 central and important to the stack . We 768 00:30:56,320 --> 00:30:58,431 judge that the United States actually 769 00:30:58,431 --> 00:31:01,270 is ahead of china in A . I 770 00:31:01,840 --> 00:31:04,970 technology is more broadly but what 771 00:31:04,970 --> 00:31:08,590 we're saying is the chinese are far 772 00:31:08,590 --> 00:31:11,590 more organized for a competition and 773 00:31:11,590 --> 00:31:14,270 have a strategy to win the competition 774 00:31:14,840 --> 00:31:17,770 and are putting in a lot of resources . 775 00:31:18,440 --> 00:31:21,460 So as lieutenant General grown said 776 00:31:21,470 --> 00:31:23,910 they want to be the world leader in A I 777 00:31:23,910 --> 00:31:27,250 technology by 2030 . As soon as they 778 00:31:27,250 --> 00:31:28,972 say that that means to me they 779 00:31:28,972 --> 00:31:31,083 recognize that they are not the world 780 00:31:31,083 --> 00:31:33,250 ai leader now And it's gonna take them 781 00:31:33,250 --> 00:31:35,472 about 10 years , they think eight years 782 00:31:35,472 --> 00:31:39,050 or so uh to surpass the United States . 783 00:31:39,440 --> 00:31:41,607 That's why we say look we better be in 784 00:31:41,607 --> 00:31:44,640 this competition full on by 2025 . If 785 00:31:44,640 --> 00:31:47,640 we don't then we run the risk of them 786 00:31:47,650 --> 00:31:49,710 surpassing us . So I just wanted to 787 00:31:49,710 --> 00:31:51,710 clarify that , I wasn't saying that 788 00:31:51,710 --> 00:31:54,860 china is ahead of us in a . I . Uh The 789 00:31:54,860 --> 00:31:57,770 second thing part of your question is 790 00:31:58,640 --> 00:32:00,807 all you got to do is look at what they 791 00:32:00,807 --> 00:32:03,560 did with Huawei to say the way they 792 00:32:03,560 --> 00:32:06,670 think about becoming a global power is 793 00:32:06,670 --> 00:32:09,990 not by invading countries . It is 794 00:32:09,990 --> 00:32:12,300 putting out a I . Platform , excuse me . 795 00:32:12,300 --> 00:32:15,270 Technology platforms that allow their 796 00:32:15,280 --> 00:32:18,760 values to proliferate around the 797 00:32:18,760 --> 00:32:20,871 world . And that's what happened with 798 00:32:20,871 --> 00:32:23,560 Huawei um And the other place they're 799 00:32:23,560 --> 00:32:25,910 going really hog wild on our global 800 00:32:25,910 --> 00:32:28,710 standard setting , which is kind of the 801 00:32:28,710 --> 00:32:31,190 U . S . That's in our wheelhouse . 802 00:32:31,190 --> 00:32:33,357 We've been doing that since the end of 803 00:32:33,357 --> 00:32:35,357 World War two . And the chinese are 804 00:32:35,357 --> 00:32:37,670 actually coordinating with the Russians 805 00:32:38,140 --> 00:32:41,270 to set global standards in ai that 806 00:32:41,270 --> 00:32:44,170 prefer their type of technology . 807 00:32:44,640 --> 00:32:47,550 So without question , I agree with uh 808 00:32:47,560 --> 00:32:49,560 with lieutenant general groan , the 809 00:32:49,560 --> 00:32:51,800 chinese have ambitions to be a global 810 00:32:51,800 --> 00:32:55,090 power . They say by 2050 actually 2049 811 00:32:55,090 --> 00:32:57,360 is the 100 year anniversary . They want 812 00:32:57,360 --> 00:32:59,690 to be have the largest economy in the 813 00:32:59,690 --> 00:33:02,530 world and they want to be the foremost 814 00:33:02,530 --> 00:33:05,450 military power in the world . Um That's 815 00:33:05,450 --> 00:33:07,450 not a future that the United States 816 00:33:07,450 --> 00:33:09,561 should say . Yeah let's just let that 817 00:33:09,561 --> 00:33:12,160 happen , let's compete because we want 818 00:33:12,160 --> 00:33:15,460 to be the world's foremost uh 819 00:33:15,840 --> 00:33:18,560 military power and we want to be the 820 00:33:18,560 --> 00:33:22,010 most dynamic innovative economy in the 821 00:33:22,010 --> 00:33:25,490 world . So the chinese definitely 822 00:33:25,490 --> 00:33:28,040 have global ambitions . They are 823 00:33:28,040 --> 00:33:30,630 regional power now , but they're really 824 00:33:30,630 --> 00:33:32,720 starting to move more broadly on the 825 00:33:32,720 --> 00:33:36,410 world stage . Next question 826 00:33:36,420 --> 00:33:38,587 goes to Sydney Freedberg from breaking 827 00:33:38,587 --> 00:33:41,960 defense . Go ahead . Hi , thank you 828 00:33:41,960 --> 00:33:43,738 very much for doing us . Sydney 829 00:33:43,738 --> 00:33:45,904 Freedberg . Breaking defense here . Uh 830 00:33:45,904 --> 00:33:48,071 let me let me ask a question perfectly 831 00:33:48,071 --> 00:33:50,182 for general grown uh , of the various 832 00:33:50,182 --> 00:33:52,460 recommendations in the Commission final 833 00:33:52,460 --> 00:33:56,440 report , which ones is A D O D . Uh 834 00:33:56,450 --> 00:33:58,630 contemplating which ones are actually 835 00:33:58,640 --> 00:34:00,584 concurred with that . You guys are 836 00:34:00,584 --> 00:34:03,880 going to try to put forward uh by by 837 00:34:03,880 --> 00:34:05,936 yourselves or by asking Congress for 838 00:34:05,936 --> 00:34:08,102 legislation and which ones do you guys 839 00:34:08,102 --> 00:34:10,470 actually not concur with the things , 840 00:34:10,470 --> 00:34:12,740 you know , the fair conditions like the 841 00:34:12,750 --> 00:34:15,610 steering committee , like uh setting 842 00:34:15,610 --> 00:34:17,900 the various targets uh like , you know , 843 00:34:17,900 --> 00:34:20,122 coming up with a strategy antics and so 844 00:34:20,122 --> 00:34:22,011 forth . Uh Can you go through the 845 00:34:22,011 --> 00:34:23,956 checklist of things the commission 846 00:34:23,956 --> 00:34:26,690 wants to do ? That you guys are green 847 00:34:26,690 --> 00:34:28,746 light , yellow light or red light on 848 00:34:28,746 --> 00:34:31,360 proceeding with ? Yeah . Great question . 849 00:34:31,360 --> 00:34:34,010 Sydney , Good morning . Um so , so 850 00:34:34,020 --> 00:34:36,131 really , really good question . Now , 851 00:34:36,131 --> 00:34:38,242 the NsC AI report , if you look at it 852 00:34:38,242 --> 00:34:41,060 in its full breath addresses a lot of 853 00:34:41,060 --> 00:34:43,282 the recommendations are at the national 854 00:34:43,282 --> 00:34:45,449 level , a place where defense may play 855 00:34:45,449 --> 00:34:48,350 a part defense might not lead . Uh 856 00:34:48,360 --> 00:34:50,630 There are there is a subset Of 857 00:34:50,640 --> 00:34:53,590 recommendations on the order of 40 that 858 00:34:53,590 --> 00:34:55,646 that that we that we've taken a hard 859 00:34:55,646 --> 00:34:57,600 look at that , that our military 860 00:34:57,600 --> 00:34:59,880 specific and that that really by all 861 00:34:59,880 --> 00:35:01,936 rights defence would lead . So as we 862 00:35:01,936 --> 00:35:04,158 look at that list , or I'm sorry , it's 863 00:35:04,158 --> 00:35:06,158 closer to 100 recommendations as we 864 00:35:06,158 --> 00:35:08,940 look at that list . Um a good number of 865 00:35:08,940 --> 00:35:10,829 them . About about half , maybe a 866 00:35:10,829 --> 00:35:12,996 little bit more . We're already moving 867 00:35:12,996 --> 00:35:15,162 out on to a significant degree . So in 868 00:35:15,162 --> 00:35:17,273 those cases it's really just a matter 869 00:35:17,273 --> 00:35:19,551 for us of taking a look at the N S C A . 870 00:35:19,551 --> 00:35:21,690 I reckon recommendations in detail to 871 00:35:21,690 --> 00:35:23,690 make sure that we've considered the 872 00:35:23,690 --> 00:35:25,857 full scope of what might appear in one 873 00:35:25,857 --> 00:35:27,968 of those recommendations and then see 874 00:35:27,968 --> 00:35:30,079 if what we're doing today aligns with 875 00:35:30,079 --> 00:35:32,190 those . So that's that's that's , you 876 00:35:32,190 --> 00:35:34,301 know , kind of one large subset which 877 00:35:34,301 --> 00:35:36,523 is which is the majority . Then there's 878 00:35:36,523 --> 00:35:38,523 another set of recommendations that 879 00:35:38,523 --> 00:35:40,579 that we've looked at , but we really 880 00:35:40,579 --> 00:35:42,746 don't have a plan for yet . You know , 881 00:35:42,746 --> 00:35:44,801 we recognize that it's a problem but 882 00:35:44,801 --> 00:35:44,430 we're not quite ready to move out in 883 00:35:44,430 --> 00:35:46,263 that direction , just because of 884 00:35:46,263 --> 00:35:48,208 limited bandwidth here . So that's 885 00:35:48,208 --> 00:35:50,319 that's another subset that that we're 886 00:35:50,319 --> 00:35:52,430 looking at . And then there's a third 887 00:35:52,430 --> 00:35:54,597 subset that , you know , those that we 888 00:35:54,597 --> 00:35:56,819 really have to look hard at . There are 889 00:35:56,819 --> 00:35:58,819 things that we hadn't thought about 890 00:35:58,819 --> 00:35:58,280 before , and we really need to kind of 891 00:35:58,280 --> 00:36:00,502 pull the strings on the implications of 892 00:36:00,502 --> 00:36:02,558 those . So there's that third subset 893 00:36:02,558 --> 00:36:04,780 when you when you talk about which ones 894 00:36:04,780 --> 00:36:06,891 we agree with or don't agree with , I 895 00:36:06,891 --> 00:36:09,058 can't think of any that we don't agree 896 00:36:09,058 --> 00:36:10,836 with . The things that are most 897 00:36:10,836 --> 00:36:13,058 pressing that most closely aligned with 898 00:36:13,058 --> 00:36:15,340 what we're doing today are these ideas , 899 00:36:15,340 --> 00:36:18,180 the ideas associated with starting to 900 00:36:18,180 --> 00:36:22,030 create uh enterprise of capabilities ? 901 00:36:22,040 --> 00:36:25,150 Um all of the recommendations about the 902 00:36:25,160 --> 00:36:28,010 ethical foundations , We are all about 903 00:36:28,020 --> 00:36:30,530 fleshing out our ethical foundations 904 00:36:30,540 --> 00:36:32,762 and really really integrating that into 905 00:36:32,762 --> 00:36:34,920 every aspect of our process . The 906 00:36:34,920 --> 00:36:38,360 recommendations about organizing uh you 907 00:36:38,360 --> 00:36:40,471 know , with with defense priorities , 908 00:36:40,471 --> 00:36:42,638 um you know , that will be the subject 909 00:36:42,638 --> 00:36:44,582 of the department . So we as an Ai 910 00:36:44,582 --> 00:36:46,693 community can advocate but that's the 911 00:36:46,693 --> 00:36:48,693 department process that will decide 912 00:36:48,693 --> 00:36:50,749 what the priorities are . And so and 913 00:36:50,749 --> 00:36:52,471 we'll adhere to whatever those 914 00:36:52,471 --> 00:36:53,971 priorities articulated the 915 00:36:53,971 --> 00:36:55,693 recommendation about workforce 916 00:36:55,693 --> 00:36:57,360 development and the family of 917 00:36:57,360 --> 00:36:59,138 recommendations about workforce 918 00:36:59,138 --> 00:37:01,360 development . We could not agree more . 919 00:37:01,360 --> 00:37:03,638 So how do we , how do we go ? You know , 920 00:37:03,638 --> 00:37:05,638 have a full range of train training 921 00:37:05,638 --> 00:37:07,860 environment or an education environment 922 00:37:07,860 --> 00:37:09,916 that includes , you know , just like 923 00:37:09,916 --> 00:37:09,750 short short duration tactical training 924 00:37:09,750 --> 00:37:11,861 for example for you know a code or to 925 00:37:11,861 --> 00:37:14,480 get on a platform all the way to 926 00:37:14,490 --> 00:37:16,980 building service academies are building 927 00:37:16,990 --> 00:37:18,934 ROTC scholarships and that sort of 928 00:37:18,934 --> 00:37:21,760 thing . So so across the department , 929 00:37:21,800 --> 00:37:23,911 as some of these recommendations with 930 00:37:23,911 --> 00:37:26,860 large scale and large scope , it starts 931 00:37:26,860 --> 00:37:29,027 to uh you know supersede what just the 932 00:37:29,027 --> 00:37:31,082 Ai community and the department does 933 00:37:31,082 --> 00:37:33,060 too . So we work closely with the 934 00:37:33,060 --> 00:37:35,171 research and Engineering department , 935 00:37:35,171 --> 00:37:38,570 We work closely with the personnel and 936 00:37:38,570 --> 00:37:40,237 readiness and the acquisition 937 00:37:40,237 --> 00:37:42,170 sustainment to start to form the 938 00:37:42,170 --> 00:37:45,470 coalitions to to get after the problems 939 00:37:45,470 --> 00:37:47,850 that that that that that are underneath 940 00:37:47,850 --> 00:37:50,072 those recommendations to make sure that 941 00:37:50,072 --> 00:37:52,017 we understand them and that we are 942 00:37:52,017 --> 00:37:54,380 actually moving towards this this new 943 00:37:54,390 --> 00:37:57,120 uh operational model for how we are 944 00:37:57,120 --> 00:37:59,240 going to operate as a department . 945 00:38:00,020 --> 00:38:02,242 Thank you . Sydney Sydney , I guess the 946 00:38:02,242 --> 00:38:04,131 way I would answer this , I can't 947 00:38:04,131 --> 00:38:05,964 really add too much more to what 948 00:38:05,964 --> 00:38:08,810 Lieutenant general grown said is just a 949 00:38:08,810 --> 00:38:10,866 little while ago , Secretary Defense 950 00:38:10,866 --> 00:38:13,460 Mark esper said AI is the number one 951 00:38:13,460 --> 00:38:15,980 priority for me as a secretary Defense . 952 00:38:15,990 --> 00:38:18,890 And he went on to say the competitor 953 00:38:18,890 --> 00:38:21,780 that really wins in the AI competition 954 00:38:21,790 --> 00:38:24,060 will have a battlefield advantage for 955 00:38:24,060 --> 00:38:27,750 decades now , if you believe that , 956 00:38:28,220 --> 00:38:30,400 and I certainly do and I believe the 957 00:38:30,400 --> 00:38:32,233 commission not to think that's a 958 00:38:32,233 --> 00:38:34,380 unanimous consensus . If you really 959 00:38:34,380 --> 00:38:37,140 believe that you can't keep doing what 960 00:38:37,140 --> 00:38:39,307 we're doing now . I mean , the Defense 961 00:38:39,307 --> 00:38:41,720 Science board said in 2014 , the one 962 00:38:41,720 --> 00:38:44,000 thing you gotta get right is A I and A 963 00:38:44,000 --> 00:38:46,260 I . Enabled autonomy . So here we are , 964 00:38:46,260 --> 00:38:48,950 seven years later we're saying okay , 965 00:38:49,420 --> 00:38:52,710 if we really believe That Ai is going 966 00:38:52,710 --> 00:38:54,932 to give a competitor an advantage for a 967 00:38:54,932 --> 00:38:56,877 decade , are we satisfied with the 968 00:38:56,877 --> 00:38:59,240 progress that has happened since 2014 ? 969 00:39:00,320 --> 00:39:02,800 And if the answer is no , then you have 970 00:39:02,800 --> 00:39:04,689 to say , then we've got to change 971 00:39:04,689 --> 00:39:06,744 things up . And of course people are 972 00:39:06,744 --> 00:39:08,800 gonna say , hey , why would you make 973 00:39:08,800 --> 00:39:11,133 the undersecretary of defense for Arnie , 974 00:39:11,133 --> 00:39:13,840 the co chair and the chief science 975 00:39:13,840 --> 00:39:16,330 officer of the J . Rock , J Rock works 976 00:39:16,330 --> 00:39:19,210 Perfect . Well does every single 977 00:39:19,210 --> 00:39:21,990 program have a plug in it for A . I . 978 00:39:22,000 --> 00:39:24,430 And being able to receive data for 979 00:39:24,430 --> 00:39:26,970 machine learning chips ? Does it have 980 00:39:26,970 --> 00:39:29,320 the ports to allow them to pass on 981 00:39:29,330 --> 00:39:31,390 information ? If the answer is no , 982 00:39:31,390 --> 00:39:33,630 we're not doing good enough . I think 983 00:39:33,640 --> 00:39:35,807 Lieutenant General Excuse me . General 984 00:39:35,807 --> 00:39:38,029 Heighten the vice chair . Vice chairman 985 00:39:38,029 --> 00:39:40,140 has said this very clearly . He's not 986 00:39:40,140 --> 00:39:42,251 satisfied with the way that J Rock is 987 00:39:42,251 --> 00:39:44,473 functioning and he wants to change it . 988 00:39:44,473 --> 00:39:46,640 So it really pushes these more broader 989 00:39:46,640 --> 00:39:49,260 joint system of system things that uh 990 00:39:49,270 --> 00:39:51,381 look , Lieutenant General growing was 991 00:39:51,381 --> 00:39:52,830 talking about so from the 992 00:39:52,840 --> 00:39:56,140 commissioner's point of view , look uh 993 00:39:57,420 --> 00:39:59,820 right now we do not believe we are 994 00:39:59,820 --> 00:40:03,240 moving as fast as we should . And if 995 00:40:04,010 --> 00:40:06,440 the department agrees with that general 996 00:40:06,440 --> 00:40:08,551 assessment , then they need to change 997 00:40:08,551 --> 00:40:12,140 things . Mhm . Go ahead . 998 00:40:13,110 --> 00:40:15,054 Thank you for taking my question , 999 00:40:15,054 --> 00:40:17,250 Christina Anderson . A WPS News . Um I 1000 00:40:17,250 --> 00:40:19,430 wonder if you could speak to getting 1001 00:40:19,430 --> 00:40:21,486 the data , the security , the secure 1002 00:40:21,486 --> 00:40:23,770 data fabric . Right ? And then taking 1003 00:40:23,770 --> 00:40:27,110 that up a notch to kind of the global a 1004 00:40:27,120 --> 00:40:30,960 structure of ai how can you think about 1005 00:40:30,970 --> 00:40:33,540 building this structure so that 1006 00:40:33,550 --> 00:40:35,494 security is one of the fundamental 1007 00:40:35,494 --> 00:40:37,550 elements of that . That's one of the 1008 00:40:37,550 --> 00:40:39,717 criticisms of the internet right now . 1009 00:40:39,717 --> 00:40:41,272 Is that when it was built , 1010 00:40:41,272 --> 00:40:43,494 notwithstanding the tremendous benefits 1011 00:40:43,494 --> 00:40:45,661 that we have that it has was not built 1012 00:40:45,661 --> 00:40:48,200 with security in mind . Yeah . Thanks , 1013 00:40:48,200 --> 00:40:50,367 Christina . That's that's an excellent 1014 00:40:50,367 --> 00:40:52,256 question . And to me , that's the 1015 00:40:52,256 --> 00:40:54,533 operative question , because I I think , 1016 00:40:54,533 --> 00:40:53,760 you know , there's a there's a there's 1017 00:40:53,760 --> 00:40:56,660 a good alignment as we talk about , you 1018 00:40:56,660 --> 00:40:58,827 know , the operational effects that we 1019 00:40:58,827 --> 00:41:00,549 want to achieve . There's good 1020 00:41:00,549 --> 00:41:02,660 alignment when we talk about building 1021 00:41:02,660 --> 00:41:04,771 platforms and how we , you know , how 1022 00:41:04,771 --> 00:41:06,993 we're going to integrate data and share 1023 00:41:06,993 --> 00:41:09,216 data . The very first question we start 1024 00:41:09,216 --> 00:41:11,271 to ask at that point is , okay , How 1025 00:41:11,271 --> 00:41:13,493 are we going to secure this ? How do we 1026 00:41:13,493 --> 00:41:15,660 secure this environment ? And so we we 1027 00:41:15,660 --> 00:41:17,604 have a full court press on . So of 1028 00:41:17,604 --> 00:41:19,549 course we have , you know , native 1029 00:41:19,549 --> 00:41:21,716 cloud security , you know , additional 1030 00:41:21,716 --> 00:41:21,650 security that that that that we've 1031 00:41:21,660 --> 00:41:23,660 we've been able to add , you know , 1032 00:41:23,660 --> 00:41:25,771 we've got lots of , you know , uh you 1033 00:41:25,771 --> 00:41:27,604 know , cybersecurity specialists 1034 00:41:27,604 --> 00:41:29,716 helping us look at this problem set , 1035 00:41:29,716 --> 00:41:31,882 but more importantly , uh we're trying 1036 00:41:31,882 --> 00:41:33,993 to keep an eye on the entire research 1037 00:41:33,993 --> 00:41:36,216 and development ecosystem . So not just 1038 00:41:36,216 --> 00:41:38,438 from a cyber security perspective , but 1039 00:41:38,438 --> 00:41:40,604 how do we deal with adversarial ai for 1040 00:41:40,604 --> 00:41:42,271 example , how do we deal with 1041 00:41:42,271 --> 00:41:44,610 purposeful intent to intervene or to 1042 00:41:44,620 --> 00:41:46,787 interfere with our algorithms or spoof 1043 00:41:46,787 --> 00:41:49,009 our algorithms ? So this is a very this 1044 00:41:49,009 --> 00:41:50,898 is probably , I would say this is 1045 00:41:50,898 --> 00:41:53,270 certainly the top priority and probably 1046 00:41:53,270 --> 00:41:55,270 our largest effort right now from a 1047 00:41:55,270 --> 00:41:57,492 research and development perspective is 1048 00:41:57,492 --> 00:41:59,548 how do we make sure that as we build 1049 00:41:59,548 --> 00:42:01,437 this out , we squeeze out all the 1050 00:42:01,437 --> 00:42:03,670 vulnerabilities that we can we will 1051 00:42:03,670 --> 00:42:05,781 never have a perfect system . We will 1052 00:42:05,781 --> 00:42:07,948 never have a perfect internet , but we 1053 00:42:07,948 --> 00:42:09,892 need to protect it . Like we would 1054 00:42:09,892 --> 00:42:12,114 protect any weapons system or any other 1055 00:42:12,114 --> 00:42:14,114 critical note . Thank you . Central 1056 00:42:14,114 --> 00:42:17,020 question . Uh Christina . Um 1057 00:42:19,800 --> 00:42:21,744 as a lieutenant general grown said 1058 00:42:21,744 --> 00:42:23,580 we're moving into an era of ai 1059 00:42:23,580 --> 00:42:27,470 competition and poisoning data is a 1060 00:42:27,470 --> 00:42:30,110 way to gain an advantage . We have to 1061 00:42:30,110 --> 00:42:32,840 be able to guard against that . We need 1062 00:42:32,840 --> 00:42:35,300 to read team the heck out of our 1063 00:42:35,300 --> 00:42:37,490 databases . I need to have people 1064 00:42:37,490 --> 00:42:39,910 trying to break into the database and 1065 00:42:39,910 --> 00:42:43,900 poison data often so that we 1066 00:42:43,900 --> 00:42:46,010 can identify vulnerabilities and fix 1067 00:42:46,010 --> 00:42:48,940 them . We have to have means by which 1068 00:42:48,940 --> 00:42:52,010 to check the data . And there's all 1069 00:42:52,010 --> 00:42:54,660 sorts of different things that the 1070 00:42:54,660 --> 00:42:58,390 commercial , the commercial 1071 00:42:58,390 --> 00:43:00,990 sector is doing this also . They're 1072 00:43:01,000 --> 00:43:03,940 looking how do you protect the data and 1073 00:43:03,940 --> 00:43:06,820 how do you protect your algorithms um 1074 00:43:07,200 --> 00:43:09,110 to make sure that no biases are 1075 00:43:09,120 --> 00:43:12,940 inserted ? So um look , we don't have 1076 00:43:12,940 --> 00:43:14,970 the answers of , you know , all the 1077 00:43:14,970 --> 00:43:17,800 answers for this yet , but it's central 1078 00:43:17,800 --> 00:43:20,022 to the thinking of the jake I think you 1079 00:43:20,022 --> 00:43:21,420 heard and 1080 00:43:25,100 --> 00:43:27,820 our AI has to be better than their A I 1081 00:43:28,300 --> 00:43:30,760 all you have to do is envision an AI 1082 00:43:30,760 --> 00:43:33,910 enables cyber attack and if there AI is 1083 00:43:33,910 --> 00:43:36,150 better on offense than our AI is better 1084 00:43:36,150 --> 00:43:38,261 on defense . That's going to be a bad 1085 00:43:38,261 --> 00:43:41,590 day for us . So um you know , constant 1086 00:43:41,590 --> 00:43:43,812 red teaming , constant development with 1087 00:43:43,880 --> 00:43:47,060 cops in mind , constant testing and 1088 00:43:47,060 --> 00:43:49,810 evaluation validation and verification . 1089 00:43:50,300 --> 00:43:53,250 Uh This is our future . Now it's going 1090 00:43:53,250 --> 00:43:55,472 to be something we just have to take as 1091 00:43:55,472 --> 00:43:59,020 a matter of course . Next question goes 1092 00:43:59,020 --> 00:44:01,076 out to Tony from Bloomberg News . Go 1093 00:44:01,076 --> 00:44:03,610 ahead Tony uh huh Hi , this is Tony to 1094 00:44:03,610 --> 00:44:05,630 paseo . I have a question in an 1095 00:44:05,640 --> 00:44:07,862 operational application question that I 1096 00:44:07,862 --> 00:44:09,973 think most of the citizens can relate 1097 00:44:09,973 --> 00:44:11,940 to next month marks the 10th 1098 00:44:11,940 --> 00:44:13,996 anniversary of the bin laden raid by 1099 00:44:13,996 --> 00:44:17,820 Seal team six . Conceptually if AI was 1100 00:44:17,830 --> 00:44:20,720 in widespread use in 2011 , how might 1101 00:44:20,730 --> 00:44:23,060 it have been employed in planning and 1102 00:44:23,060 --> 00:44:24,949 executing the raid ? I'm thinking 1103 00:44:24,949 --> 00:44:27,160 facial recognition , pinpointing the 1104 00:44:27,160 --> 00:44:29,382 movements of activity in and around the 1105 00:44:29,382 --> 00:44:31,500 compound , calculating the height of 1106 00:44:31,500 --> 00:44:33,778 the walls and their thickness etcetera . 1107 00:44:33,778 --> 00:44:36,640 Can you think outside the box and give 1108 00:44:36,640 --> 00:44:38,696 us a couple examples of how it might 1109 00:44:38,696 --> 00:44:41,510 have been used in that raid ? Yeah . 1110 00:44:41,520 --> 00:44:43,631 Hey , great , great question , Tony . 1111 00:44:43,631 --> 00:44:45,742 And I think , I think that raised , I 1112 00:44:45,742 --> 00:44:47,909 mean , so when I look at , when I look 1113 00:44:47,909 --> 00:44:49,964 at that , remember when I said , you 1114 00:44:49,964 --> 00:44:51,520 know , amateur study apps , 1115 00:44:51,520 --> 00:44:53,576 professional study architectures , I 1116 00:44:53,576 --> 00:44:55,353 think I think we if we take any 1117 00:44:55,353 --> 00:44:57,353 military operation , I can't really 1118 00:44:57,353 --> 00:44:59,020 speak to , you know , to that 1119 00:44:59,020 --> 00:45:01,242 particular event . But but any military 1120 00:45:01,242 --> 00:45:03,464 operation , it's easy to get fixated on 1121 00:45:03,464 --> 00:45:06,020 the on the applications that that exist 1122 00:45:06,020 --> 00:45:08,020 on the tactical edge . But when you 1123 00:45:08,020 --> 00:45:10,242 when you walk back a military problem , 1124 00:45:10,242 --> 00:45:12,520 I mean you start with those , you know , 1125 00:45:12,520 --> 00:45:14,798 you start with those tactical warnings , 1126 00:45:14,798 --> 00:45:14,720 you know , on the objective or near the 1127 00:45:14,720 --> 00:45:17,100 objective and you back up a step and 1128 00:45:17,100 --> 00:45:19,080 you need to be broadly situational 1129 00:45:19,080 --> 00:45:21,870 aware and you back up another step and 1130 00:45:21,870 --> 00:45:24,370 you need to be aware of not just the 1131 00:45:24,370 --> 00:45:26,426 red capabilities and the red force , 1132 00:45:26,426 --> 00:45:28,426 but you also need to know where the 1133 00:45:28,426 --> 00:45:30,537 blue forces and where your own forces 1134 00:45:30,537 --> 00:45:32,537 are and their readiness and they're 1135 00:45:32,537 --> 00:45:34,759 ready and their availability . You also 1136 00:45:34,759 --> 00:45:36,870 need to understand the Green forces , 1137 00:45:36,870 --> 00:45:36,610 you know , those partner forces that we 1138 00:45:36,610 --> 00:45:38,610 might have in the area or the White 1139 00:45:38,610 --> 00:45:40,554 forces , you know , the civilian , 1140 00:45:40,554 --> 00:45:42,777 innocent civilian populations who might 1141 00:45:42,777 --> 00:45:44,777 be in the area like so all of those 1142 00:45:44,777 --> 00:45:46,443 kind of situational awareness 1143 00:45:46,443 --> 00:45:48,499 activities can be worked through A i 1144 00:45:48,499 --> 00:45:50,721 right that can be done much better than 1145 00:45:50,721 --> 00:45:52,388 a human being can do it by by 1146 00:45:52,388 --> 00:45:54,670 leveraging ai to work on all that data . 1147 00:45:54,680 --> 00:45:56,791 And you start backing up even further 1148 00:45:56,791 --> 00:45:58,902 when you talk about , well how do you 1149 00:45:58,902 --> 00:46:01,280 have uh affects integration ? Like when 1150 00:46:01,290 --> 00:46:03,570 you know when do you when do you get 1151 00:46:03,580 --> 00:46:05,469 onto the objective and how do you 1152 00:46:05,469 --> 00:46:07,636 coordinate with an adjacent unit ? How 1153 00:46:07,636 --> 00:46:10,190 do you make sure that your you know 1154 00:46:10,190 --> 00:46:13,120 that your fires are safe and and uh are 1155 00:46:13,120 --> 00:46:16,720 focused on the good targets ? Again , 1156 00:46:16,730 --> 00:46:19,880 a I can help with the information flow 1157 00:46:19,880 --> 00:46:22,900 that informs that decision making back 1158 00:46:22,900 --> 00:46:25,233 up further , you know , weather effects . 1159 00:46:25,233 --> 00:46:27,344 Do we have global weather that's in a 1160 00:46:27,344 --> 00:46:29,344 database that everybody can use and 1161 00:46:29,344 --> 00:46:31,456 integrate into their application ? Do 1162 00:46:31,456 --> 00:46:33,122 we have threat picture that's 1163 00:46:33,122 --> 00:46:35,178 integrated into our applications and 1164 00:46:35,178 --> 00:46:37,011 defense ? Do we know threatening 1165 00:46:37,011 --> 00:46:39,233 behavior ? Have we modeled that ? Do we 1166 00:46:39,233 --> 00:46:41,178 use it for understanding the human 1167 00:46:41,178 --> 00:46:43,344 populations uh predictive modeling and 1168 00:46:43,344 --> 00:46:45,511 you know , the list goes on and on and 1169 00:46:45,511 --> 00:46:47,733 you know , the further you go back into 1170 00:46:47,733 --> 00:46:47,630 the into the institution , you're 1171 00:46:47,630 --> 00:46:49,852 talking about modeling and simulation , 1172 00:46:49,852 --> 00:46:51,741 platform maintenance , you know , 1173 00:46:51,741 --> 00:46:53,852 preventive maintenance for helicopter 1174 00:46:53,852 --> 00:46:55,963 platforms for example . Uh Integrated 1175 00:46:55,963 --> 00:46:58,074 logistics contingency management , uh 1176 00:46:58,074 --> 00:47:00,297 you know fleet maintenance , you know , 1177 00:47:00,297 --> 00:47:02,352 think think of like an electronic or 1178 00:47:02,352 --> 00:47:05,060 electric car company that broadcast 1179 00:47:05,070 --> 00:47:07,010 updates to their entire fleet of 1180 00:47:07,010 --> 00:47:08,940 vehicles . This is the sort of 1181 00:47:08,940 --> 00:47:11,080 capabilities that Ai brings to the 1182 00:47:11,080 --> 00:47:13,330 department and when you start stacking 1183 00:47:13,330 --> 00:47:15,620 those up , you really see how it 1184 00:47:15,620 --> 00:47:19,130 focuses . You focus that lens on a 1185 00:47:19,140 --> 00:47:21,450 tactical military problem . It's not 1186 00:47:21,450 --> 00:47:23,617 just the Ai at the tactical edge , but 1187 00:47:23,617 --> 00:47:26,050 it's all of the Ai that has contributed 1188 00:47:26,060 --> 00:47:28,171 all the way to the back office of the 1189 00:47:28,171 --> 00:47:30,227 pentagon where we're doing financial 1190 00:47:30,227 --> 00:47:31,949 records , right ? Or inventory 1191 00:47:31,949 --> 00:47:34,004 management or all of the sort of the 1192 00:47:34,004 --> 00:47:36,770 business of the defense focus through 1193 00:47:36,780 --> 00:47:39,560 data into that objective . So I hope I 1194 00:47:39,560 --> 00:47:41,616 hope that helps you know what , I'll 1195 00:47:41,616 --> 00:47:43,893 just give you one other point . I mean , 1196 00:47:43,893 --> 00:47:45,893 for every for almost every military 1197 00:47:45,893 --> 00:47:48,060 activity , there's a commercial analog 1198 00:47:48,060 --> 00:47:50,171 to that activity . I mean , you think 1199 00:47:50,171 --> 00:47:52,282 about that , you know , a large scale 1200 00:47:52,282 --> 00:47:54,282 uh shopping online shopping network 1201 00:47:54,282 --> 00:47:56,227 that has to deal with ordering and 1202 00:47:56,227 --> 00:47:58,393 buying and recommending and presenting 1203 00:47:58,393 --> 00:48:00,282 options and selecting options and 1204 00:48:00,282 --> 00:48:02,393 delivering , you know , for every one 1205 00:48:02,393 --> 00:48:04,616 of those has a parallel in the military 1206 00:48:04,616 --> 00:48:06,727 space . The ai that we integrate from 1207 00:48:06,727 --> 00:48:08,560 commercial industry today , that 1208 00:48:08,560 --> 00:48:10,560 technology that's readily available 1209 00:48:10,560 --> 00:48:12,727 helps us do those same things with the 1210 00:48:12,727 --> 00:48:15,010 efficiency and productivity that that 1211 00:48:15,020 --> 00:48:17,970 any large scale successful commercial 1212 00:48:17,970 --> 00:48:20,020 corporation does today . And from a 1213 00:48:20,020 --> 00:48:22,131 business perspective , that's exactly 1214 00:48:22,131 --> 00:48:24,298 what we need to have . Thanks . It's a 1215 00:48:24,298 --> 00:48:27,350 clever question , Tony . And to me the 1216 00:48:27,360 --> 00:48:30,880 biggest change would be our 1217 00:48:30,880 --> 00:48:34,830 ability to look at enormous amounts of 1218 00:48:34,830 --> 00:48:38,820 social media data etcetera uh 1219 00:48:38,830 --> 00:48:41,160 to make predictive analysis and also 1220 00:48:41,160 --> 00:48:44,740 make judgments . I'm a movie aficionado . 1221 00:48:44,740 --> 00:48:46,684 So everything I know about the bin 1222 00:48:46,684 --> 00:48:48,740 laden raid I learned in Zero Dark 30 1223 00:48:49,270 --> 00:48:52,090 and if zero Dark 30 is correct , 1224 00:48:52,970 --> 00:48:56,760 What D . C . I . A , the director of C . 1225 00:48:56,760 --> 00:48:58,871 I . A . Panetta was constantly asking 1226 00:48:58,871 --> 00:49:01,440 us , how sure are we that he's in the 1227 00:49:01,440 --> 00:49:04,880 compound , you know , before we execute 1228 00:49:04,880 --> 00:49:07,070 a raid in another sovereign country ? 1229 00:49:07,080 --> 00:49:10,780 How sure are we ? Well , I just go 1230 00:49:10,780 --> 00:49:14,420 to the shoot down of the Ukrainian 1231 00:49:14,420 --> 00:49:17,770 airliner and we knew the Russians did 1232 00:49:17,770 --> 00:49:19,548 it immediately through national 1233 00:49:19,548 --> 00:49:21,770 technical means and other other stuff . 1234 00:49:21,870 --> 00:49:23,814 But we didn't want to release that 1235 00:49:23,814 --> 00:49:26,490 because of sources and methods . There 1236 00:49:26,490 --> 00:49:28,900 was a company called Bell and Cat who 1237 00:49:28,900 --> 00:49:31,122 essentially put together the storyboard 1238 00:49:31,122 --> 00:49:33,330 for the entire shoot down using social 1239 00:49:33,330 --> 00:49:35,920 media . You know , they had a picture 1240 00:49:35,930 --> 00:49:39,900 of a tell with three surface 1241 00:49:39,900 --> 00:49:42,122 to air missiles on it . A picture of it 1242 00:49:42,122 --> 00:49:45,890 crossing the border , uh , into eastern 1243 00:49:45,890 --> 00:49:48,510 Ukraine with the serial number on the 1244 00:49:48,510 --> 00:49:51,340 side . They had another picture of a 1245 00:49:51,350 --> 00:49:54,070 missile contrail right next to the 1246 00:49:54,080 --> 00:49:56,590 village where the shoot down occurred . 1247 00:49:56,970 --> 00:49:59,960 Had another picture of the same tell 1248 00:49:59,970 --> 00:50:02,700 with the same serial number going back 1249 00:50:02,700 --> 00:50:04,811 into Russia with two instead of three 1250 00:50:04,811 --> 00:50:06,867 missiles . They put together a story 1251 00:50:06,867 --> 00:50:09,180 board just using social media . It was 1252 00:50:09,180 --> 00:50:12,890 100%, , you know , any 1253 00:50:13,460 --> 00:50:16,040 objective person would say , whoa . The 1254 00:50:16,040 --> 00:50:18,040 Russians really did shoot down that 1255 00:50:18,040 --> 00:50:20,750 airliner . And had we had the 1256 00:50:20,750 --> 00:50:23,490 capability we have now to go through 1257 00:50:23,500 --> 00:50:26,280 all sorts of data . 1258 00:50:26,760 --> 00:50:29,220 Uh then I think the analysts would have 1259 00:50:29,220 --> 00:50:31,442 been able to tell director Panetta , we 1260 00:50:31,442 --> 00:50:35,330 are 100 certain that Bin laden is in 1261 00:50:35,330 --> 00:50:37,950 that compound and here's all of the 1262 00:50:37,950 --> 00:50:41,050 data that we can show you . And then 1263 00:50:41,050 --> 00:50:43,290 predictive analysis like Lieutenant 1264 00:50:43,290 --> 00:50:46,280 General Grown said the president might 1265 00:50:46,280 --> 00:50:48,790 have asked , what do we expect to be 1266 00:50:48,790 --> 00:50:52,070 the reaction of uh the muslim 1267 00:50:52,070 --> 00:50:54,300 community if it becomes aware that we 1268 00:50:54,300 --> 00:50:56,680 execute a raid and we killed bin laden . 1269 00:50:57,160 --> 00:51:00,400 Uh AI is able to do that type of 1270 00:51:00,400 --> 00:51:03,820 predictive . I I n w we're doing it 1271 00:51:03,820 --> 00:51:06,580 right now uh in Afghanistan 1272 00:51:07,060 --> 00:51:09,760 using a i to predict when the attacks 1273 00:51:09,760 --> 00:51:12,740 might occur or predict um you know , 1274 00:51:12,750 --> 00:51:15,680 actions by our adversaries . 1275 00:51:16,360 --> 00:51:18,820 Um I don't think a I would have made 1276 00:51:18,820 --> 00:51:21,520 that much difference in the raid force 1277 00:51:21,520 --> 00:51:23,670 itself unless they had specific 1278 00:51:23,670 --> 00:51:26,690 applications that they needed uh 1279 00:51:27,860 --> 00:51:30,470 to say what is the most up to date 1280 00:51:30,470 --> 00:51:32,359 intelligence , you know , what is 1281 00:51:32,359 --> 00:51:34,414 happening ? Do we need to change our 1282 00:51:34,414 --> 00:51:36,950 plan , etcetera ? But to me , we 1283 00:51:36,950 --> 00:51:39,228 already have kind of an answer for you . 1284 00:51:39,460 --> 00:51:43,340 Uh AI gives you a tool uh that 1285 00:51:43,340 --> 00:51:46,030 we've never ever really had . one of 1286 00:51:46,030 --> 00:51:48,810 our commissioners Ken Ford refers to 1287 00:51:48,810 --> 00:51:51,910 this as a I gives commanders eyeglasses 1288 00:51:51,910 --> 00:51:54,860 for the mind and I thought it was such 1289 00:51:54,860 --> 00:51:58,650 a pithy observation . It helps look 1290 00:51:58,650 --> 00:52:00,872 through enormous amounts of data that a 1291 00:52:00,872 --> 00:52:02,428 human would be incapable of 1292 00:52:02,428 --> 00:52:05,100 interpreting , and the Ai is able to 1293 00:52:05,100 --> 00:52:07,200 find patterns , make inferences , 1294 00:52:07,210 --> 00:52:10,520 etcetera . So that's what we mean by 1295 00:52:10,520 --> 00:52:13,320 human machine collaboration . You let 1296 00:52:13,320 --> 00:52:15,550 the machine do all that hard number 1297 00:52:15,550 --> 00:52:17,870 crunching and stuff like that and you 1298 00:52:17,870 --> 00:52:19,648 leave the Commander , the human 1299 00:52:19,648 --> 00:52:21,703 Commander to exercise their creative 1300 00:52:21,703 --> 00:52:23,814 spirit and their initiative and their 1301 00:52:23,814 --> 00:52:25,981 understanding of the broader strategic 1302 00:52:25,981 --> 00:52:28,560 concept . Human machine collaboration 1303 00:52:29,150 --> 00:52:32,640 uh is a big , big deal in the future of 1304 00:52:32,640 --> 00:52:36,570 ai . Next question goes out to jasmine 1305 00:52:36,580 --> 00:52:40,030 from National Defense . I thank you so 1306 00:52:40,030 --> 00:52:42,410 much for doing this . Um My question 1307 00:52:42,410 --> 00:52:45,220 has to do with comments that the 1308 00:52:45,220 --> 00:52:47,220 chairman of the commission has made 1309 00:52:47,220 --> 00:52:49,730 before , eric Schmidt , he said um that 1310 00:52:49,730 --> 00:52:52,050 china is maybe two years behind the 1311 00:52:52,050 --> 00:52:54,560 United States , um the time general 1312 00:52:54,560 --> 00:52:56,727 growing , I was wondering if you agree 1313 00:52:56,727 --> 00:52:58,782 with that assessment or do you think 1314 00:52:58,782 --> 00:53:00,893 that it's a bit we have a bit more of 1315 00:53:00,893 --> 00:53:03,130 an advantage . Yeah , thanks Jasmine . 1316 00:53:03,130 --> 00:53:05,740 I I think I would echo what what 1317 00:53:05,750 --> 00:53:07,972 secretary work articulated before , you 1318 00:53:07,972 --> 00:53:10,420 know , trying to measure advantage in a 1319 00:53:10,420 --> 00:53:12,587 space like this is a very , you know , 1320 00:53:12,587 --> 00:53:14,531 is a very difficult undertaking or 1321 00:53:14,531 --> 00:53:16,753 undertaking . I think uh you know , you 1322 00:53:16,753 --> 00:53:19,080 can look at places where there is , 1323 00:53:19,090 --> 00:53:21,470 there's clear superiority on the US 1324 00:53:21,470 --> 00:53:23,680 side . I think like our academic 1325 00:53:23,680 --> 00:53:26,880 environment , I mean the United States 1326 00:53:26,880 --> 00:53:29,460 academic community is unsurpassed 1327 00:53:29,460 --> 00:53:31,570 globally . Uh you look at our small 1328 00:53:31,570 --> 00:53:33,570 innovative companies and the things 1329 00:53:33,570 --> 00:53:35,514 that they're working uh you know , 1330 00:53:35,514 --> 00:53:37,626 every almost every company these days 1331 00:53:37,626 --> 00:53:39,848 as an ai company and a lot of them have 1332 00:53:39,848 --> 00:53:41,570 really good vertical stovepipe 1333 00:53:41,570 --> 00:53:43,237 capability . So there's great 1334 00:53:43,237 --> 00:53:45,237 innovation that goes all across the 1335 00:53:45,237 --> 00:53:47,459 United States . Um you look at on the , 1336 00:53:47,459 --> 00:53:49,570 on the chinese side , I mean , you do 1337 00:53:49,570 --> 00:53:51,681 have the organizational efficiency of 1338 00:53:51,681 --> 00:53:54,180 autocracy and you have all of the , you 1339 00:53:54,180 --> 00:53:56,180 know , the moral impacts of that as 1340 00:53:56,180 --> 00:53:58,291 well . But , you know , I I think the 1341 00:53:58,291 --> 00:54:00,513 competition and if you really wanted to 1342 00:54:00,513 --> 00:54:02,736 simplify it might , you know , might be 1343 00:54:02,736 --> 00:54:05,330 as in a sense , um the the 1344 00:54:05,340 --> 00:54:07,410 organizational efficiency versus 1345 00:54:07,420 --> 00:54:10,060 innovation and innovation of efficiency . 1346 00:54:10,140 --> 00:54:11,751 And so when you look at that 1347 00:54:11,751 --> 00:54:13,696 competition from that from through 1348 00:54:13,696 --> 00:54:15,807 those two lenses , you really have to 1349 00:54:15,807 --> 00:54:17,862 pay attention to both . Right . It's 1350 00:54:17,862 --> 00:54:19,973 like how do we achieve organizational 1351 00:54:19,973 --> 00:54:22,029 efficiency in our efforts so that we 1352 00:54:22,029 --> 00:54:24,196 can compete , compete , keep pace with 1353 00:54:24,196 --> 00:54:26,307 with a bigger machine , but then also 1354 00:54:26,307 --> 00:54:28,529 how can we continue to innovate so that 1355 00:54:28,529 --> 00:54:28,360 we're not stuck in yesterday's 1356 00:54:28,360 --> 00:54:30,527 technology and we continue to push the 1357 00:54:30,527 --> 00:54:32,693 envelope . So it's a really hard thing 1358 00:54:32,693 --> 00:54:34,693 to measure . I think both countries 1359 00:54:34,693 --> 00:54:37,100 have uh have demonstrated significant 1360 00:54:37,110 --> 00:54:40,620 global capabilities and so we have to 1361 00:54:40,630 --> 00:54:44,380 be in this fight for sure . Yeah , I 1362 00:54:44,380 --> 00:54:46,730 mean , I agree . It's just really a 1363 00:54:46,730 --> 00:54:49,740 tough thing to kind of judge The way we 1364 00:54:49,740 --> 00:54:51,962 did it . As I explained earlier , as we 1365 00:54:51,962 --> 00:54:54,210 broke down the Ai stack into its six 1366 00:54:54,210 --> 00:54:56,860 components . We judge that were ahead , 1367 00:54:56,860 --> 00:54:59,082 slightly ahead or ahead in three of the 1368 00:54:59,082 --> 00:55:01,190 six and China is ahead or slightly 1369 00:55:01,190 --> 00:55:03,580 ahead in 3-6 . So it's a really , 1370 00:55:03,580 --> 00:55:06,520 really tight competition . We admitted 1371 00:55:06,520 --> 00:55:08,720 that the Chinese could probably catch 1372 00:55:08,720 --> 00:55:11,520 up with us in algorithms within 5-10 1373 00:55:11,520 --> 00:55:14,990 years . We also say that we're 100 1374 00:55:14,990 --> 00:55:17,770 miles away from becoming from being two 1375 00:55:17,770 --> 00:55:20,200 generations ahead in hardware to being 1376 00:55:20,200 --> 00:55:23,620 two generations behind if for example , 1377 00:55:23,620 --> 00:55:26,840 China sees Taiwan and the fabricating 1378 00:55:26,850 --> 00:55:28,910 the chip fabricating facilities that 1379 00:55:28,910 --> 00:55:31,160 are on Taiwan . So 1380 00:55:33,140 --> 00:55:36,660 eric Schmidt is been working in this 1381 00:55:36,670 --> 00:55:39,190 area for a long time and his judgment 1382 00:55:39,200 --> 00:55:41,760 is , look , I think we're about two 1383 00:55:41,760 --> 00:55:43,920 years ahead , but he will tell anyone 1384 00:55:43,920 --> 00:55:46,470 who listens the chinese are coming on 1385 00:55:46,470 --> 00:55:49,430 fast . Uh , you know , they're ahead in 1386 00:55:49,430 --> 00:55:52,380 some were ahead in some , we need to 1387 00:55:52,380 --> 00:55:54,047 take this company , take this 1388 00:55:54,047 --> 00:55:56,269 competition . Like a politician takes a 1389 00:55:56,269 --> 00:55:58,269 political race you have to run like 1390 00:55:58,269 --> 00:56:00,900 you're losing . Um , and so it's 1391 00:56:00,900 --> 00:56:03,850 important that we really gear up and go , 1392 00:56:05,830 --> 00:56:07,663 okay , we have time for one more 1393 00:56:07,663 --> 00:56:09,719 question that will go out to Jackson 1394 00:56:09,719 --> 00:56:11,830 from bed scoop . Go ahead , Jackson , 1395 00:56:11,830 --> 00:56:13,886 Thank you so much . I hope I have my 1396 00:56:13,886 --> 00:56:15,997 dates right here . But within general 1397 00:56:15,997 --> 00:56:18,052 growing , I believe we're six months 1398 00:56:18,052 --> 00:56:20,330 out from your announcement of jake 2.0 , 1399 00:56:20,330 --> 00:56:22,497 and shifting to be more of an enabling 1400 00:56:22,497 --> 00:56:24,663 force , hoping you can give us just an 1401 00:56:24,663 --> 00:56:26,886 update on how that changes going . Um , 1402 00:56:26,886 --> 00:56:29,108 and if I could ask specifically about , 1403 00:56:29,108 --> 00:56:31,230 are you now sending out officials to 1404 00:56:31,230 --> 00:56:33,730 kind of be liaisons to specific ai 1405 00:56:33,730 --> 00:56:36,020 offices across the force ? How is that 1406 00:56:36,020 --> 00:56:38,110 going ? Is there any tension with , 1407 00:56:38,110 --> 00:56:40,380 with Jacob maybe showing up and and 1408 00:56:40,380 --> 00:56:42,658 offering help ? How is that successful ? 1409 00:56:42,658 --> 00:56:45,290 Maybe ? How are things ? Uh are there 1410 00:56:45,290 --> 00:56:47,234 any things you might change in the 1411 00:56:47,234 --> 00:56:49,346 future ? And then if I could also ask 1412 00:56:49,346 --> 00:56:51,940 Mr work previously , you've said that 1413 00:56:51,950 --> 00:56:55,170 the jake should take a naval nuclear 1414 00:56:55,170 --> 00:56:58,140 reactor Rickover type strategy to being 1415 00:56:58,140 --> 00:57:01,150 kind of an ai coordination office . Do 1416 00:57:01,150 --> 00:57:03,290 you think that holds any tension 1417 00:57:03,290 --> 00:57:05,123 between the kind of 1000 flowers 1418 00:57:05,123 --> 00:57:07,290 blooming approach that's being taken ? 1419 00:57:07,290 --> 00:57:09,568 What is your current stance on on that ? 1420 00:57:09,568 --> 00:57:12,980 Thank you . So , so I'll start with 1421 00:57:12,990 --> 00:57:15,212 thanks Jackson , a great question . And 1422 00:57:15,212 --> 00:57:18,230 uh so so as we you I think accurately 1423 00:57:18,230 --> 00:57:20,286 described what we want to do in jake 1424 00:57:20,286 --> 00:57:22,330 2.0 , so we realized kind of our 1425 00:57:22,340 --> 00:57:25,180 initial business model uh wasn't 1426 00:57:25,180 --> 00:57:27,291 getting us where we needed to go , it 1427 00:57:27,291 --> 00:57:29,402 was not transformational enough . And 1428 00:57:29,402 --> 00:57:31,569 so we really started focusing on broad 1429 00:57:31,569 --> 00:57:33,569 enablement . And I think we've been 1430 00:57:33,569 --> 00:57:35,791 fairly successful in that space . We do 1431 00:57:35,791 --> 00:57:38,160 have a great outreach organizations . 1432 00:57:38,160 --> 00:57:40,730 We do pay keen attention to all of the 1433 00:57:40,730 --> 00:57:42,674 service developments and we try to 1434 00:57:42,674 --> 00:57:44,950 partner with all of them . Uh , we pay 1435 00:57:44,950 --> 00:57:47,006 keen attention to the demand signals 1436 00:57:47,006 --> 00:57:49,228 from the combatant commands and we want 1437 00:57:49,228 --> 00:57:51,339 to , we want to work with anybody who 1438 00:57:51,339 --> 00:57:53,770 is doing a I today . But here's how we 1439 00:57:53,770 --> 00:57:55,937 approach that problem set right ? Like 1440 00:57:55,937 --> 00:57:58,048 our first duty I think are one of the 1441 00:57:58,048 --> 00:58:00,740 things that we do well , um , is 1442 00:58:00,740 --> 00:58:02,907 measure our success and the success of 1443 00:58:02,907 --> 00:58:04,962 others . And the second thing that I 1444 00:58:04,962 --> 00:58:07,020 think we do well is we don't go to 1445 00:58:07,020 --> 00:58:09,187 these organizations are partnered with 1446 00:58:09,187 --> 00:58:11,353 these organizations from a position of 1447 00:58:11,353 --> 00:58:14,160 teacher student . We come in as , as 1448 00:58:14,170 --> 00:58:16,420 archivist of best practice across the 1449 00:58:16,420 --> 00:58:18,930 department and and say , hey , show us 1450 00:58:18,930 --> 00:58:20,986 how you're doing that . Let us learn 1451 00:58:20,986 --> 00:58:23,360 from you and then we can share , hey , 1452 00:58:23,360 --> 00:58:25,416 you know , there's another agency in 1453 00:58:25,416 --> 00:58:27,582 the department that has a problem very 1454 00:58:27,582 --> 00:58:29,804 similar to yours and here's how they're 1455 00:58:29,804 --> 00:58:32,027 addressing that . So we play broker for 1456 00:58:32,027 --> 00:58:34,193 information and expertise across , you 1457 00:58:34,193 --> 00:58:36,249 know , across agencies , across uh , 1458 00:58:36,249 --> 00:58:38,304 services , across command commands . 1459 00:58:38,304 --> 00:58:40,360 And then what we can do is then turn 1460 00:58:40,360 --> 00:58:42,527 that into because of our congressional 1461 00:58:42,527 --> 00:58:44,790 congressional authority now to do our 1462 00:58:44,790 --> 00:58:46,970 own acquisition , for example , now we 1463 00:58:46,970 --> 00:58:48,980 can actually start providing a much 1464 00:58:48,980 --> 00:58:51,250 broader array of , of support , service 1465 00:58:51,250 --> 00:58:53,750 and services and enabling services that 1466 00:58:53,750 --> 00:58:55,583 help make all of those customers 1467 00:58:55,583 --> 00:58:57,806 successful . We think we're a force for 1468 00:58:57,806 --> 00:59:00,980 good here . We , we we approach the 1469 00:59:00,980 --> 00:59:03,840 challenge with humility and we are , we 1470 00:59:03,840 --> 00:59:06,007 measure our success and the success of 1471 00:59:06,007 --> 00:59:08,062 others . And so that has gotten us a 1472 00:59:08,062 --> 00:59:10,490 long way . I will say this . Um , as I 1473 00:59:10,490 --> 00:59:12,546 look at the challenge that secretary 1474 00:59:12,546 --> 00:59:14,768 work has laid out so effectively , um , 1475 00:59:14,768 --> 00:59:18,230 even now , I wonder is that is jake two 1476 00:59:18,230 --> 00:59:20,452 point oh enough . Right . Are we moving 1477 00:59:20,452 --> 00:59:22,674 fast enough ? Are we moving fast enough 1478 00:59:22,674 --> 00:59:24,897 to create enterprises of capability and 1479 00:59:24,897 --> 00:59:27,008 overcome stovepipe developments ? Are 1480 00:59:27,008 --> 00:59:29,320 we moving fast enough to really change 1481 00:59:29,330 --> 00:59:32,200 our operating model to data driven uh 1482 00:59:32,200 --> 00:59:34,000 and data visibility across the 1483 00:59:34,000 --> 00:59:36,167 department ? Are we moving fast enough 1484 00:59:36,167 --> 00:59:38,310 in integrating innovative technology 1485 00:59:38,310 --> 00:59:40,570 into the department ? And uh sometimes 1486 00:59:40,570 --> 00:59:42,626 I lay out , I lay awake at night and 1487 00:59:42,626 --> 00:59:44,800 saying that the answer is no . And uh 1488 00:59:44,810 --> 00:59:47,220 that challenge and feeling the hot 1489 00:59:47,220 --> 00:59:49,387 breath on the back of our next is what 1490 00:59:49,387 --> 00:59:51,498 keeps the jake motivated and keeps us 1491 00:59:51,498 --> 00:59:53,387 working hard every day because we 1492 00:59:53,387 --> 00:59:55,609 recognize how big this is and the scale 1493 00:59:55,609 --> 00:59:57,664 of the Department of Defense and how 1494 00:59:57,664 --> 00:59:59,664 necessary this transformation is at 1495 00:59:59,664 --> 01:00:01,553 scale . Thanks for the question , 1496 01:00:01,553 --> 01:00:03,776 That's great Jackson . You know , every 1497 01:00:03,776 --> 01:00:05,609 now and then somebody asked me a 1498 01:00:05,609 --> 01:00:07,776 question like yours and that gay God , 1499 01:00:07,776 --> 01:00:10,330 did I really say that ? Uh but uh at 1500 01:00:10,330 --> 01:00:12,552 the time , what I was saying is , do we 1501 01:00:12,552 --> 01:00:14,497 really believe that we're going to 1502 01:00:14,497 --> 01:00:16,274 build the department around the 1503 01:00:16,274 --> 01:00:18,260 capabilities of AI and Ai enabled 1504 01:00:18,260 --> 01:00:21,630 autonomy and nuclear reactors made the 1505 01:00:21,640 --> 01:00:23,751 you're gonna build a submarine around 1506 01:00:23,751 --> 01:00:25,918 the reactor . And you're going to have 1507 01:00:25,918 --> 01:00:27,807 to have the people who understand 1508 01:00:27,807 --> 01:00:30,029 everything about how that reactor works 1509 01:00:30,029 --> 01:00:31,973 and how it interfaces with all the 1510 01:00:31,973 --> 01:00:34,196 other systems on the submarines . We're 1511 01:00:34,196 --> 01:00:36,196 going to make sure that we pick the 1512 01:00:36,196 --> 01:00:39,770 people who are in charge . Uh We're 1513 01:00:39,770 --> 01:00:42,910 gonna set the standards . No one no one 1514 01:00:42,910 --> 01:00:45,132 can touch the standards except for us . 1515 01:00:45,410 --> 01:00:47,800 And so at the time , I was saying , you 1516 01:00:47,800 --> 01:00:49,911 know , there's a lot of advantages of 1517 01:00:49,911 --> 01:00:51,967 this . But over the last two years , 1518 01:00:51,967 --> 01:00:54,530 working with 14 other brilliant 1519 01:00:54,530 --> 01:00:57,420 commissioners , the recommendations 1520 01:00:57,420 --> 01:00:59,420 that we put into the commission I'm 1521 01:00:59,420 --> 01:01:03,370 fully behind . Uh and I personally 1522 01:01:03,370 --> 01:01:06,630 think that if you use 1523 01:01:07,110 --> 01:01:09,240 well , I'll just lay my cards on the 1524 01:01:09,240 --> 01:01:11,330 table . We thought about this as a 1525 01:01:11,330 --> 01:01:13,840 blueprint . We said , look , you really 1526 01:01:13,840 --> 01:01:15,451 shouldn't look at all of our 1527 01:01:15,451 --> 01:01:17,618 recommendations and say I kind of like 1528 01:01:17,618 --> 01:01:19,896 that one . I'll pull that off the wall . 1529 01:01:19,896 --> 01:01:22,118 You have to do them all together to get 1530 01:01:22,118 --> 01:01:25,330 the effect that the commission feels is 1531 01:01:25,340 --> 01:01:28,350 important . So right now , I would say 1532 01:01:28,350 --> 01:01:30,461 I've changed from the nuclear reactor 1533 01:01:30,461 --> 01:01:32,520 model to the National Commission on 1534 01:01:32,530 --> 01:01:35,520 Artificial Intelligence model . And I 1535 01:01:35,520 --> 01:01:37,687 would just like to say thank you again 1536 01:01:37,687 --> 01:01:39,798 for all of the people who listened in 1537 01:01:40,010 --> 01:01:42,660 The report is voluminous . You know , 1538 01:01:42,660 --> 01:01:46,310 it's over 760 pages . But our staff 1539 01:01:46,310 --> 01:01:50,140 which is like a world class staff 1540 01:01:50,610 --> 01:01:52,840 uh did everything they could for it to 1541 01:01:52,840 --> 01:01:55,062 be interactive for you to be able to go 1542 01:01:55,062 --> 01:01:57,062 into that final report and find the 1543 01:01:57,062 --> 01:01:59,284 information that you would like . There 1544 01:01:59,284 --> 01:02:01,396 are so many recommendations . This is 1545 01:02:01,396 --> 01:02:03,396 why I have so much paper . I mean I 1546 01:02:03,396 --> 01:02:05,118 can't keep track of all of the 1547 01:02:05,118 --> 01:02:07,284 recommendations in the report . I need 1548 01:02:07,284 --> 01:02:09,340 to be reminded of them , but I would 1549 01:02:09,710 --> 01:02:11,940 ask all of you to read the report 1550 01:02:11,950 --> 01:02:14,117 because we feel it is so important for 1551 01:02:14,117 --> 01:02:15,950 economic competitiveness and our 1552 01:02:15,950 --> 01:02:18,006 military competitiveness . I want to 1553 01:02:18,006 --> 01:02:20,250 thank you for hosting us today and 1554 01:02:20,250 --> 01:02:23,370 allowing us to kind of pitch our 1555 01:02:23,370 --> 01:02:26,300 product . Thank you for the 760 page to 1556 01:02:26,300 --> 01:02:30,050 do list sir . Mhm . I'm 1557 01:02:30,050 --> 01:02:31,994 afraid we're out of time . But for 1558 01:02:31,994 --> 01:02:34,217 those of you that we didn't get to with 1559 01:02:34,217 --> 01:02:35,939 questions , please submit your 1560 01:02:35,939 --> 01:02:38,250 questions to osd public affairs and we 1561 01:02:38,250 --> 01:02:40,460 can answer those so thanks everyone on 1562 01:02:40,460 --> 01:02:42,680 the lines and everybody here today for 1563 01:02:42,680 --> 01:02:45,030 attending and thank you very much . 1564 01:02:46,710 --> 01:02:46,940 Mhm .