1
00:00:00,100 --> 00:00:02,480
Hi and welcome to the Neil Ashton podcast.

2
00:00:03,019 --> 00:00:03,839
In each episode,

3
00:00:03,849 --> 00:00:07,309
we explained some of the fascinating ways that science and engineering

4
00:00:07,469 --> 00:00:09,159
are changing the world around us.

5
00:00:09,699 --> 00:00:14,170
We talk to leading engineers from elite level sports like cycling and Formula One

6
00:00:14,760 --> 00:00:18,670
to some of the world's top academics to understand how fluid dynamics,

7
00:00:18,680 --> 00:00:21,069
machine learning and supercomputing

8
00:00:21,229 --> 00:00:22,729
are bringing in a new era of discovery.

9
00:00:23,790 --> 00:00:27,149
We also hear some of their life stories, their career advice

10
00:00:27,540 --> 00:00:31,590
and lessons they've learned on the way that I hope will be helpful to you too.

11
00:00:31,959 --> 00:00:34,330
So sit back and enjoy this episode.

12
00:00:39,610 --> 00:00:41,849
Hi and welcome back to the Neil Ashton podcast.

13
00:00:42,189 --> 00:00:44,840
So today's guest is Professor Dore

14
00:00:45,069 --> 00:00:48,720
Zamy who is a professor at the University of Michigan and is

15
00:00:48,729 --> 00:00:51,290
also the director of the Michigan

16
00:00:51,299 --> 00:00:54,229
Institute for Computational Discovery and Engineering.

17
00:00:54,909 --> 00:01:02,090
And he has become actually one of the key voices in this area of AI for science coming

18
00:01:02,220 --> 00:01:07,190
from the fluid dynamics background. In fact, actually, we we discussed that

19
00:01:07,389 --> 00:01:10,650
many of the people moving and being pioneers in A

20
00:01:10,660 --> 00:01:14,250
I for science which covers of course science in general,

21
00:01:14,260 --> 00:01:17,980
actually coming from a fluid dynamics uh background and he is probably

22
00:01:18,430 --> 00:01:23,690
to some best um best known for his work actually in turbulence modelling

23
00:01:24,069 --> 00:01:29,480
uh in the sense of the seminal paper that he did turbulence modelling in the age of data

24
00:01:30,180 --> 00:01:30,599
that

25
00:01:30,739 --> 00:01:33,290
back in 2019, although I,

26
00:01:33,459 --> 00:01:37,239
you know, we discussed, he actually started that work 2012, 2014,

27
00:01:37,250 --> 00:01:38,930
which was well before

28
00:01:39,080 --> 00:01:42,900
the current uh hype and, and rise of machine learning and AI

29
00:01:43,230 --> 00:01:45,529
and that paper. And that work really,

30
00:01:45,940 --> 00:01:46,559
um

31
00:01:46,809 --> 00:01:52,080
I would say motivated the whole community to, to really look at machine learning.

32
00:01:52,430 --> 00:01:54,529
And he's progressed from that point

33
00:01:54,750 --> 00:01:59,629
to, to actually broaden out um not only into more general fluid dynamics,

34
00:01:59,639 --> 00:02:01,940
but also covering, you know, the sciences.

35
00:02:02,199 --> 00:02:05,540
Uh and, and really focusing in the role of machine learning and AI.

36
00:02:06,099 --> 00:02:06,489
In fact,

37
00:02:06,500 --> 00:02:09,220
one of the uh Pacific areas that he has really

38
00:02:09,229 --> 00:02:13,000
tried to push in recent years and bring the community together

39
00:02:13,220 --> 00:02:16,750
is around uh foundational models for science.

40
00:02:16,759 --> 00:02:19,500
Uh certainly a hot topic and something of, you know,

41
00:02:19,509 --> 00:02:21,250
potentially huge significance.

42
00:02:21,759 --> 00:02:26,940
So in this episode, we, we mainly focus on these debates around AI for science

43
00:02:27,070 --> 00:02:30,169
and we do limit ourselves um a little bit into

44
00:02:30,179 --> 00:02:34,050
the fluid dynamics and computational fluids um fluid dynamics domain,

45
00:02:35,000 --> 00:02:38,149
but really trying to get a sense of where this field is going.

46
00:02:38,160 --> 00:02:39,690
And here's some of his thoughts

47
00:02:39,979 --> 00:02:42,600
uh starting in the turbulence modelling side.

48
00:02:42,610 --> 00:02:45,050
So where does he see the current progress when

49
00:02:45,059 --> 00:02:47,059
it comes to using machine learning in that side.

50
00:02:47,309 --> 00:02:49,009
And he's, you know, careful to,

51
00:02:49,940 --> 00:02:52,119
to remind everybody that he,

52
00:02:52,130 --> 00:02:54,960
he's not trying to say that this replaces the need for humans.

53
00:02:54,970 --> 00:02:59,169
It's just an additional tool that they can use in the development of these methods.

54
00:02:59,500 --> 00:03:00,059
And then we,

55
00:03:00,070 --> 00:03:02,119
we broaden the discussion a little bit to go

56
00:03:02,130 --> 00:03:04,800
into more surrogate modeling and his thoughts around,

57
00:03:04,809 --> 00:03:06,630
you know, PINNs—physics-informed neural networks,

58
00:03:07,380 --> 00:03:10,279
how he sees their potential to replace

59
00:03:10,449 --> 00:03:13,460
or does he see their potential to replace PDEs

60
00:03:13,600 --> 00:03:14,220
solvers?

61
00:03:14,520 --> 00:03:15,070
Um

62
00:03:15,190 --> 00:03:16,839
And then we really get into a,

63
00:03:16,850 --> 00:03:19,300
a reasonably detailed discussion on foundational models.

64
00:03:19,309 --> 00:03:21,220
What he believes foundational

65
00:03:21,360 --> 00:03:23,740
models are his thoughts on maybe even

66
00:03:23,750 --> 00:03:27,279
how large language models using transformer um

67
00:03:27,729 --> 00:03:31,630
architectures could still be used in many parts of scientific discovery.

68
00:03:31,850 --> 00:03:33,160
But using um

69
00:03:33,470 --> 00:03:36,229
you know, simulation tools as agents within that,

70
00:03:36,240 --> 00:03:38,660
I thought that was a really interesting part of the discussion we had,

71
00:03:39,389 --> 00:03:45,369
I also asked him about the role of academia and industry and start ups in this space.

72
00:03:45,839 --> 00:03:49,990
And we have a discussion around also some of his advice for,

73
00:03:50,000 --> 00:03:52,050
for students and aspiring academics and

74
00:03:52,259 --> 00:03:53,750
some of his opinions on,

75
00:03:53,759 --> 00:03:57,830
on his enjoyment of being in academia and the career path that that he's taken

76
00:03:58,679 --> 00:04:01,779
as we I say this in almost every single episode.

77
00:04:01,789 --> 00:04:04,440
Um Even though this, you know, we spoke for more than an hour,

78
00:04:04,630 --> 00:04:07,889
there were many topics we didn't really get to cover.

79
00:04:08,059 --> 00:04:10,960
And that's why I put some links uh in the chat.

80
00:04:10,970 --> 00:04:13,220
Certainly, if you're watching this on, on YouTube,

81
00:04:13,500 --> 00:04:15,970
uh if you're listening to this uh on Spotify, Apple,

82
00:04:15,979 --> 00:04:18,760
maybe you can go to the YouTube channel to see some of those links

83
00:04:19,190 --> 00:04:20,420
where I,

84
00:04:20,570 --> 00:04:23,149
some of the summer schools that he's organized and symposia,

85
00:04:23,320 --> 00:04:25,420
which had some really fantastic speakers.

86
00:04:25,700 --> 00:04:27,260
And also I think they have the,

87
00:04:27,269 --> 00:04:31,959
some of the slides and talks that he gave where he goes into even more detail into

88
00:04:31,970 --> 00:04:37,029
some of the specifics that perhaps we were ever able to get into in this episode.

89
00:04:37,040 --> 00:04:37,470
So,

90
00:04:37,649 --> 00:04:41,359
um I, yeah, this is uh along the theme of um

91
00:04:41,730 --> 00:04:45,269
as I mentioned, the, the the past episode we did and there's a few more coming

92
00:04:45,440 --> 00:04:49,440
that really focuses on this AI for science and, and he is really a key voice in

93
00:04:49,559 --> 00:04:52,790
this that I hope you'll learn a lot from. So please sit back

94
00:04:52,920 --> 00:04:55,119
and enjoy this episode with Professor Do

95
00:04:55,769 --> 00:04:55,779
Ay,

96
00:04:56,100 --> 00:04:59,929
the MICDE or the Michigan Institute for Computational Discovery and Engineering.

97
00:04:59,940 --> 00:05:00,390
So

98
00:05:00,670 --> 00:05:04,309
it's a institute across the entire um campus.

99
00:05:04,649 --> 00:05:07,109
There are more than 180 faculty members

100
00:05:07,829 --> 00:05:09,809
looking at all kinds of disciplines, you know,

101
00:05:09,820 --> 00:05:15,220
from computational medicine to biology, to uh astrophysics, to climate modeling.

102
00:05:15,600 --> 00:05:17,359
But of course, as a researcher,

103
00:05:17,529 --> 00:05:20,250
my uh applications tend to be

104
00:05:20,709 --> 00:05:23,920
somewhat close to fluids, but not

105
00:05:24,250 --> 00:05:26,690
in the sense of seven or eight years ago.

106
00:05:27,130 --> 00:05:27,809
Um

107
00:05:28,170 --> 00:05:33,480
So my work has maybe moved closer to computational science and AI,

108
00:05:34,290 --> 00:05:40,429
um, rather than fluid specific. But of course, many of my applications tend to be

109
00:05:40,640 --> 00:05:42,670
close to fluids but not exclusively.

110
00:05:43,470 --> 00:05:44,329
Ok. Yeah,

111
00:05:44,709 --> 00:05:45,000
I mean,

112
00:05:45,369 --> 00:05:48,619
that's, it's always nice, isn't it to expand a little bit?

113
00:05:48,630 --> 00:05:51,019
I guess there's a, there's a trade off, isn't there?

114
00:05:51,029 --> 00:05:53,140
Like, there's a lot of transferable knowledge

115
00:05:53,350 --> 00:05:57,329
from, I guess, fluid dynamics outside, which is useful,

116
00:05:57,670 --> 00:05:59,700
um, to bring, do you think that's

117
00:06:00,130 --> 00:06:01,040
important,

118
00:06:01,049 --> 00:06:04,459
particularly in the age of um AI

119
00:06:04,470 --> 00:06:07,750
where methods are coming from different disciplines?

120
00:06:08,040 --> 00:06:08,220
In fact,

121
00:06:09,279 --> 00:06:09,450
since

122
00:06:10,209 --> 00:06:10,529
we are on this

123
00:06:13,709 --> 00:06:16,609
uh few months ago, I kind of

124
00:06:16,970 --> 00:06:18,970
had a consortium of

125
00:06:19,549 --> 00:06:22,250
uh institute directors across various universities. There was

126
00:06:22,350 --> 00:06:22,489
Petros

127
00:06:22,609 --> 00:06:22,700
Koumoutsakos

128
00:06:23,179 --> 00:06:25,059
at Harvard, Karen Willcox at

129
00:06:25,410 --> 00:06:26,619
Texas, Gianluca

130
00:06:27,130 --> 00:06:28,079
Iaccarino at Stanford,

131
00:06:28,649 --> 00:06:29,929
Youssef Marzouk at MIT.

132
00:06:30,750 --> 00:06:34,029
And what is common to all of these people, fluids applications?

133
00:06:34,040 --> 00:06:35,390
It's kind of weird, right?

134
00:06:35,600 --> 00:06:38,459
You know, they are, the institutes are very broad in scope.

135
00:06:38,750 --> 00:06:40,369
But I think

136
00:06:40,820 --> 00:06:44,390
because of the scale of the problem and

137
00:06:44,799 --> 00:06:49,519
certain peculiarities about fluid dynamics, uh I think some of the

138
00:06:50,910 --> 00:06:53,679
knowledge and methods that we develop here

139
00:06:53,890 --> 00:06:55,140
tend to be

140
00:06:55,380 --> 00:06:57,450
asking questions that are very hard.

141
00:06:57,929 --> 00:07:02,170
And so I think assimilating other information or taking these to other methods,

142
00:07:03,160 --> 00:07:05,299
I think it's a bit of a bridge but of course,

143
00:07:05,459 --> 00:07:06,619
I could be biased.

144
00:07:07,329 --> 00:07:07,510
Um

145
00:07:08,940 --> 00:07:11,000
but it sounds like a good career advice for people.

146
00:07:11,010 --> 00:07:13,730
If you want to become the director of a top institute,

147
00:07:13,739 --> 00:07:15,309
go down the fluid dynamics route.

148
00:07:15,369 --> 00:07:16,750
Because statistically

149
00:07:17,670 --> 00:07:20,899
the other way to think about it is if you never

150
00:07:20,910 --> 00:07:24,269
want to solve the problem that you want to solve,

151
00:07:24,679 --> 00:07:27,690
you want to pick fluids and turbulence in particular.

152
00:07:27,989 --> 00:07:30,929
But you gain so many different skills in that

153
00:07:31,559 --> 00:07:34,369
frustrating process of not being able to solve anything.

154
00:07:34,679 --> 00:07:36,660
I'm being tongue in cheek here. But,

155
00:07:37,059 --> 00:07:41,600
but, but, but seriously, I do think there is something here, right? Because

156
00:07:42,170 --> 00:07:44,390
um you know, if you look at

157
00:07:44,519 --> 00:07:45,529
uh

158
00:07:45,709 --> 00:07:49,359
you know, the development of computational fluid dynamics,

159
00:07:49,890 --> 00:07:54,299
uh it has a very strong connection to numerical methods in general,

160
00:07:54,829 --> 00:07:55,320
right?

161
00:07:55,630 --> 00:07:56,339
And you can and

162
00:07:56,500 --> 00:07:59,940
many other disciplines, you can see like a diffusion of these ideas.

163
00:07:59,950 --> 00:08:01,299
But yes, there is certainly

164
00:08:02,040 --> 00:08:03,679
some of the problems are so hard,

165
00:08:03,869 --> 00:08:07,100
you know, and that is this quote. I don't know who said it.

166
00:08:08,019 --> 00:08:10,540
It said turbulence is the graveyard of all

167
00:08:10,549 --> 00:08:12,799
ideas that are successful in other fields.

168
00:08:13,950 --> 00:08:17,179
But you can also flip that and you can say

169
00:08:17,390 --> 00:08:21,230
many of the methods that we use to probe turbulence and modern it,

170
00:08:21,390 --> 00:08:24,940
you know, can have applications in many other disciplines because

171
00:08:25,149 --> 00:08:26,600
this is in a sense, a

172
00:08:26,880 --> 00:08:28,700
hard and intractable problem.

173
00:08:29,230 --> 00:08:29,869
But

174
00:08:30,059 --> 00:08:32,609
end of the day, I'm still an engineer, right? So

175
00:08:32,808 --> 00:08:35,159
we still can get very useful things out.

176
00:08:36,320 --> 00:08:36,340
Yeah,

177
00:08:36,950 --> 00:08:39,700
I I maybe want to start a little bit on um

178
00:08:40,400 --> 00:08:41,179
and it would be good to

179
00:08:41,549 --> 00:08:44,739
please correct me if I'm wrong. But I think one of the areas that

180
00:08:45,349 --> 00:08:46,010
um

181
00:08:46,359 --> 00:08:51,049
really brought you to the national or international stage

182
00:08:51,369 --> 00:08:53,289
was the pioneering work,

183
00:08:53,679 --> 00:08:54,140
the turbt

184
00:08:54,260 --> 00:08:57,630
modeling paper that you did looking at data driven turbulence modelling.

185
00:08:57,940 --> 00:09:00,460
Uh at least from my side, I suddenly

186
00:09:00,849 --> 00:09:04,140
I found those papers and those review papers

187
00:09:04,369 --> 00:09:08,349
incredibly useful. And I know many other colleagues, it sort of

188
00:09:08,989 --> 00:09:13,210
brought it to the attention to the, to the point I would say arguably now,

189
00:09:13,659 --> 00:09:15,049
so many

190
00:09:15,590 --> 00:09:18,130
people I know at least in the Fluids Domain

191
00:09:18,669 --> 00:09:21,940
have PhD students have projects and many of them are

192
00:09:21,950 --> 00:09:24,919
citing that sort of as an original piece of work.

193
00:09:24,929 --> 00:09:28,869
That sort of kick started. Uh a lot of this off uh turbots

194
00:09:29,020 --> 00:09:32,090
modeling in the age of data was that your,

195
00:09:32,580 --> 00:09:34,979
was that also your sort of first

196
00:09:35,260 --> 00:09:38,239
uh move into machine learning and, and

197
00:09:38,520 --> 00:09:43,549
sort of more data science from your prior background, maybe more as a pure

198
00:09:43,830 --> 00:09:44,320
uh

199
00:09:44,479 --> 00:09:47,380
quote unquote turbulence modeling or, or engineer.

200
00:09:48,280 --> 00:09:51,380
Yeah, I mean, uh yeah, there is, I mean, first of all,

201
00:09:51,390 --> 00:09:55,419
I that paper you referred to is he wrote it in 2018,

202
00:09:56,140 --> 00:09:59,679
that was already six years into the work that I've been doing in the field.

203
00:10:00,039 --> 00:10:01,359
But you're right, that is

204
00:10:01,510 --> 00:10:04,159
the first entry for me into

205
00:10:04,280 --> 00:10:06,549
data science, machine learning type of areas.

206
00:10:07,159 --> 00:10:13,919
Um And I I was never really a turbulence modeler as you or your adviser were. Um But uh

207
00:10:14,190 --> 00:10:18,440
it was certainly AAA good chunk of my thinking back in

208
00:10:19,080 --> 00:10:20,859
15 years ago or so.

209
00:10:21,719 --> 00:10:25,090
Um And, and the entry into the field was a bit interesting.

210
00:10:25,099 --> 00:10:28,349
So I was working on this idea called structure based turbulence modeling.

211
00:10:28,359 --> 00:10:29,539
It's very advanced.

212
00:10:29,929 --> 00:10:30,419
So

213
00:10:30,900 --> 00:10:37,530
think of writing equations for a three dimensional tensor. So 27 equations,

214
00:10:37,820 --> 00:10:40,419
of course, there symmetries and things you can eliminate a few.

215
00:10:41,119 --> 00:10:45,909
And there, even though we were going more and more fundamental into the theory,

216
00:10:46,780 --> 00:10:47,960
by the time you were

217
00:10:48,909 --> 00:10:49,710
developing a,

218
00:10:50,080 --> 00:10:50,090
a

219
00:10:50,500 --> 00:10:52,650
true turbulence model,

220
00:10:53,169 --> 00:10:55,650
there are just so many uncertainties, so many

221
00:10:55,820 --> 00:10:59,190
functions that we had to make up so many coefficients we had to fit.

222
00:10:59,770 --> 00:11:01,969
So we said, yeah, this is probably

223
00:11:02,390 --> 00:11:03,609
a task that

224
00:11:04,000 --> 00:11:05,479
data science can address.

225
00:11:06,229 --> 00:11:10,130
And of course, at that time, big data was

226
00:11:10,239 --> 00:11:11,770
everywhere. This is,

227
00:11:12,390 --> 00:11:13,609
this is the world of,

228
00:11:14,109 --> 00:11:18,489
you know, Walmart saying big data is going to be the next big thing kind of thing.

229
00:11:19,030 --> 00:11:21,919
So yeah, that's how I entered the field. You're right in that sense.

230
00:11:22,710 --> 00:11:23,820
Um Yeah, I

231
00:11:24,820 --> 00:11:26,330
I started this in

232
00:11:26,520 --> 00:11:29,840
this 2012, 2013 time frame

233
00:11:30,330 --> 00:11:32,119
and I think for a few years,

234
00:11:32,669 --> 00:11:34,760
there was literally nobody at least in

235
00:11:35,840 --> 00:11:37,630
turbulence and CFD doing that.

236
00:11:37,979 --> 00:11:38,830
Uh at least,

237
00:11:39,559 --> 00:11:41,840
yeah, for these kind of problems and then, yeah, the field kind of

238
00:11:41,940 --> 00:11:42,349
took off.

239
00:11:45,739 --> 00:11:47,700
Yeah, that's kind of incredible in a way.

240
00:11:47,710 --> 00:11:50,919
Now when you look back, it's hard to split the now to then

241
00:11:51,109 --> 00:11:53,280
because now machine learning AI is literally

242
00:11:53,289 --> 00:11:55,460
everywhere you you can't even watch the television,

243
00:11:55,469 --> 00:11:57,900
listen to anything without being on

244
00:11:58,119 --> 00:12:00,539
where probably in 2012, 2013,

245
00:12:00,679 --> 00:12:03,500
was it even a struggle to get funding with the

246
00:12:03,510 --> 00:12:06,820
funding agencies not even fully appreciating it or was it,

247
00:12:06,830 --> 00:12:08,739
as you say, more under a different banner?

248
00:12:08,750 --> 00:12:09,059
This

249
00:12:09,500 --> 00:12:10,460
big data

250
00:12:10,700 --> 00:12:11,000
um

251
00:12:11,260 --> 00:12:11,500
banner?

252
00:12:12,030 --> 00:12:16,599
No, I must count myself lucky because the very first proposal I wrote when I

253
00:12:16,739 --> 00:12:18,190
came to the University of Michigan,

254
00:12:18,859 --> 00:12:21,159
I think it was called turbulence modeling.

255
00:12:22,559 --> 00:12:26,380
I think it was something to do with big data and turbulence modelling in, in, in the title

256
00:12:27,299 --> 00:12:29,799
and they got funded. It was one of,

257
00:12:30,219 --> 00:12:31,190
I think they had,

258
00:12:32,109 --> 00:12:32,119
I

259
00:12:32,400 --> 00:12:33,260
submitted to

260
00:12:33,380 --> 00:12:35,869
NASA, I think they had about 90 proposals and this

261
00:12:36,280 --> 00:12:38,250
14 and haven't been one of them.

262
00:12:38,739 --> 00:12:40,770
So I was lucky in the sense that

263
00:12:41,520 --> 00:12:44,820
when I started, even before my ideas were fully formed,

264
00:12:45,500 --> 00:12:50,260
I just shot out a proposal and it got funded and I must say

265
00:12:50,799 --> 00:12:54,640
it has not been a struggle at any point to get funding because

266
00:12:55,530 --> 00:12:57,349
uh maybe

267
00:12:57,760 --> 00:13:01,710
I got the timing right or, or whatever I asked the right questions, I don't know.

268
00:13:02,510 --> 00:13:03,489
But uh

269
00:13:03,859 --> 00:13:05,239
being ahead of

270
00:13:06,010 --> 00:13:09,200
the curve in terms of, you know, what needs to be done or what

271
00:13:09,440 --> 00:13:10,640
needs to be looked at,

272
00:13:11,090 --> 00:13:11,799
I think helps.

273
00:13:12,340 --> 00:13:13,469
But that doesn't mean

274
00:13:14,030 --> 00:13:17,719
the problem is anywhere close to being solved as I mentioned earlier.

275
00:13:18,200 --> 00:13:19,119
But yeah, it has been,

276
00:13:19,289 --> 00:13:20,429
yeah, that was my entry point.

277
00:13:21,099 --> 00:13:22,849
And yeah, it's been good to be

278
00:13:23,619 --> 00:13:24,479
um

279
00:13:24,750 --> 00:13:27,280
close to the center of the field and

280
00:13:27,390 --> 00:13:28,950
see what people have been doing.

281
00:13:29,859 --> 00:13:32,179
So maybe for people

282
00:13:32,799 --> 00:13:36,330
who are not as familiar, how would you describe

283
00:13:37,580 --> 00:13:41,179
the state as it is now? I mean, it essentially,

284
00:13:42,190 --> 00:13:45,349
I think still it's quite a ferocious debate depending

285
00:13:45,359 --> 00:13:47,190
on how close people are to the research.

286
00:13:47,200 --> 00:13:50,409
Perhaps some on one side of the coin being

287
00:13:51,349 --> 00:13:52,440
almost anti

288
00:13:52,599 --> 00:13:55,130
machine learning anti this sense

289
00:13:55,640 --> 00:13:58,770
that it's just all hype and, and why we've been doing this

290
00:13:58,940 --> 00:14:00,090
to the other side

291
00:14:00,190 --> 00:14:02,330
that really do see it as being,

292
00:14:02,440 --> 00:14:03,409
you know, the next

293
00:14:03,929 --> 00:14:06,299
generation of, of methods from,

294
00:14:06,460 --> 00:14:08,539
I know we'll maybe expand this to

295
00:14:08,549 --> 00:14:11,090
a broader debate around scientific foundational models.

296
00:14:11,099 --> 00:14:12,489
But on the turbulence modelling side,

297
00:14:12,619 --> 00:14:15,369
how would you summarize how it is today?

298
00:14:16,729 --> 00:14:21,010
Yeah. So, I mean, I've been saying this almost, I mean, first of all, I think

299
00:14:21,559 --> 00:14:23,099
you're right, that has been

300
00:14:24,239 --> 00:14:25,539
ferocious debate

301
00:14:25,830 --> 00:14:27,099
right from the beginning.

302
00:14:27,979 --> 00:14:31,280
And uh I don't know which side is louder

303
00:14:31,549 --> 00:14:32,270
right now.

304
00:14:32,669 --> 00:14:33,239
Um uh

305
00:14:33,710 --> 00:14:35,250
But, but I think

306
00:14:35,390 --> 00:14:38,280
uh if you want me to summarize the state of the field,

307
00:14:39,010 --> 00:14:41,840
I want to like separate a few things here, right? One

308
00:14:43,510 --> 00:14:48,570
early on especially and maybe even recently when somebody publishes a paper

309
00:14:48,719 --> 00:14:50,390
saying, hey, here is a method,

310
00:14:50,900 --> 00:14:53,150
right? So from the viewpoint of the person

311
00:14:53,359 --> 00:14:54,330
who publishes it,

312
00:14:54,960 --> 00:14:57,900
the viewpoint of the turbulence model, they're looking for a model,

313
00:14:57,909 --> 00:14:59,700
they're not looking for a method, right?

314
00:14:59,940 --> 00:15:02,739
So that is that disconnect, um which

315
00:15:03,270 --> 00:15:07,849
I think can be bridged better by being very clear about what the objective is,

316
00:15:08,099 --> 00:15:08,500
right?

317
00:15:09,150 --> 00:15:11,169
Um Yeah, but beyond that,

318
00:15:11,489 --> 00:15:12,849
I've been saying this

319
00:15:13,369 --> 00:15:15,799
all the way from 2013 now,

320
00:15:16,320 --> 00:15:19,809
uh maybe as a way to, you know, hedge,

321
00:15:19,969 --> 00:15:22,469
I don't know what about why I started

322
00:15:22,890 --> 00:15:26,010
presenting my work this way. So I would go through this

323
00:15:26,119 --> 00:15:27,760
hour long talk on

324
00:15:28,210 --> 00:15:29,880
data driven turbulence modeling.

325
00:15:30,159 --> 00:15:31,679
And then at the end, I would say

326
00:15:32,250 --> 00:15:33,130
I have not

327
00:15:34,190 --> 00:15:37,369
presented to you a new approach to turbulence modelling.

328
00:15:38,510 --> 00:15:40,960
I'm providing a new tool

329
00:15:41,320 --> 00:15:42,929
that you can use, right?

330
00:15:43,119 --> 00:15:44,609
And from that perspective,

331
00:15:44,890 --> 00:15:47,349
you know, what have we do? What have we been,

332
00:15:47,739 --> 00:15:50,940
what have we been doing as a community from 1960

333
00:15:51,320 --> 00:15:54,289
right? We have some ideas, we put together

334
00:15:55,119 --> 00:15:58,659
a model and then we leave some coefficients to be calibrated.

335
00:15:59,250 --> 00:15:59,659
And

336
00:16:00,549 --> 00:16:02,469
in the way we make up some functions

337
00:16:02,900 --> 00:16:04,380
right along the way

338
00:16:04,750 --> 00:16:05,559
and then

339
00:16:05,679 --> 00:16:10,030
using some very few data points like the log layer and

340
00:16:10,130 --> 00:16:14,349
jets and things like that we calibrate coefficients or constants.

341
00:16:15,840 --> 00:16:16,419
Um

342
00:16:17,090 --> 00:16:18,989
So at least my view has always been

343
00:16:19,630 --> 00:16:21,690
do not abandon that by any means,

344
00:16:22,729 --> 00:16:24,440
but use these new tools,

345
00:16:25,020 --> 00:16:26,260
right, which can

346
00:16:26,510 --> 00:16:28,070
potentially help you

347
00:16:28,400 --> 00:16:29,559
uh in just

348
00:16:29,700 --> 00:16:31,190
more information

349
00:16:32,229 --> 00:16:35,989
and uh maybe do things in a, in a better way. But that doesn't mean

350
00:16:36,679 --> 00:16:41,140
again, I do not want the focus to be on the machine learning. In fact,

351
00:16:41,719 --> 00:16:46,500
I organized a very successful conference in Ann Arbor about 2017

352
00:16:46,969 --> 00:16:48,260
and I was on a war path.

353
00:16:49,080 --> 00:16:51,219
It was on data driven turbulence modeling.

354
00:16:51,229 --> 00:16:53,630
I said we should never use machine learning

355
00:16:53,820 --> 00:16:55,659
in the abstract or in the title

356
00:16:56,140 --> 00:16:58,739
because that is one piece, right?

357
00:16:58,979 --> 00:17:01,609
The modeling is the most important piece

358
00:17:01,940 --> 00:17:02,619
and then

359
00:17:02,849 --> 00:17:04,900
the data is another piece,

360
00:17:05,329 --> 00:17:08,790
right? So the machine learning is a little tool in this process.

361
00:17:09,209 --> 00:17:09,699
And

362
00:17:09,930 --> 00:17:11,358
even without using it,

363
00:17:11,630 --> 00:17:14,540
there are many ways to like massage the data, right,

364
00:17:14,550 --> 00:17:16,250
to get you the information you need.

365
00:17:16,260 --> 00:17:16,579
So,

366
00:17:17,108 --> 00:17:22,729
so it is a tool and I do not think a machine learning first or an AI first approach,

367
00:17:23,170 --> 00:17:26,530
at least in turbulence mind, we can talk about other fields where

368
00:17:27,170 --> 00:17:31,689
purely data driven machine learning type of or AI approaches can

369
00:17:32,260 --> 00:17:33,689
take you almost all the way.

370
00:17:34,310 --> 00:17:37,310
But here it is a tool in the modeller's toolkit

371
00:17:37,750 --> 00:17:39,569
and to use it judiciously

372
00:17:40,020 --> 00:17:42,050
as you have been using it in the past,

373
00:17:42,520 --> 00:17:43,609
right? Without

374
00:17:43,729 --> 00:17:45,109
overfitting, et cetera.

375
00:17:45,760 --> 00:17:47,130
I think that's the way forward.

376
00:17:48,060 --> 00:17:50,219
It has always been the way forward. In my opinion.

377
00:17:50,400 --> 00:17:53,420
Even though I would publish a paper, I did not claim to

378
00:17:53,680 --> 00:17:56,939
say I have solved this problem. I said here is a method you can use

379
00:17:57,859 --> 00:17:58,959
with your own

380
00:17:59,439 --> 00:18:03,390
uh judicious interpretation of what needs to be done with this tool.

381
00:18:04,239 --> 00:18:04,770
Yeah.

382
00:18:05,290 --> 00:18:07,199
Yeah, I guess it's always um

383
00:18:07,699 --> 00:18:13,030
like with anything, there's always an overreaction to things sometimes where

384
00:18:13,380 --> 00:18:16,109
people. But I guess the turbulence modelling community in general

385
00:18:17,290 --> 00:18:18,099
has,

386
00:18:19,479 --> 00:18:22,790
has a large sort of quite conservative nature.

387
00:18:22,800 --> 00:18:27,189
I would argue as well, which probably makes it even harder for

388
00:18:27,599 --> 00:18:28,339
um

389
00:18:28,680 --> 00:18:32,410
newer methods to perhaps come in to, to, you know,

390
00:18:32,420 --> 00:18:34,800
to the phrase because people are used to doing a certain

391
00:18:35,040 --> 00:18:36,339
way. Um

392
00:18:37,709 --> 00:18:38,790
What, um

393
00:18:40,209 --> 00:18:41,469
and I guess this would also

394
00:18:41,920 --> 00:18:47,229
I'm not as familiar, but I would assume transition models as well are equally, I

395
00:18:47,420 --> 00:18:47,949
mean,

396
00:18:48,439 --> 00:18:52,550
maybe a slightly changing uh angle a little bit.

397
00:18:52,560 --> 00:18:57,290
I think one of the areas you also work on is some of the hypersonics work. And I believe

398
00:18:57,640 --> 00:19:01,349
transition modeling is one of the most complicated parts of that.

399
00:19:01,760 --> 00:19:04,160
Have you seen equally that,

400
00:19:04,550 --> 00:19:07,770
you know, for all these models of tons of coefficients and models that

401
00:19:07,979 --> 00:19:09,010
machine learning can,

402
00:19:09,020 --> 00:19:11,290
can play a part also in transition modeling that

403
00:19:11,300 --> 00:19:13,410
it's not just about turbulence modelling per se.

404
00:19:15,060 --> 00:19:17,760
Yeah, there's certainly quite a bit of work and

405
00:19:18,160 --> 00:19:20,150
I've always said this um

406
00:19:20,890 --> 00:19:26,020
that I think even though neither problem has been solved to an extent

407
00:19:26,229 --> 00:19:28,790
that, you know, we would be happy with the methods.

408
00:19:29,380 --> 00:19:32,099
I've always maintained that transition modeling may be

409
00:19:32,770 --> 00:19:35,589
more amenable to this data driven disciplines

410
00:19:36,319 --> 00:19:37,589
because

411
00:19:38,670 --> 00:19:39,760
at least there,

412
00:19:40,630 --> 00:19:41,689
there is a hope

413
00:19:42,189 --> 00:19:45,119
that you can cover many possible regimes using

414
00:19:45,229 --> 00:19:45,430
hyper

415
00:19:45,709 --> 00:19:46,810
data and experiments

416
00:19:46,969 --> 00:19:48,880
because the Reynolds numbers tend to be

417
00:19:49,150 --> 00:19:49,969
smaller

418
00:19:50,449 --> 00:19:51,400
by definition

419
00:19:51,550 --> 00:19:53,010
right than the other way.

420
00:19:53,319 --> 00:19:55,119
But of course, in some senses,

421
00:19:55,410 --> 00:19:56,239
the

422
00:19:56,770 --> 00:19:58,890
the quality of measurements

423
00:19:59,030 --> 00:20:00,849
has to be better. But

424
00:20:01,699 --> 00:20:03,439
I think one of especially when

425
00:20:03,790 --> 00:20:04,760
you know, the

426
00:20:05,079 --> 00:20:09,119
the field has become more model consistent, I can tell you more what that is

427
00:20:09,849 --> 00:20:11,270
the ability to work with

428
00:20:11,489 --> 00:20:12,270
um

429
00:20:12,540 --> 00:20:14,300
integral data like ski

430
00:20:14,540 --> 00:20:20,369
friction or heat transfer wall he flux can still drive these models. So,

431
00:20:20,790 --> 00:20:22,079
so I think in a sense,

432
00:20:22,310 --> 00:20:24,140
I mean, quite, I would say

433
00:20:25,819 --> 00:20:28,160
about 20% of the work may be

434
00:20:28,380 --> 00:20:31,359
moving towards transition these days. That was not the case earlier.

435
00:20:32,030 --> 00:20:33,770
And one of our first um

436
00:20:34,479 --> 00:20:38,689
applications in this field was bypass transition modeling

437
00:20:38,699 --> 00:20:41,699
or at least methods for bypass transition modeling

438
00:20:41,849 --> 00:20:42,270
to be more

439
00:20:42,410 --> 00:20:42,719
size.

440
00:20:43,250 --> 00:20:46,020
So yes, there is a lot happening in the field and

441
00:20:46,510 --> 00:20:49,140
the fact that it is possible to run

442
00:20:50,390 --> 00:20:51,530
uh DNS

443
00:20:52,660 --> 00:20:56,500
at four or at least close to the condition that we want.

444
00:20:57,250 --> 00:21:02,270
And that being a rich source of high quality data, I think is something that is

445
00:21:02,650 --> 00:21:03,670
very useful.

446
00:21:04,510 --> 00:21:06,280
So the turbulence modelling on one side.

447
00:21:06,290 --> 00:21:10,119
So you could argue you're still, as you said, maintaining, um

448
00:21:10,930 --> 00:21:12,949
you're still using a traditional solver,

449
00:21:13,099 --> 00:21:15,150
whether it's commercial solver, open source government

450
00:21:15,290 --> 00:21:17,839
and you're plugging in your tuning coefficient.

451
00:21:18,079 --> 00:21:22,900
Um One of the points that you brought up and seems to be strong interest is

452
00:21:23,709 --> 00:21:25,089
replacing

453
00:21:25,510 --> 00:21:26,209
all of it

454
00:21:27,209 --> 00:21:28,680
versus a surrogate model.

455
00:21:30,089 --> 00:21:31,459
What's your

456
00:21:31,829 --> 00:21:36,560
view on that is this, let's say if we take just for now, um

457
00:21:37,270 --> 00:21:40,609
like I say an external aerodynamic example, like like an aircraft,

458
00:21:41,430 --> 00:21:42,569
do you see that

459
00:21:42,729 --> 00:21:42,739
a

460
00:21:43,069 --> 00:21:45,979
turbulence modelling machine learning fix is the way

461
00:21:45,989 --> 00:21:48,130
to ultimately get more accuracy and speed and,

462
00:21:48,140 --> 00:21:50,040
and all the the merits that people discuss

463
00:21:50,349 --> 00:21:53,250
or do you see that actually a surrogate model

464
00:21:53,400 --> 00:21:59,229
is the sort of better route and better angle given the advances in machine learning?

465
00:22:00,160 --> 00:22:00,410
Yeah, I,

466
00:22:00,420 --> 00:22:03,510
I think it certainly depends on the application how

467
00:22:03,520 --> 00:22:06,430
that computational result is going to be processed.

468
00:22:07,219 --> 00:22:09,680
You know, if you're doing design exploration,

469
00:22:10,410 --> 00:22:14,089
I do feel that a surrogate model for quantities of interest

470
00:22:15,000 --> 00:22:16,099
can take you,

471
00:22:17,510 --> 00:22:19,650
you know, can give you a lot of benefits, right?

472
00:22:20,040 --> 00:22:21,219
But if you are

473
00:22:21,890 --> 00:22:24,979
going to be probing some details of the flow physics,

474
00:22:26,150 --> 00:22:26,680
um

475
00:22:26,859 --> 00:22:31,180
and you want it to be somewhat general, I do not think a surrogate model

476
00:22:31,489 --> 00:22:32,390
can get it

477
00:22:32,729 --> 00:22:33,310
done.

478
00:22:34,030 --> 00:22:37,050
Um unless you parameterize the problem

479
00:22:38,219 --> 00:22:42,359
and there are a few parameters and you hit parameter space properly and then

480
00:22:42,540 --> 00:22:45,229
you can crunch through some, some

481
00:22:45,339 --> 00:22:46,359
large data.

482
00:22:47,030 --> 00:22:51,410
Uh You can't quite do it. The reason tends to be following, right? Like

483
00:22:53,000 --> 00:22:56,060
whenever we think of a surrogate model, we think of a system,

484
00:22:56,859 --> 00:23:00,199
right? And you have a lot of data on that system and you want to be

485
00:23:00,640 --> 00:23:01,359
um

486
00:23:02,030 --> 00:23:02,939
developing

487
00:23:03,160 --> 00:23:05,209
some representation of that system.

488
00:23:06,319 --> 00:23:06,939
But

489
00:23:07,319 --> 00:23:11,140
the take the classical CFD turbulence modeling approach where

490
00:23:12,239 --> 00:23:13,300
you can

491
00:23:13,829 --> 00:23:15,020
at least in theory

492
00:23:15,380 --> 00:23:18,060
run a problem for which you have seen no data for,

493
00:23:18,979 --> 00:23:19,510
right,

494
00:23:20,069 --> 00:23:21,900
a different system from the

495
00:23:22,709 --> 00:23:27,810
systems or subsystems where you trained your model, right? So that's the whole

496
00:23:28,359 --> 00:23:29,849
discovery process where

497
00:23:30,890 --> 00:23:33,229
let's say you want to design a hypersonic aircraft,

498
00:23:33,430 --> 00:23:35,630
maybe I have data for the

499
00:23:36,140 --> 00:23:38,680
an inlet. I have data from some shock bound

500
00:23:38,920 --> 00:23:39,709
interaction.

501
00:23:40,040 --> 00:23:40,530
I have some

502
00:23:40,689 --> 00:23:40,869
eran

503
00:23:41,040 --> 00:23:41,579
data.

504
00:23:42,689 --> 00:23:47,510
The advantage of going through the classical CFD slash PDE route is you can

505
00:23:47,949 --> 00:23:51,670
in principle extract information from these different sub components.

506
00:23:53,280 --> 00:23:56,790
And if you do that properly, you can take that to this unseen problem,

507
00:23:57,270 --> 00:23:57,579
right?

508
00:23:57,589 --> 00:24:00,489
Whereas if you go with the surrogate, you do not have that flexibility,

509
00:24:00,500 --> 00:24:03,189
you have to have data from that system

510
00:24:03,640 --> 00:24:06,020
or a parameterized version of that system.

511
00:24:07,260 --> 00:24:09,579
And if it is a discovery task, then

512
00:24:09,790 --> 00:24:11,380
you cannot get off the ground,

513
00:24:11,650 --> 00:24:12,160
right.

514
00:24:12,520 --> 00:24:13,829
So that's why I think,

515
00:24:14,099 --> 00:24:18,599
you know, as I said earlier at uh uh machine learning is a is a tool of the modelers,

516
00:24:18,609 --> 00:24:19,250
toolkit.

517
00:24:19,469 --> 00:24:20,770
All of these are

518
00:24:20,949 --> 00:24:22,800
tools in the designer toolkit.

519
00:24:23,459 --> 00:24:24,839
So they help you

520
00:24:25,520 --> 00:24:29,300
address different parts of the design or discovery or analysis process.

521
00:24:29,510 --> 00:24:31,209
So yeah, both have a room

522
00:24:31,979 --> 00:24:32,640
and

523
00:24:33,199 --> 00:24:35,489
in certain circumstances,

524
00:24:36,640 --> 00:24:38,859
it makes a lot of sense

525
00:24:39,729 --> 00:24:40,579
to

526
00:24:40,689 --> 00:24:43,109
go with the surrogate model for quantities of interest.

527
00:24:43,849 --> 00:24:46,869
And in many cases, it makes no sense to do that right?

528
00:24:47,209 --> 00:24:52,199
And go with this other approach. So yes, both need both need to be like you would

529
00:24:53,050 --> 00:24:57,630
as a source of information to address different parts of the spectrum.

530
00:24:58,510 --> 00:25:00,150
But what about um

531
00:25:00,699 --> 00:25:04,849
physics-informed approaches, physics-driven approaches like, like PINNs that,

532
00:25:05,280 --> 00:25:09,140
uh depending on the paper you read and the person you speak to have a,

533
00:25:09,270 --> 00:25:14,729
a much bolder claim of, of ultimately being able to solve the PDEs and therefore

534
00:25:15,010 --> 00:25:16,739
potentially move away

535
00:25:16,910 --> 00:25:20,290
or combine those two together in the sense of being able to

536
00:25:20,560 --> 00:25:21,199
um

537
00:25:21,390 --> 00:25:24,239
not be relying just on the data that you've trained it on. Uh

538
00:25:24,530 --> 00:25:27,849
what's your uh opinion on those methods?

539
00:25:28,339 --> 00:25:32,880
Yeah. So I have worked on PINNs almost from

540
00:25:33,449 --> 00:25:34,979
the early days of PINNs.

541
00:25:35,109 --> 00:25:36,869
Obviously, I looked at

542
00:25:37,599 --> 00:25:38,619
uh George

543
00:25:39,079 --> 00:25:42,020
Karniadakis papers and other things and from probably the very first,

544
00:25:42,369 --> 00:25:46,420
very first week it was put on arXiv or maybe even before that because

545
00:25:47,050 --> 00:25:49,229
I think one of the students had given a thought,

546
00:25:49,550 --> 00:25:51,189
I've been trying it, right. So

547
00:25:52,449 --> 00:25:55,550
I think if your problem is simple,

548
00:25:55,979 --> 00:25:56,680
right?

549
00:25:57,239 --> 00:25:58,140
Um

550
00:25:58,689 --> 00:25:59,949
And you do not have a code,

551
00:26:01,290 --> 00:26:03,319
I, I think it's a good idea, right?

552
00:26:03,689 --> 00:26:06,420
In the forward sense in terms of solving a PDE,

553
00:26:07,469 --> 00:26:08,520
uh but where it,

554
00:26:08,949 --> 00:26:14,099
you know, runs into tough ground is we are pretty damn good at solving PDEs, you know,

555
00:26:14,109 --> 00:26:17,729
over the past century, we have so many reliable methods.

556
00:26:18,839 --> 00:26:19,869
Um

557
00:26:20,310 --> 00:26:20,859
And

558
00:26:22,010 --> 00:26:28,829
uh putting the PDE in the loss function, there are a few, uh, few things that are not

559
00:26:29,189 --> 00:26:29,910
um

560
00:26:31,369 --> 00:26:34,119
uh appealing. Right? First,

561
00:26:34,979 --> 00:26:38,140
you, it's in the loss function. So by definition, you're doing it,

562
00:26:38,410 --> 00:26:39,119
you're not

563
00:26:39,260 --> 00:26:40,949
solving it exactly, you're,

564
00:26:41,300 --> 00:26:44,400
you know, reducing the residual to some level

565
00:26:44,589 --> 00:26:47,959
and that may be OK as a surrogate model in some applications.

566
00:26:49,130 --> 00:26:53,829
But even more so, right, if you think about all of the things we have learned

567
00:26:54,290 --> 00:26:58,010
in our numerical analysis classes and in our compressible flows

568
00:26:58,369 --> 00:26:59,010
is

569
00:26:59,270 --> 00:27:01,790
the discreteness of the problem matters, right?

570
00:27:02,020 --> 00:27:04,660
Your weak form matters. Um

571
00:27:05,380 --> 00:27:07,800
You have Riemann solvers, you have

572
00:27:08,199 --> 00:27:11,109
when you a continuous form is not the same as a discrete form.

573
00:27:11,540 --> 00:27:16,079
So you kind of lose that uh at least in the classic formulation of the PINNs.

574
00:27:16,969 --> 00:27:20,020
And there are some approaches that try to approach it from a different way like

575
00:27:21,069 --> 00:27:23,969
Koumoutsakos at Harvard, he has a method called ODIL

576
00:27:24,130 --> 00:27:25,900
that tries to keep the discrete form.

577
00:27:26,910 --> 00:27:30,010
But even if you ignore whatever I said so far,

578
00:27:31,000 --> 00:27:32,939
purely in terms of speed,

579
00:27:34,079 --> 00:27:34,680
I

580
00:27:35,170 --> 00:27:38,979
have a hard time believing that for complex PDEs, these methods can

581
00:27:39,810 --> 00:27:40,469
beat

582
00:27:40,900 --> 00:27:43,550
conventional ways of solving PDEs just because

583
00:27:44,000 --> 00:27:47,089
I think the baseline is really good. So this is how,

584
00:27:47,479 --> 00:27:48,630
you know, I, when I

585
00:27:48,900 --> 00:27:51,189
maybe we'll get into this later, when I talk about

586
00:27:51,420 --> 00:27:53,770
what are all the successful problems in AI for science?

587
00:27:53,780 --> 00:27:56,699
And I list a few things and PDEs will almost never be there.

588
00:27:57,459 --> 00:27:58,280
And that's

589
00:27:58,689 --> 00:28:00,040
partly because

590
00:28:00,280 --> 00:28:01,739
the baseline is very good,

591
00:28:02,250 --> 00:28:02,599
right?

592
00:28:02,709 --> 00:28:04,729
We are very good at solving PDEs.

593
00:28:05,630 --> 00:28:06,770
All of that said

594
00:28:07,089 --> 00:28:09,770
when you go the other way inverse problems,

595
00:28:10,609 --> 00:28:13,500
then I find a good application for, for PINNs.

596
00:28:14,079 --> 00:28:17,510
I still can't say it's the best in class. It cannot,

597
00:28:17,819 --> 00:28:19,160
it may or may not be

598
00:28:19,719 --> 00:28:20,380
uh

599
00:28:20,849 --> 00:28:23,550
it probably would not be the best approach to do

600
00:28:24,000 --> 00:28:27,109
inverse problems uh using classical methods,

601
00:28:27,560 --> 00:28:29,109
but it may be competitive

602
00:28:29,930 --> 00:28:30,630
and

603
00:28:31,729 --> 00:28:32,150
you

604
00:28:32,780 --> 00:28:35,790
it does not have to compete against 100 years of,

605
00:28:36,209 --> 00:28:39,609
of accumulated wealth of knowledge. So

606
00:28:39,949 --> 00:28:41,829
so again, to summarize it

607
00:28:42,420 --> 00:28:44,920
in certain circumstances, as a surrogate,

608
00:28:46,619 --> 00:28:47,479
it could help.

609
00:28:48,550 --> 00:28:50,569
And there are also hybrid versions, right?

610
00:28:50,640 --> 00:28:53,410
You can inject some physics, you can inject some data

611
00:28:54,560 --> 00:28:58,609
and it has that flexibility without all of this machinery that we need. And

612
00:28:58,910 --> 00:29:01,939
your entire code could be like 60 lines and by

613
00:29:02,199 --> 00:29:03,689
options about 60,000 lines

614
00:29:05,550 --> 00:29:06,810
and inverse

615
00:29:06,920 --> 00:29:10,239
problem solutions. So it has a place there.

616
00:29:11,390 --> 00:29:12,300
Yeah, II I uh

617
00:29:12,579 --> 00:29:14,890
I must confess, I always find it.

618
00:29:15,069 --> 00:29:17,369
It's a tricky one because there's so

619
00:29:17,849 --> 00:29:20,079
like I guess in the early days of any

620
00:29:20,400 --> 00:29:21,130
field,

621
00:29:21,140 --> 00:29:23,290
there's so many counter claims and claims that

622
00:29:23,300 --> 00:29:26,089
it is sometimes hard to have a truly independent

623
00:29:26,349 --> 00:29:26,979
view on it.

624
00:29:26,989 --> 00:29:30,520
I suppose you have to ultimately like you, you know, you have to get into the code,

625
00:29:30,530 --> 00:29:32,849
you have to look at it to really understand it.

626
00:29:33,130 --> 00:29:34,010
But um

627
00:29:34,119 --> 00:29:34,969
there was one

628
00:29:35,959 --> 00:29:36,939
comment

629
00:29:37,180 --> 00:29:40,380
and maybe this is a bit of a segue into the AI for science

630
00:29:40,780 --> 00:29:43,270
when I was speaking to one of the previous guests that

631
00:29:43,630 --> 00:29:44,859
scale

632
00:29:45,250 --> 00:29:46,300
has become

633
00:29:46,510 --> 00:29:50,699
the the main thing that that actually what has solved many of

634
00:29:50,709 --> 00:29:54,380
the large language problem issues is just the sheer amount of data

635
00:29:54,670 --> 00:29:58,699
and to the point that that is the focus not on

636
00:29:59,160 --> 00:30:00,199
the uh

637
00:30:00,599 --> 00:30:04,119
you know accuracy of the individual or the beauty of the algorithm.

638
00:30:04,390 --> 00:30:06,060
So with that in mind,

639
00:30:06,390 --> 00:30:09,119
and this discussion of data versus physics driven

640
00:30:09,130 --> 00:30:11,319
is the real problem for fluid dynamics.

641
00:30:11,329 --> 00:30:13,729
And more generally, AI for science, just

642
00:30:14,329 --> 00:30:16,349
the data. And if we solve that

643
00:30:16,609 --> 00:30:17,469
actually,

644
00:30:17,479 --> 00:30:21,540
we will come a long way and we're just trying to solve a problem with so little data

645
00:30:22,170 --> 00:30:23,630
that that's really the issue.

646
00:30:24,780 --> 00:30:25,530
Um

647
00:30:25,650 --> 00:30:28,560
So obviously like having more data helps,

648
00:30:29,420 --> 00:30:30,430
it can only help

649
00:30:30,770 --> 00:30:32,119
if we use it the right way.

650
00:30:33,040 --> 00:30:33,670
But

651
00:30:34,219 --> 00:30:37,979
if you want to like solve hard science problems,

652
00:30:38,630 --> 00:30:41,349
I don't think any amount of data would be enough

653
00:30:42,130 --> 00:30:45,609
or the amount of data to be required is so large that

654
00:30:46,689 --> 00:30:50,630
that is not the main problem because we just cannot reach them, right? Again, unless

655
00:30:51,420 --> 00:30:56,319
your problem is so well defined like alpha fold, you know, going from

656
00:30:56,449 --> 00:30:58,680
amino acid sequences to protein structure.

657
00:30:59,089 --> 00:30:59,760
So there

658
00:31:00,060 --> 00:31:01,540
the problem is so

659
00:31:02,030 --> 00:31:06,719
well posted, I shouldn't say well post but conducive to this kind of approach,

660
00:31:06,920 --> 00:31:07,359
right?

661
00:31:08,089 --> 00:31:08,630
But

662
00:31:10,739 --> 00:31:12,800
because you asked a more gentle question,

663
00:31:13,050 --> 00:31:16,520
I do not think that data is the biggest bottleneck

664
00:31:16,989 --> 00:31:19,219
because you will not have enough to like

665
00:31:19,489 --> 00:31:20,079
do that.

666
00:31:20,819 --> 00:31:26,160
Um But uh so inductive biases or or being able to bring in physics

667
00:31:26,560 --> 00:31:27,349
in tuition

668
00:31:27,930 --> 00:31:29,560
and I need to be very clear

669
00:31:29,859 --> 00:31:30,719
the right

670
00:31:31,089 --> 00:31:33,750
or the most appropriate level of physics, not

671
00:31:33,979 --> 00:31:37,880
anything you think, not some anything variant of something just because it,

672
00:31:37,890 --> 00:31:39,459
it feels appealing to us.

673
00:31:40,719 --> 00:31:41,270
Uh

674
00:31:41,479 --> 00:31:45,640
It's, you know, people talk about symmetries and variances, beautiful ideas,

675
00:31:45,650 --> 00:31:46,699
nothing wrong with that.

676
00:31:46,709 --> 00:31:47,079
But

677
00:31:47,400 --> 00:31:50,280
is that the most important thing for you to get

678
00:31:51,160 --> 00:31:52,969
the answer you desire

679
00:31:53,219 --> 00:31:56,420
or is that one other way of constraining the model

680
00:31:57,219 --> 00:31:58,229
to

681
00:31:58,699 --> 00:32:00,670
do what you think it

682
00:32:00,959 --> 00:32:02,800
should be doing with that information

683
00:32:03,020 --> 00:32:03,640
or

684
00:32:04,000 --> 00:32:08,119
should it be just doing something else to, to get it to where you want to go? So,

685
00:32:08,839 --> 00:32:09,459
um

686
00:32:09,739 --> 00:32:14,969
again, if the question is is well posed and you have a means of acquiring the data,

687
00:32:15,660 --> 00:32:16,670
then I would say

688
00:32:17,079 --> 00:32:18,400
data is the biggest thing,

689
00:32:18,540 --> 00:32:19,000
right?

690
00:32:19,160 --> 00:32:20,680
So again, going back to alpha for,

691
00:32:20,839 --> 00:32:25,000
I think they only had 170,000 proteins or something like that, that's not a lot.

692
00:32:25,339 --> 00:32:25,760
The common

693
00:32:26,060 --> 00:32:27,560
space is immense,

694
00:32:28,180 --> 00:32:28,599
right?

695
00:32:29,010 --> 00:32:32,000
And they were able to really solve the problem and advance the field.

696
00:32:32,770 --> 00:32:34,949
But if you take something like turbulence modeling

697
00:32:36,719 --> 00:32:37,520
in general,

698
00:32:39,030 --> 00:32:43,530
that is not the answer. But if you have a system or a

699
00:32:43,689 --> 00:32:45,640
class of assists you care about

700
00:32:46,369 --> 00:32:48,739
and you have your data driven modeling tools

701
00:32:49,339 --> 00:32:50,339
and you have a

702
00:32:50,599 --> 00:32:54,819
connection to some experimental or numerical simulation approaches

703
00:32:54,829 --> 00:32:56,520
that can get you the data you want,

704
00:32:57,459 --> 00:32:58,060
then

705
00:32:58,459 --> 00:33:01,560
yeah, I would say data is the most important piece. So it depends on

706
00:33:03,270 --> 00:33:05,520
your problem and and what you want to do.

707
00:33:06,660 --> 00:33:09,400
So maybe this is a good then se way to

708
00:33:10,050 --> 00:33:13,109
this discussion around foundational models because

709
00:33:13,489 --> 00:33:18,290
I think this, at least with some of the people I speak to is the most contested or

710
00:33:18,609 --> 00:33:22,010
argued point within science that just as

711
00:33:22,199 --> 00:33:26,060
large language models seem to be able to do so much

712
00:33:26,420 --> 00:33:32,020
without being explicitly taught the grammar of a language or explicitly done

713
00:33:32,030 --> 00:33:35,290
something then logically and people even confuse the terms and say,

714
00:33:35,300 --> 00:33:36,400
well, can we not build

715
00:33:36,819 --> 00:33:41,569
a foundational model for science or foundational model for fluid dynamics?

716
00:33:42,280 --> 00:33:43,989
Where do you, you, you put,

717
00:33:44,000 --> 00:33:47,199
you put recently on I believe a very um successful

718
00:33:47,209 --> 00:33:51,500
symposium looking at this question of foundational models for science.

719
00:33:51,510 --> 00:33:54,599
So maybe you could introduce a little bit the the challenge

720
00:33:54,849 --> 00:33:56,900
and some of the ways that you think we can

721
00:33:57,089 --> 00:33:57,819
address it.

722
00:33:58,290 --> 00:33:59,869
This is a favorite

723
00:34:00,479 --> 00:34:00,520
of

724
00:34:00,750 --> 00:34:00,780
mine

725
00:34:03,439 --> 00:34:08,228
just earlier. This year, we started a new center on scientific foundation models.

726
00:34:08,659 --> 00:34:11,458
Um So there are foundation models for science

727
00:34:11,750 --> 00:34:14,620
and then there are foundation models for

728
00:34:14,909 --> 00:34:16,679
particular domains within science.

729
00:34:16,688 --> 00:34:19,739
So there's I I want to make the distinction and we can cover both.

730
00:34:20,478 --> 00:34:23,820
But uh I think to answer your question. Um

731
00:34:25,570 --> 00:34:26,350
so

732
00:34:26,899 --> 00:34:31,300
by definition, right, this foundation models tend to

733
00:34:31,850 --> 00:34:32,489
do

734
00:34:33,239 --> 00:34:33,958
um

735
00:34:34,908 --> 00:34:37,489
general things in a very broad domain.

736
00:34:38,560 --> 00:34:41,810
Whereas the classical AI techniques in science,

737
00:34:41,958 --> 00:34:43,100
they tend to do,

738
00:34:43,830 --> 00:34:45,649
they want to have a narrow application

739
00:34:46,388 --> 00:34:48,259
and they tend to do something specific,

740
00:34:48,398 --> 00:34:48,768
right?

741
00:34:49,458 --> 00:34:50,320
Um

742
00:34:50,469 --> 00:34:54,199
So some people say broad and shallow and narrow and deep.

743
00:34:54,208 --> 00:34:57,070
But, you know, I don't want to go go that route.

744
00:34:57,489 --> 00:34:58,040
Um

745
00:34:58,149 --> 00:34:58,629
But,

746
00:34:58,800 --> 00:34:59,879
but I think the,

747
00:35:00,530 --> 00:35:01,439
the

748
00:35:02,149 --> 00:35:06,870
uh discussion around large language models has been incredibly fascinating.

749
00:35:06,879 --> 00:35:08,889
Right. So, because JJ, that was

750
00:35:09,129 --> 00:35:10,969
maybe for natural language, it is,

751
00:35:11,979 --> 00:35:14,800
it has really been

752
00:35:15,219 --> 00:35:18,040
surprising even to, I think, OpenAI,

753
00:35:18,320 --> 00:35:22,590
I do not think they expected it to expected 3.5 to be,

754
00:35:23,489 --> 00:35:26,209
to make that much of a splash because they pretty much had

755
00:35:26,850 --> 00:35:29,669
whatever we saw in, in November 2022 they had it

756
00:35:30,320 --> 00:35:31,949
a year ago, they just put an api

757
00:35:32,120 --> 00:35:32,360
play.

758
00:35:33,239 --> 00:35:33,790
Um

759
00:35:34,550 --> 00:35:35,300
But I think,

760
00:35:35,610 --> 00:35:40,250
I think the buzz can be explained in many different ways. So one is uh

761
00:35:43,310 --> 00:35:47,169
the ability to ability of these large language models to

762
00:35:47,929 --> 00:35:49,629
bring together different

763
00:35:49,739 --> 00:35:50,830
concepts

764
00:35:51,250 --> 00:35:54,229
and compose them, right? That I think is what

765
00:35:54,870 --> 00:35:56,090
um I think

766
00:35:56,550 --> 00:35:59,939
quite people off track, there's a beautiful paper from Princeton from

767
00:36:00,070 --> 00:36:00,290
Sanjiv

768
00:36:00,639 --> 00:36:01,020
aa's group

769
00:36:01,659 --> 00:36:02,979
where he defines

770
00:36:03,909 --> 00:36:08,870
topics and skills, right? A topic could be suing or mending or whatever, right?

771
00:36:08,879 --> 00:36:10,389
And a skill could be

772
00:36:11,060 --> 00:36:13,199
uh metaphor could be a skill

773
00:36:13,370 --> 00:36:17,649
and then he passes these topics uh and then he in his prompt,

774
00:36:17,820 --> 00:36:19,479
he he asks

775
00:36:19,649 --> 00:36:20,439
um

776
00:36:22,649 --> 00:36:26,239
the prompt in such a way that many of these skills and topics have to be

777
00:36:26,610 --> 00:36:27,399
covered

778
00:36:27,770 --> 00:36:32,879
and composed and then you get answers. And what he found was the larger the model,

779
00:36:33,530 --> 00:36:37,149
the more composition was happening, right? Between these topics

780
00:36:37,530 --> 00:36:39,709
and he made a very clear argument that

781
00:36:40,590 --> 00:36:41,310
the commentator

782
00:36:41,469 --> 00:36:47,250
space of topics and skills is so large that it is impossible for this

783
00:36:48,159 --> 00:36:50,419
to have seen all of this in the data, right? So

784
00:36:50,689 --> 00:36:52,300
the ability of these models to

785
00:36:52,449 --> 00:36:53,620
to compose

786
00:36:54,209 --> 00:36:56,830
and combine these different topics to a degree,

787
00:36:56,840 --> 00:36:59,500
if you go more than seven or eight compositions, it breaks down

788
00:37:00,209 --> 00:37:02,530
and and the small model would break down at three.

789
00:37:03,010 --> 00:37:06,600
So there is already something like intelligence there.

790
00:37:07,909 --> 00:37:08,520
And

791
00:37:09,209 --> 00:37:11,179
if you so so to me,

792
00:37:11,860 --> 00:37:16,120
even if these large language models do not get much better,

793
00:37:17,159 --> 00:37:20,379
I think they can still be extremely useful, right? And

794
00:37:20,879 --> 00:37:22,540
um I have a lot of thoughts on it,

795
00:37:22,550 --> 00:37:26,620
but I want you to like ask me something more specific so that we can we can hit,

796
00:37:26,629 --> 00:37:28,780
hit what you want to hit, not what

797
00:37:29,209 --> 00:37:29,889
I want. No,

798
00:37:30,090 --> 00:37:30,560
no, no, no,

799
00:37:30,750 --> 00:37:31,800
this is,

800
00:37:32,139 --> 00:37:34,260
this is good. So maybe um

801
00:37:36,189 --> 00:37:37,689
one question is

802
00:37:38,139 --> 00:37:43,610
do the architectures and the methods that are used for current foundational models

803
00:37:44,260 --> 00:37:47,439
for things like ChatGPT or Llama or et cetera.

804
00:37:47,649 --> 00:37:52,010
Can they be used largely as is or with minor modifications

805
00:37:52,120 --> 00:37:53,840
for applications and science

806
00:37:54,020 --> 00:37:57,280
or do we need completely different architectures to deal

807
00:37:57,439 --> 00:37:58,310
with a?

808
00:37:58,320 --> 00:37:59,310
So at a high level,

809
00:37:59,375 --> 00:38:01,514
is it just a matter of using the same

810
00:38:01,524 --> 00:38:04,104
thing and just plugging it into different data sources

811
00:38:04,405 --> 00:38:06,485
or do we need fundamentally different

812
00:38:06,495 --> 00:38:08,745
approaches than the sort of transformer style

813
00:38:08,955 --> 00:38:10,034
that has been used to date.

814
00:38:10,304 --> 00:38:11,814
I think the answer is both. Right.

815
00:38:11,824 --> 00:38:14,435
Let's, let's talk about the first thing, let's say you wanna

816
00:38:14,774 --> 00:38:15,695
have a uh

817
00:38:15,814 --> 00:38:18,955
a scientific version of GPT-4, right?

818
00:38:19,534 --> 00:38:20,784
So, as you said,

819
00:38:21,205 --> 00:38:24,945
uh plugging into different data sources is certainly a start.

820
00:38:24,955 --> 00:38:26,875
And I think that would certainly be

821
00:38:27,294 --> 00:38:28,875
uh something that

822
00:38:29,820 --> 00:38:30,870
would probably

823
00:38:31,040 --> 00:38:32,360
be very beneficial.

824
00:38:33,090 --> 00:38:35,129
So, for instance, uh

825
00:38:35,760 --> 00:38:36,020
you know,

826
00:38:36,030 --> 00:38:40,909
I was working on a fairly complicated mathematical derivation a few months ago

827
00:38:41,780 --> 00:38:46,389
and I would have pinged GPT for about 50 times no less than 50 times

828
00:38:46,810 --> 00:38:48,199
during the process about,

829
00:38:48,489 --> 00:38:48,679
you know,

830
00:38:48,689 --> 00:38:51,489
some statistical expectations and some expressions

831
00:38:51,500 --> 00:38:52,820
that I was struggling to derive

832
00:38:54,139 --> 00:38:56,600
maybe 49 out of those 50 times, it was wrong,

833
00:38:57,520 --> 00:39:01,739
but 50 out of those 50 times, it was useful because I was prompting it.

834
00:39:01,919 --> 00:39:02,979
And it was always

835
00:39:03,340 --> 00:39:05,780
telling me pushing me in directions

836
00:39:06,310 --> 00:39:08,649
that as a reasonably

837
00:39:09,379 --> 00:39:11,909
sophisticated expert,

838
00:39:12,310 --> 00:39:13,949
I could use that information

839
00:39:14,610 --> 00:39:15,209
and know that

840
00:39:15,340 --> 00:39:17,610
it is doing this arithmetic or is math wrong,

841
00:39:17,620 --> 00:39:19,250
but it's thinking in the right direction, right.

842
00:39:19,260 --> 00:39:19,659
So,

843
00:39:20,219 --> 00:39:23,350
so, so that's why I feel there is a, there is a place for this.

844
00:39:24,040 --> 00:39:28,080
And if you look at what GPT-4 and some of these other models have been trained of,

845
00:39:28,090 --> 00:39:30,439
you never know what GPT-4 is trained on really.

846
00:39:30,870 --> 00:39:32,889
But, you know, something like Google's P,

847
00:39:33,159 --> 00:39:33,939
they've released

848
00:39:35,139 --> 00:39:37,580
at least some statistics on what they trained on.

849
00:39:37,830 --> 00:39:39,540
Science tends to be like

850
00:39:39,899 --> 00:39:42,020
less than 5% or maybe

851
00:39:42,429 --> 00:39:45,350
3% of the entire data corpus

852
00:39:45,540 --> 00:39:47,939
sports was probably bigger than science

853
00:39:47,949 --> 00:39:49,939
entertainment was probably 20 times bigger.

854
00:39:49,949 --> 00:39:50,389
I don't know.

855
00:39:50,620 --> 00:39:51,120
But

856
00:39:51,590 --> 00:39:53,179
so just changing that balance

857
00:39:53,429 --> 00:39:56,979
is gonna be enormously useful. And

858
00:39:57,080 --> 00:39:59,500
our colleagues at Oregon National lab

859
00:39:59,750 --> 00:40:01,979
have actually started that process of

860
00:40:03,050 --> 00:40:07,070
curated scientific data, you know, from journal papers

861
00:40:08,429 --> 00:40:10,780
and guiding the process. So I think that

862
00:40:11,659 --> 00:40:17,060
even if the base is just in large language model with some multimodal capabilities,

863
00:40:18,040 --> 00:40:22,090
I think that is still going to be very useful uh in many different tasks.

864
00:40:22,739 --> 00:40:25,340
Um And there's also this this

865
00:40:26,270 --> 00:40:27,649
confusion

866
00:40:28,060 --> 00:40:30,250
or maybe misguided emphasis on

867
00:40:30,479 --> 00:40:33,610
these things being able to like solve your problems.

868
00:40:34,320 --> 00:40:37,050
But I think the best use of these models

869
00:40:37,959 --> 00:40:40,040
in my opinion, at least for the next

870
00:40:40,370 --> 00:40:41,600
15 years,

871
00:40:42,510 --> 00:40:45,600
nobody can predict the future. But at least the next decade

872
00:40:46,350 --> 00:40:48,290
is to use these as assistance,

873
00:40:49,020 --> 00:40:49,399
right?

874
00:40:50,149 --> 00:40:52,409
For things that you want to be doing

875
00:40:52,850 --> 00:40:54,810
and for this to be doing the

876
00:40:54,949 --> 00:40:56,090
the low end of the

877
00:40:57,169 --> 00:40:58,030
uh

878
00:40:58,399 --> 00:41:00,669
scientific process, right? You were still,

879
00:41:00,870 --> 00:41:02,120
you know, looking at the

880
00:41:03,389 --> 00:41:04,449
details and

881
00:41:04,560 --> 00:41:06,010
you're using, you're guiding it.

882
00:41:06,729 --> 00:41:07,310
But

883
00:41:07,500 --> 00:41:09,909
yeah, to answer your question more directly, I feel

884
00:41:10,110 --> 00:41:12,790
even a GPT-4 like

885
00:41:13,179 --> 00:41:13,949
base

886
00:41:14,280 --> 00:41:14,939
um

887
00:41:15,389 --> 00:41:15,949
but

888
00:41:16,129 --> 00:41:20,459
architecture but with uh the right data can take us a long way.

889
00:41:21,010 --> 00:41:22,949
But I also want to talk a little bit about,

890
00:41:23,379 --> 00:41:25,590
you know, even within this architecture,

891
00:41:26,669 --> 00:41:29,169
there is a lot we can do to improve

892
00:41:29,629 --> 00:41:31,169
the output, right? For instance,

893
00:41:32,189 --> 00:41:34,409
these things are trained to complete the next

894
00:41:34,709 --> 00:41:35,620
token, right? In,

895
00:41:36,280 --> 00:41:36,340
in,

896
00:41:36,530 --> 00:41:39,520
in at least supervised unsupervised pre training,

897
00:41:39,530 --> 00:41:40,949
they just complete the next word.

898
00:41:42,239 --> 00:41:43,370
And it's not really

899
00:41:44,090 --> 00:41:46,189
as some people say, thinking before speaking.

900
00:41:47,159 --> 00:41:50,699
But if even that part can be iterative,

901
00:41:51,560 --> 00:41:54,610
I think you can still get a huge performance gain.

902
00:41:55,070 --> 00:41:55,510
OK.

903
00:41:55,860 --> 00:41:56,560
And

904
00:41:57,030 --> 00:41:57,729
um

905
00:41:58,409 --> 00:42:01,469
being able to provide additional context

906
00:42:02,399 --> 00:42:04,129
or being able to

907
00:42:04,840 --> 00:42:07,879
connected to agents that can go look for other things,

908
00:42:09,139 --> 00:42:10,649
even this architecture,

909
00:42:11,340 --> 00:42:12,139
even if

910
00:42:12,389 --> 00:42:14,510
the architecture doesn't get better. And even if

911
00:42:15,129 --> 00:42:15,840
um

912
00:42:16,199 --> 00:42:18,479
the language part of the model doesn't get better

913
00:42:19,750 --> 00:42:20,929
if it's done, right?

914
00:42:21,570 --> 00:42:23,870
I think it can be massively beneficial

915
00:42:24,280 --> 00:42:25,060
for science.

916
00:42:26,090 --> 00:42:29,590
So are you referring a little bit more to the idea of

917
00:42:29,870 --> 00:42:32,439
during the conceptual phase?

918
00:42:32,449 --> 00:42:36,419
So not simulations now, but just more scientific discovery like

919
00:42:36,850 --> 00:42:41,949
developing a hypothesis or analyzing flow fields? Or

920
00:42:42,219 --> 00:42:43,979
you're saying that you could be using these

921
00:42:43,989 --> 00:42:47,189
transformer style models to help in that side,

922
00:42:48,679 --> 00:42:51,540
not just the replace

923
00:42:51,870 --> 00:42:53,689
physics based simulations

924
00:42:53,830 --> 00:42:56,379
by machine learning. That's kind of what you

925
00:42:57,360 --> 00:43:00,629
Yeah. Yeah. So I only answered the first part of your previous question, right.

926
00:43:00,639 --> 00:43:01,290
I said

927
00:43:01,820 --> 00:43:03,020
if we just stick with

928
00:43:03,639 --> 00:43:04,979
LLMs and transformers,

929
00:43:05,689 --> 00:43:06,830
even if you do that,

930
00:43:06,989 --> 00:43:09,219
right? With some tinkering with the

931
00:43:09,899 --> 00:43:12,270
with the way it's trained, like I said,

932
00:43:12,550 --> 00:43:15,479
not just complete the next word but maybe one day

933
00:43:15,639 --> 00:43:15,719
today

934
00:43:16,409 --> 00:43:17,760
and that's not too hard to do.

935
00:43:18,669 --> 00:43:20,110
So I think that will

936
00:43:20,239 --> 00:43:22,060
automatically help but then,

937
00:43:23,290 --> 00:43:25,879
and it can help data processing, it can help you,

938
00:43:26,040 --> 00:43:29,810
you know, code up, it can help you in so many ways as scientific assistant.

939
00:43:30,659 --> 00:43:33,959
But for other more regular science tasks that we do,

940
00:43:34,570 --> 00:43:38,629
of course transformer architectures on their own won't cut it. We need

941
00:43:39,280 --> 00:43:40,939
newer architectures and

942
00:43:41,179 --> 00:43:42,699
I wouldn't even say newer.

943
00:43:43,399 --> 00:43:44,169
Uh

944
00:43:44,729 --> 00:43:47,879
most of these things have existed in, in some form for a while.

945
00:43:47,889 --> 00:43:49,409
We just, as you said earlier

946
00:43:49,679 --> 00:43:52,070
scale, we are scaling it up and we are

947
00:43:52,699 --> 00:43:53,520
uh

948
00:43:54,020 --> 00:43:55,300
I would say

949
00:43:55,729 --> 00:43:57,840
you can hobble it as

950
00:43:59,169 --> 00:43:59,250
Ian

951
00:43:59,479 --> 00:43:59,669
Foster

952
00:43:59,830 --> 00:44:00,040
from

953
00:44:00,159 --> 00:44:00,330
or God

954
00:44:00,439 --> 00:44:00,969
says it.

955
00:44:02,000 --> 00:44:02,469
But yes,

956
00:44:02,489 --> 00:44:05,310
uh different architectures but not necessarily completely

957
00:44:05,320 --> 00:44:07,000
radical that doesn't exist so far.

958
00:44:08,449 --> 00:44:11,399
So with that, like maybe diving into

959
00:44:11,540 --> 00:44:14,510
perhaps some examples and because I've always been

960
00:44:15,010 --> 00:44:17,560
having this sort of theoretical debate

961
00:44:17,850 --> 00:44:20,139
with um with some colleagues.

962
00:44:21,229 --> 00:44:22,679
So if you wanted,

963
00:44:24,459 --> 00:44:26,929
what do you see as the definition of foundation or do?

964
00:44:26,939 --> 00:44:30,010
Is it really realistic if we just look at CFD

965
00:44:30,709 --> 00:44:36,719
that instead of using a code by company X, you know that you put settings in

966
00:44:37,409 --> 00:44:39,909
that it could be economically viable,

967
00:44:40,419 --> 00:44:42,550
that there could be a model

968
00:44:42,810 --> 00:44:44,659
trained on enough

969
00:44:45,060 --> 00:44:47,850
incompressible compressible, low speed, high speed, you know,

970
00:44:47,860 --> 00:44:49,750
all different flow regimes

971
00:44:50,000 --> 00:44:54,709
that you could essentially then give it a geometry and it has seen somewhere

972
00:44:55,000 --> 00:44:56,909
enough to be able to give you an answer

973
00:44:57,489 --> 00:44:58,270
or

974
00:44:59,179 --> 00:45:00,250
do you see

975
00:45:00,800 --> 00:45:05,800
that it is more likely a quote unquote foundational model that is

976
00:45:05,949 --> 00:45:11,120
for a particular flow regime or a particular use case.

977
00:45:14,010 --> 00:45:16,320
Um answer could be both. But

978
00:45:16,659 --> 00:45:18,189
if you'd ask me maybe

979
00:45:19,479 --> 00:45:23,780
six months ago, I would have said the first thing is fiction,

980
00:45:23,919 --> 00:45:26,010
but my thinking has evolved on this

981
00:45:26,379 --> 00:45:30,060
and there's also been some work trying to train these models on different

982
00:45:30,379 --> 00:45:31,860
kinds of domains and

983
00:45:32,139 --> 00:45:35,340
introducing body conditions by masking and things like that.

984
00:45:36,100 --> 00:45:39,280
Um So I think it is possible, right? So think about

985
00:45:40,000 --> 00:45:42,820
it is, but, but your question was to get an answer,

986
00:45:43,320 --> 00:45:44,459
even get an answer.

987
00:45:45,199 --> 00:45:47,219
I don't think that answer would be terrible,

988
00:45:47,750 --> 00:45:49,620
but it probably won't

989
00:45:50,820 --> 00:45:53,370
satisfy all of your needs, right? So think about,

990
00:45:54,310 --> 00:45:55,780
think about uh

991
00:45:56,479 --> 00:45:57,320
uh

992
00:45:58,389 --> 00:46:00,050
an immerse boundary type approach

993
00:46:00,699 --> 00:46:04,439
where you can put in any geometry, you can put in boundary conditions with the meshes

994
00:46:04,629 --> 00:46:05,510
condition and then

995
00:46:05,810 --> 00:46:07,120
your geometry gets

996
00:46:07,360 --> 00:46:08,520
represented,

997
00:46:08,729 --> 00:46:11,520
you know, as a cut cell or mask or whatever it is,

998
00:46:12,629 --> 00:46:15,020
I really think and there's a paper from Anima

999
00:46:15,510 --> 00:46:15,550
Anan

1000
00:46:15,850 --> 00:46:16,449
Kumar's group,

1001
00:46:16,629 --> 00:46:16,639
a

1002
00:46:17,060 --> 00:46:19,409
Kumar's group that does something very similar.

1003
00:46:20,310 --> 00:46:21,560
So I do think

1004
00:46:22,379 --> 00:46:25,239
even, and it's not like very complicated,

1005
00:46:25,250 --> 00:46:28,399
I think even within the next couple of years, we may see

1006
00:46:28,979 --> 00:46:30,419
something like that.

1007
00:46:30,590 --> 00:46:31,159
Uh

1008
00:46:32,100 --> 00:46:33,719
that is somewhat reliable,

1009
00:46:34,250 --> 00:46:35,310
that can give you

1010
00:46:36,070 --> 00:46:37,129
an answer. That

1011
00:46:37,320 --> 00:46:38,100
could be,

1012
00:46:38,610 --> 00:46:40,350
I'm just making up a number here

1013
00:46:40,469 --> 00:46:42,489
like 75 80% accurate.

1014
00:46:43,270 --> 00:46:45,290
In my opinion, that is still useful

1015
00:46:45,629 --> 00:46:46,330
because

1016
00:46:46,979 --> 00:46:49,639
that's not your final thing. That could be your

1017
00:46:50,320 --> 00:46:52,580
precondition or that's the first guess

1018
00:46:52,989 --> 00:46:55,060
that you want to explore in many different ways.

1019
00:46:55,590 --> 00:46:56,739
I really feel

1020
00:46:57,010 --> 00:46:57,540
that

1021
00:46:57,889 --> 00:47:00,580
that kind of capability is not fiction.

1022
00:47:00,590 --> 00:47:03,689
I would not have been as confident in my answer six months ago.

1023
00:47:04,340 --> 00:47:06,800
Um So right now we're running a summer school and we are,

1024
00:47:07,219 --> 00:47:09,689
one of the hackathon groups is actually trying something

1025
00:47:09,699 --> 00:47:12,020
like that and I'm actually seeing that in action.

1026
00:47:13,179 --> 00:47:14,860
So yes, it is possible to,

1027
00:47:15,179 --> 00:47:16,060
to get there. But

1028
00:47:16,550 --> 00:47:18,850
you know, if you go to a purist and say,

1029
00:47:19,000 --> 00:47:20,899
hey, look at this answer, they only look at

1030
00:47:21,310 --> 00:47:22,330
where it's wrong.

1031
00:47:22,479 --> 00:47:24,179
And I think that that view is

1032
00:47:24,310 --> 00:47:25,489
OK from

1033
00:47:25,860 --> 00:47:27,020
uh uh uh

1034
00:47:27,729 --> 00:47:29,590
guardrail perspective.

1035
00:47:30,360 --> 00:47:30,379
Um

1036
00:47:30,649 --> 00:47:36,169
But then the other question you asked, you know, for a class of problems,

1037
00:47:36,629 --> 00:47:37,340
I think

1038
00:47:37,840 --> 00:47:39,149
yeah, many tools that

1039
00:47:39,560 --> 00:47:42,449
even pre foundation models, many tools we've been developing

1040
00:47:43,370 --> 00:47:47,879
can actually get us there. And in fact, I even covered one such example earlier

1041
00:47:48,040 --> 00:47:49,070
and I spoke about

1042
00:47:49,409 --> 00:47:51,149
if you're able to parameterize the problem,

1043
00:47:52,120 --> 00:47:52,689
then

1044
00:47:53,469 --> 00:47:54,169
I think

1045
00:47:54,310 --> 00:47:57,620
AI can, these MS techniques purely data driven

1046
00:47:58,250 --> 00:47:59,030
can do

1047
00:47:59,260 --> 00:48:02,229
you know a lot of damage already.

1048
00:48:03,209 --> 00:48:07,320
And then with these foundation models thrown in especially foundation models

1049
00:48:07,870 --> 00:48:09,600
that can interact with agents,

1050
00:48:10,350 --> 00:48:14,409
right? At least that's my notion of how these things would be useful in the future.

1051
00:48:14,860 --> 00:48:15,649
Even if

1052
00:48:16,510 --> 00:48:20,600
these GPT-4-like models or GPT-5 or whatever they are, even if

1053
00:48:21,330 --> 00:48:23,139
they plateau in their accuracy,

1054
00:48:23,149 --> 00:48:26,889
their ability to interface with more specialized agents

1055
00:48:27,969 --> 00:48:31,310
to carry out tasks in more specific tasks

1056
00:48:31,570 --> 00:48:33,939
and then being able to synthesize information back.

1057
00:48:34,399 --> 00:48:37,469
I think to me that's the most realistic or pragmatic

1058
00:48:37,979 --> 00:48:42,260
scenario of how these tools will evolve in the next decade or so.

1059
00:48:43,699 --> 00:48:44,580
But um

1060
00:48:45,350 --> 00:48:49,239
some would say that this opens up a very interesting uh

1061
00:48:50,040 --> 00:48:52,840
commercial or ethical angle or both.

1062
00:48:53,070 --> 00:48:54,179
So one is

1063
00:48:55,389 --> 00:48:56,360
if,

1064
00:48:56,899 --> 00:48:59,389
if, if you were to make a foundational model,

1065
00:49:00,300 --> 00:49:01,479
let's say that is

1066
00:49:01,610 --> 00:49:06,080
for conceptual design, so not super high accuracy, but for conceptual design,

1067
00:49:06,590 --> 00:49:07,070
um

1068
00:49:08,669 --> 00:49:10,810
most of that data is public,

1069
00:49:12,080 --> 00:49:15,830
you know, like a hypersonic vehicle, not many uh hypersonic vehicles,

1070
00:49:15,840 --> 00:49:17,899
all the results are in the public domain

1071
00:49:18,110 --> 00:49:19,939
or for a um

1072
00:49:20,899 --> 00:49:25,360
or even for like road cars, you know, a lot of those are kept commercially sensitive.

1073
00:49:25,770 --> 00:49:26,870
How do you think

1074
00:49:27,320 --> 00:49:31,100
we would get around the the data availability

1075
00:49:31,439 --> 00:49:32,040
issue?

1076
00:49:33,280 --> 00:49:37,340
Yeah, that's an interesting question for which I don't think one person can answer

1077
00:49:37,469 --> 00:49:41,699
but, but remember there are two stages that is this pre training stage

1078
00:49:42,439 --> 00:49:46,100
and then there is the fine tuning stage. So when I say this will be useful,

1079
00:49:46,790 --> 00:49:49,510
the pre training may not be as sophisticated,

1080
00:49:49,520 --> 00:49:51,989
but every organization can have its own

1081
00:49:52,600 --> 00:49:53,729
fine tuned

1082
00:49:54,560 --> 00:49:56,209
fine tunable data sets

1083
00:49:56,939 --> 00:49:57,689
and that

1084
00:49:57,840 --> 00:49:58,919
can be private,

1085
00:49:59,060 --> 00:50:01,479
that can be as sophisticated as they want,

1086
00:50:01,969 --> 00:50:02,479
right?

1087
00:50:02,879 --> 00:50:05,199
And then there may be enough, you know,

1088
00:50:05,209 --> 00:50:08,510
there is enough open source activity out there even for

1089
00:50:09,000 --> 00:50:11,360
something like hypersonics, even though

1090
00:50:12,560 --> 00:50:16,139
I think at least in the near future things will be more closed than open.

1091
00:50:18,590 --> 00:50:20,929
There is enough open source research

1092
00:50:21,149 --> 00:50:21,889
happening

1093
00:50:22,419 --> 00:50:23,010
um

1094
00:50:23,709 --> 00:50:26,770
in this country and in Europe and other countries that

1095
00:50:27,550 --> 00:50:28,610
I think getting the

1096
00:50:29,010 --> 00:50:31,010
data sets for pre training

1097
00:50:31,449 --> 00:50:32,889
is not unrealistic,

1098
00:50:33,550 --> 00:50:36,489
but then the magic would be but that on its own,

1099
00:50:36,760 --> 00:50:38,840
I do not think is going to be

1100
00:50:40,340 --> 00:50:41,879
helpful for problems

1101
00:50:42,939 --> 00:50:45,040
that are not closely related to,

1102
00:50:46,070 --> 00:50:47,530
you know, the big uh

1103
00:50:47,639 --> 00:50:48,219
the over

1104
00:50:48,469 --> 00:50:50,010
represented parts of the data set. But

1105
00:50:51,580 --> 00:50:55,929
so the fine tunable parts, I think that will be the most valuable. And as you know,

1106
00:50:56,370 --> 00:50:58,969
the pre training is a bulk of the cost, right?

1107
00:50:58,979 --> 00:51:03,729
It's about 95 99% of the cost is in pre training, generally speaking,

1108
00:51:03,870 --> 00:51:03,899
and

1109
00:51:04,500 --> 00:51:07,260
then people can have their own fine tune things

1110
00:51:07,629 --> 00:51:08,929
within their organizations.

1111
00:51:10,010 --> 00:51:11,310
So how um

1112
00:51:12,280 --> 00:51:13,260
how can

1113
00:51:13,770 --> 00:51:17,500
how can the community go about and do this then? So what are the practical

1114
00:51:18,209 --> 00:51:22,060
challenges you put on the symposium to look at foundational models?

1115
00:51:22,070 --> 00:51:23,350
But what needs to be

1116
00:51:24,320 --> 00:51:25,030
done

1117
00:51:25,139 --> 00:51:28,110
essentially to enable this?

1118
00:51:28,949 --> 00:51:31,969
Yeah, I think, you know, anytime a new field,

1119
00:51:32,469 --> 00:51:34,449
newish field emerges,

1120
00:51:35,229 --> 00:51:37,489
you need to get people together and talk, right?

1121
00:51:37,500 --> 00:51:40,379
So that's really the first step that we have

1122
00:51:41,040 --> 00:51:42,449
organized here

1123
00:51:42,899 --> 00:51:46,570
and we had this conference in April of this year. So next

1124
00:51:46,770 --> 00:51:46,780
uh

1125
00:51:46,919 --> 00:51:51,409
summer or early summer, we're planning the next version of this meeting and

1126
00:51:51,770 --> 00:51:54,040
it's gonna be a really big deal with

1127
00:51:54,870 --> 00:51:56,580
all the names you hear in the news,

1128
00:51:56,590 --> 00:52:01,129
hopefully most of the names at least uh to be to be present.

1129
00:52:01,139 --> 00:52:02,159
So I think so that's

1130
00:52:02,459 --> 00:52:04,050
one activity right to,

1131
00:52:04,209 --> 00:52:08,699
to be able to discuss ideas, to be able to discuss tools, to have tutorials, you know,

1132
00:52:08,709 --> 00:52:09,969
to propagate this thing.

1133
00:52:10,909 --> 00:52:12,770
Um But beyond that,

1134
00:52:12,820 --> 00:52:17,330
there are already a few things that are happening uh in creating these

1135
00:52:17,850 --> 00:52:19,250
subdomain

1136
00:52:19,610 --> 00:52:20,729
groups.

1137
00:52:21,219 --> 00:52:21,810
And

1138
00:52:22,229 --> 00:52:25,800
at least if some of visions, visions come true,

1139
00:52:26,290 --> 00:52:27,739
we may have a multi

1140
00:52:28,090 --> 00:52:29,199
university

1141
00:52:29,709 --> 00:52:31,969
uh government industry

1142
00:52:32,260 --> 00:52:36,370
consortium. I call it as, as futures IFM Institute

1143
00:52:37,169 --> 00:52:40,780
where we go through this process. But I would like you to look at, you

1144
00:52:41,179 --> 00:52:43,659
know, there is one organization, I don't know if you know,

1145
00:52:43,669 --> 00:52:45,790
it's called the trillion parameter consortium.

1146
00:52:46,340 --> 00:52:47,929
It's kind of laid out of organ

1147
00:52:48,689 --> 00:52:49,750
and uh

1148
00:52:49,860 --> 00:52:52,020
they already have probably like

1149
00:52:53,639 --> 00:52:58,709
50 universities. Michigan is a part of it and a few national labs and not just the US

1150
00:52:59,389 --> 00:53:02,659
it, it's easy website to remember T PC dot DEV

1151
00:53:03,399 --> 00:53:04,540
T pc.de.

1152
00:53:05,209 --> 00:53:07,340
And they're already facilitating precisely the

1153
00:53:07,350 --> 00:53:08,459
kind of things you're talking about,

1154
00:53:08,469 --> 00:53:08,850
right?

1155
00:53:08,860 --> 00:53:10,030
They're organizing,

1156
00:53:10,600 --> 00:53:13,820
they talk, they talk to ACM and they've organized

1157
00:53:14,239 --> 00:53:15,750
access to the journals.

1158
00:53:16,570 --> 00:53:20,189
They are basically setting a benchmark, they're setting up evaluations

1159
00:53:21,020 --> 00:53:24,629
that is still to support the science,

1160
00:53:24,639 --> 00:53:26,989
the Open Science Foundation model in particular.

1161
00:53:28,070 --> 00:53:30,830
But that is the community that is the farthest along

1162
00:53:31,870 --> 00:53:32,530
in

1163
00:53:32,979 --> 00:53:33,709
uh

1164
00:53:34,459 --> 00:53:35,939
enabling the kind of thing

1165
00:53:36,300 --> 00:53:37,510
you're talking. But that

1166
00:53:38,649 --> 00:53:40,110
is not a body that,

1167
00:53:41,000 --> 00:53:43,399
you know, gives you money to do research, right?

1168
00:53:43,800 --> 00:53:46,600
But, but we are, you know, envisioning this

1169
00:53:46,709 --> 00:53:48,379
this kind of institute with

1170
00:53:49,159 --> 00:53:49,800
uh

1171
00:53:50,120 --> 00:53:54,419
you know, government funding and philanthropy and venture funds

1172
00:53:54,689 --> 00:53:57,040
to make sure there are also the resources needed.

1173
00:53:57,719 --> 00:54:02,090
But uh you know, starting with some of our activities as part of the next

1174
00:54:02,850 --> 00:54:05,850
foundation models conference in in May 2025

1175
00:54:06,310 --> 00:54:09,100
we already have some of those structures built up

1176
00:54:09,820 --> 00:54:11,689
as you know, uh we

1177
00:54:12,169 --> 00:54:14,510
collaborate with larger organizations. So

1178
00:54:15,320 --> 00:54:15,750
um

1179
00:54:17,040 --> 00:54:20,860
if you take CFD, which is our traditional domain,

1180
00:54:21,709 --> 00:54:22,350
there are those

1181
00:54:22,590 --> 00:54:22,959
eco

1182
00:54:23,229 --> 00:54:25,159
tact data sets and there are

1183
00:54:25,560 --> 00:54:29,209
maybe about 10 or 12 such sizable clusters.

1184
00:54:29,899 --> 00:54:34,010
I mean that would be a start. But then if you have an open way to

1185
00:54:34,820 --> 00:54:37,159
uh bring data in

1186
00:54:37,989 --> 00:54:41,889
and hopefully some resources to train it and then more importantly, incentives.

1187
00:54:42,739 --> 00:54:45,580
So I think the incentives have to change right now. At

1188
00:54:45,840 --> 00:54:46,810
least so far,

1189
00:54:47,889 --> 00:54:51,500
the academic and the scientific community

1190
00:54:52,439 --> 00:54:53,739
incentivizes

1191
00:54:54,239 --> 00:54:55,580
certain class of

1192
00:54:56,129 --> 00:54:59,580
activities as more valuable or more procedures

1193
00:55:00,189 --> 00:55:02,590
and that has to change, right?

1194
00:55:03,189 --> 00:55:03,580
Uh

1195
00:55:04,770 --> 00:55:05,489
And I think

1196
00:55:05,639 --> 00:55:07,939
to me the one of the most exciting

1197
00:55:08,659 --> 00:55:09,360
um

1198
00:55:09,479 --> 00:55:10,729
aspects of

1199
00:55:11,149 --> 00:55:13,000
aspect of foundation models

1200
00:55:13,520 --> 00:55:15,719
is the idea that for the first time

1201
00:55:15,939 --> 00:55:17,629
in human history, I must say

1202
00:55:19,560 --> 00:55:21,010
there is a possibility of

1203
00:55:21,669 --> 00:55:25,020
say thousands of domain scientists collaborating

1204
00:55:25,959 --> 00:55:27,850
on the same platform, right?

1205
00:55:28,379 --> 00:55:30,370
You know, some of them are contributing data,

1206
00:55:30,379 --> 00:55:33,639
some of them may be fine tuning some of them are building the architecture,

1207
00:55:33,649 --> 00:55:35,379
some of them are doing evaluations.

1208
00:55:36,070 --> 00:55:38,560
Um you know, some of them are probing the models,

1209
00:55:38,669 --> 00:55:42,409
some are developing architectures, but it is the same platform.

1210
00:55:43,169 --> 00:55:43,709
That's

1211
00:55:44,530 --> 00:55:45,189
kind of

1212
00:55:45,510 --> 00:55:47,110
pretty crazy if you think about

1213
00:55:47,790 --> 00:55:51,850
how it changes the nature of collaborations, which tend to be kind of loose,

1214
00:55:51,860 --> 00:55:53,399
you know, unless you have a project like

1215
00:55:54,620 --> 00:55:55,879
and other things.

1216
00:55:56,439 --> 00:55:57,959
Um but still

1217
00:55:58,239 --> 00:56:00,560
this can be more direct, everyone's working

1218
00:56:01,050 --> 00:56:04,199
on the same platform on the same type of model

1219
00:56:05,360 --> 00:56:07,020
and bringing their pieces in.

1220
00:56:08,070 --> 00:56:08,620
And

1221
00:56:09,290 --> 00:56:11,270
I, I think many

1222
00:56:12,129 --> 00:56:15,340
countries I think will start to coalesce around us

1223
00:56:15,790 --> 00:56:18,919
because of the fact that this is a collaboration tool that is

1224
00:56:19,110 --> 00:56:20,209
that is unprecedented.

1225
00:56:20,929 --> 00:56:22,590
But again, I think the

1226
00:56:22,780 --> 00:56:24,030
the benefit would be

1227
00:56:24,860 --> 00:56:28,360
uh when this model kind of interacts with those agents, right?

1228
00:56:28,370 --> 00:56:30,760
And some people may be developing those specialized agents

1229
00:56:31,310 --> 00:56:34,439
like for protein folding or turbulence modeling or whatever.

1230
00:56:34,979 --> 00:56:38,959
So, so I think you can unify domains within science and science itself.

1231
00:56:40,199 --> 00:56:41,370
And this is the first

1232
00:56:42,010 --> 00:56:45,189
platform that allows people to do that,

1233
00:56:45,570 --> 00:56:46,050
right?

1234
00:56:47,139 --> 00:56:48,629
So I like that idea.

1235
00:56:49,479 --> 00:56:53,030
Uh I, I think what your vision is is more of a

1236
00:56:54,350 --> 00:56:57,330
and I haven't heard this articulated in such a clear way,

1237
00:56:58,840 --> 00:57:00,580
you're saying essentially that

1238
00:57:01,270 --> 00:57:06,649
we sometimes double down and focus in this fluid domain CFD domain on PINNs or on,

1239
00:57:06,659 --> 00:57:08,290
you know, neural operators on things.

1240
00:57:08,300 --> 00:57:11,449
But you, you're already seeing them as the sort of the the agents

1241
00:57:11,689 --> 00:57:14,870
that a future sort of large language model can be calling,

1242
00:57:15,050 --> 00:57:19,629
but there needs to be also the investment on the underlying or the,

1243
00:57:19,699 --> 00:57:21,689
the main thing that's calling those agents and,

1244
00:57:21,699 --> 00:57:23,699
and able to interpret what we're asking.

1245
00:57:23,709 --> 00:57:27,489
And that's something that perhaps hasn't been as widely spoken about.

1246
00:57:27,510 --> 00:57:29,520
At least I've not heard this as much and

1247
00:57:29,929 --> 00:57:33,570
I'm appreciating you saying this because it's kind of making sense in my head now.

1248
00:57:33,939 --> 00:57:34,750
Um,

1249
00:57:35,100 --> 00:57:36,510
and I guess it can learn

1250
00:57:36,699 --> 00:57:37,870
vice, you know,

1251
00:57:38,290 --> 00:57:40,219
the agents can learn from what the large,

1252
00:57:40,229 --> 00:57:43,199
large language model is doing and vice versa, I guess to,

1253
00:57:43,790 --> 00:57:44,709
you know, to do that.

1254
00:57:45,000 --> 00:57:45,590
Um

1255
00:57:46,300 --> 00:57:48,850
I do want to mention one important thing, right? So it is

1256
00:57:49,090 --> 00:57:51,080
the the large model, the agents

1257
00:57:51,290 --> 00:57:55,239
and the humans also in the loop doing this general guidance and evaluating

1258
00:57:55,600 --> 00:57:56,110
but

1259
00:57:56,780 --> 00:57:57,879
taken together.

1260
00:57:58,020 --> 00:58:00,840
I think this has the potential to

1261
00:58:01,300 --> 00:58:03,280
be a new way of

1262
00:58:04,350 --> 00:58:04,889
or

1263
00:58:05,139 --> 00:58:08,610
to be more conservative here, a new tool to do science,

1264
00:58:08,949 --> 00:58:09,370
but

1265
00:58:09,959 --> 00:58:13,159
a collaborative tool that has not existed, right?

1266
00:58:14,699 --> 00:58:18,000
Well, you, you probably know where I'm going to go next with this because

1267
00:58:18,639 --> 00:58:20,439
I'm always never sure.

1268
00:58:20,449 --> 00:58:24,639
And I think it's a global debate on regulation and, and the ethics.

1269
00:58:25,479 --> 00:58:28,120
I know that when I've spoken to various,

1270
00:58:28,580 --> 00:58:29,030
you know,

1271
00:58:30,100 --> 00:58:33,489
um colleagues or people at conferences or, or wherever

1272
00:58:33,899 --> 00:58:37,129
there then starts to be this, you know, light bulb moment where you realize, hold on,

1273
00:58:37,139 --> 00:58:39,620
you're condensing a lot of knowledge into a,

1274
00:58:39,820 --> 00:58:41,129
into one thing

1275
00:58:41,649 --> 00:58:44,300
and most software packages are

1276
00:58:44,820 --> 00:58:49,760
have IP restrictions or have some form that stops them going, you know,

1277
00:58:49,770 --> 00:58:50,939
to other places.

1278
00:58:52,010 --> 00:58:54,310
Is it the case,

1279
00:58:57,070 --> 00:59:00,250
how difficult is it going to be to do this fully open source?

1280
00:59:02,040 --> 00:59:03,020
Yeah. Well,

1281
00:59:03,179 --> 00:59:06,290
again, what is open source is, is

1282
00:59:07,429 --> 00:59:09,250
the the first question, right? Like

1283
00:59:10,530 --> 00:59:11,989
whatever is available,

1284
00:59:12,169 --> 00:59:13,870
open source right now

1285
00:59:14,179 --> 00:59:14,979
has data

1286
00:59:15,689 --> 00:59:18,310
under the use it under the right terms,

1287
00:59:18,729 --> 00:59:20,689
I think could be fair game

1288
00:59:21,169 --> 00:59:22,110
for

1289
00:59:22,570 --> 00:59:25,729
the pre training, the pre trained part of the foundation model.

1290
00:59:26,300 --> 00:59:29,649
And then there's a fine tuning and the agents that become a little more specific,

1291
00:59:29,659 --> 00:59:29,989
right?

1292
00:59:30,629 --> 00:59:33,540
So again, I'm not an expert on this topic, but even

1293
00:59:33,909 --> 00:59:34,989
in that scenario,

1294
00:59:35,949 --> 00:59:37,409
there are many things to worry about

1295
00:59:38,379 --> 00:59:39,239
what is,

1296
00:59:40,699 --> 00:59:42,840
you know, you spoke about. Uh

1297
00:59:43,280 --> 00:59:45,919
I don't know which two words you spoke about, but you didn't talk about,

1298
00:59:45,929 --> 00:59:47,439
you didn't mention security, right?

1299
00:59:47,449 --> 00:59:48,000
What if

1300
00:59:48,570 --> 00:59:50,600
you know, something comes out of this, that

1301
00:59:51,310 --> 00:59:52,610
could be potentially

1302
00:59:53,360 --> 00:59:54,290
a risk to

1303
00:59:55,850 --> 00:59:59,030
national security or, you know, some other thing that we worry about.

1304
01:00:00,219 --> 01:00:02,479
So that's certainly an open question. And I'm not,

1305
01:00:03,709 --> 01:00:03,979
I mean,

1306
01:00:03,989 --> 01:00:06,250
I was comfortable talking about everything before

1307
01:00:06,260 --> 01:00:08,300
this because I thought about this.

1308
01:00:08,310 --> 01:00:10,159
I kind of worked on some aspects,

1309
01:00:10,699 --> 01:00:12,629
but that's not my expertise. So,

1310
01:00:12,840 --> 01:00:16,610
but I would say yes, I mean, we have to look at this very carefully.

1311
01:00:17,370 --> 01:00:18,020
Um

1312
01:00:18,469 --> 01:00:21,510
But I think what is openly available

1313
01:00:21,729 --> 01:00:22,870
can be used,

1314
01:00:23,350 --> 01:00:24,699
right, for that. And then

1315
01:00:25,199 --> 01:00:26,669
for things that are more

1316
01:00:26,919 --> 01:00:27,340
uh

1317
01:00:28,419 --> 01:00:30,320
this open science foundation model

1318
01:00:32,139 --> 01:00:32,949
um

1319
01:00:33,439 --> 01:00:37,560
is not the only open, only science foundation model that would be out there,

1320
01:00:37,800 --> 01:00:38,320
right?

1321
01:00:38,810 --> 01:00:42,310
Um But the the thing there, of course, the risk there is

1322
01:00:43,929 --> 01:00:45,360
people organizations,

1323
01:00:45,370 --> 01:00:48,580
countries that have more access to more resources and

1324
01:00:48,590 --> 01:00:51,139
more compute and more data will have their own,

1325
01:00:51,149 --> 01:00:52,949
more powerful foundation models.

1326
01:00:54,050 --> 01:00:57,310
Um And these are going to be very expensive to train

1327
01:00:57,989 --> 01:01:01,419
if you ask me, are they useful? I would say, absolutely.

1328
01:01:01,429 --> 01:01:04,169
Are they always going to be accurate? Absolutely not,

1329
01:01:04,959 --> 01:01:07,689
but they can still be huge productivity,

1330
01:01:08,179 --> 01:01:09,090
they can still,

1331
01:01:09,409 --> 01:01:09,889
you know,

1332
01:01:10,790 --> 01:01:12,810
result in huge productivity gains.

1333
01:01:13,840 --> 01:01:15,030
And I think from a

1334
01:01:16,229 --> 01:01:16,659
uh

1335
01:01:17,250 --> 01:01:18,540
national interest,

1336
01:01:18,989 --> 01:01:19,600
I think

1337
01:01:19,820 --> 01:01:22,979
countries should be investing in this because

1338
01:01:23,209 --> 01:01:24,260
of the potential.

1339
01:01:25,179 --> 01:01:26,939
But is it worth investing

1340
01:01:27,320 --> 01:01:28,899
$7 trillion?

1341
01:01:29,830 --> 01:01:30,760
Like Sam Altman

1342
01:01:30,959 --> 01:01:31,580
says,

1343
01:01:32,120 --> 01:01:34,379
that's a different debate to be had.

1344
01:01:35,020 --> 01:01:35,760
Um

1345
01:01:36,000 --> 01:01:36,719
But, but,

1346
01:01:36,919 --> 01:01:37,600
but I think,

1347
01:01:38,389 --> 01:01:42,489
you know, I wish in the next year or two or at least as soon as possible.

1348
01:01:43,449 --> 01:01:43,979
At least

1349
01:01:44,479 --> 01:01:46,580
the only thing I feel strongly about this is there is

1350
01:01:46,590 --> 01:01:49,649
this whole thing about artificial general intelligence and you know,

1351
01:01:49,659 --> 01:01:51,340
we want, we will get there by

1352
01:01:52,350 --> 01:01:53,469
in five years

1353
01:01:53,780 --> 01:01:56,739
to 10 years. I think that's a big distraction

1354
01:01:58,149 --> 01:02:00,899
given what these tools can already do

1355
01:02:01,000 --> 01:02:02,500
and what these tools can,

1356
01:02:02,790 --> 01:02:06,040
can do in the next few years without that goal, if that is the goal,

1357
01:02:06,050 --> 01:02:08,620
I think people be disappointed, very disappointed.

1358
01:02:08,629 --> 01:02:09,320
In my opinion.

1359
01:02:10,270 --> 01:02:12,860
But if you have more pragmatic goals about

1360
01:02:14,149 --> 01:02:16,800
productivity boost scientific assistance

1361
01:02:17,649 --> 01:02:18,149
and

1362
01:02:19,360 --> 01:02:20,419
you're able to

1363
01:02:20,669 --> 01:02:21,939
work with the

1364
01:02:22,110 --> 01:02:24,899
realization that these things will never be perfect.

1365
01:02:25,340 --> 01:02:27,939
They may not be able to tell you something

1366
01:02:28,110 --> 01:02:29,379
drastically new,

1367
01:02:29,719 --> 01:02:32,060
but they can still be a massive productivity boost.

1368
01:02:32,070 --> 01:02:35,219
I think even within the next few years, we'll see great benefits come out of it.

1369
01:02:35,830 --> 01:02:36,510
Well, I guess

1370
01:02:36,699 --> 01:02:39,449
the previous episode gave an opinion and I

1371
01:02:39,459 --> 01:02:41,260
was interested to see what you think of this

1372
01:02:41,600 --> 01:02:45,780
around the role of academia, the tech sector and start ups

1373
01:02:46,989 --> 01:02:50,489
to achieve this vision you're talking about of, you know, a new platform,

1374
01:02:50,500 --> 01:02:52,389
new technology for scientific discovery.

1375
01:02:53,459 --> 01:02:55,810
What where do you see the roles of all three of them?

1376
01:02:55,820 --> 01:02:59,689
Where does academia fit into this? Where are start ups required? Where are the

1377
01:03:00,120 --> 01:03:01,409
tech companies needed

1378
01:03:01,810 --> 01:03:02,489
or industry?

1379
01:03:03,139 --> 01:03:05,530
This is I'm glad you asked me this because

1380
01:03:06,260 --> 01:03:07,510
one of the

1381
01:03:07,830 --> 01:03:09,949
exciting aspects of the

1382
01:03:10,209 --> 01:03:12,399
platform that I described earlier

1383
01:03:13,340 --> 01:03:14,280
was because

1384
01:03:14,770 --> 01:03:15,560
different

1385
01:03:16,840 --> 01:03:17,500
uh

1386
01:03:18,659 --> 01:03:21,360
kinds of organizations can do different things, right?

1387
01:03:22,219 --> 01:03:25,120
I do not think at least in the next two or three years,

1388
01:03:25,159 --> 01:03:27,000
academia would be doing the pre training

1389
01:03:28,620 --> 01:03:30,129
because I mean, you need

1390
01:03:30,419 --> 01:03:33,689
10,000 GPUs just to get off the ground, right? So

1391
01:03:34,530 --> 01:03:36,350
that's not what academia is

1392
01:03:37,070 --> 01:03:39,090
probably ever going to be good at. But then

1393
01:03:39,800 --> 01:03:41,620
almost everything else that I mentioned,

1394
01:03:42,679 --> 01:03:43,179
right,

1395
01:03:44,090 --> 01:03:45,580
data curation, I mean,

1396
01:03:45,959 --> 01:03:46,949
traditionally

1397
01:03:47,580 --> 01:03:50,479
academia has been keepers of knowledge, right? And this is,

1398
01:03:51,429 --> 01:03:53,209
this is just baked in,

1399
01:03:53,679 --> 01:03:54,080
right?

1400
01:03:54,209 --> 01:03:58,310
Maybe they don't have access to all kinds of data, but the open data, right. So that's

1401
01:03:58,489 --> 01:04:01,090
that is where academia is at its strongest.

1402
01:04:01,100 --> 01:04:04,199
And then let's go down downstream, right? Let's talk about

1403
01:04:04,919 --> 01:04:07,409
being specialized agents. What does academia do?

1404
01:04:07,419 --> 01:04:09,840
Basic research, specialized, narrow.

1405
01:04:10,080 --> 01:04:11,520
So that is the clean role there

1406
01:04:12,159 --> 01:04:12,659
and

1407
01:04:12,979 --> 01:04:15,419
maybe even the fine tuning process. And then,

1408
01:04:15,699 --> 01:04:16,850
you know, if you talk to

1409
01:04:17,709 --> 01:04:19,540
people who are

1410
01:04:20,030 --> 01:04:22,300
developing these science foundation models and

1411
01:04:22,310 --> 01:04:24,600
probably any kind of foundation model

1412
01:04:25,979 --> 01:04:29,449
right now, they're not complaining about when I talk to them,

1413
01:04:29,459 --> 01:04:32,600
they're not complaining about data, they're not complaining about compute.

1414
01:04:33,379 --> 01:04:35,600
They're saying the most important thing is evaluation

1415
01:04:36,639 --> 01:04:37,489
and feedback.

1416
01:04:38,500 --> 01:04:40,320
You know, I can again envision a

1417
01:04:40,560 --> 01:04:41,540
future where

1418
01:04:42,439 --> 01:04:43,340
um

1419
01:04:43,879 --> 01:04:44,679
as part of

1420
01:04:44,909 --> 01:04:45,620
many

1421
01:04:45,800 --> 01:04:49,979
classes that masters or PhD students or undergraduate students take

1422
01:04:50,389 --> 01:04:50,989
where

1423
01:04:51,239 --> 01:04:54,580
they may be evaluating or fine tuning or creating data,

1424
01:04:54,919 --> 01:04:55,439
right?

1425
01:04:56,270 --> 01:04:57,139
Uh

1426
01:04:57,360 --> 01:04:58,100
So,

1427
01:04:58,659 --> 01:05:01,060
and this kind of expertise and and

1428
01:05:01,459 --> 01:05:03,330
human scale only exists

1429
01:05:03,489 --> 01:05:06,169
for for these kind of tasks only exist in academia.

1430
01:05:06,500 --> 01:05:10,100
And of course, there's a lot of basic research needed to improve architectures

1431
01:05:10,520 --> 01:05:11,649
to uh

1432
01:05:12,310 --> 01:05:14,560
as again, as as people say,

1433
01:05:16,280 --> 01:05:17,060
coming of it,

1434
01:05:17,939 --> 01:05:19,939
much better ways of

1435
01:05:20,350 --> 01:05:23,959
much better, much better ways than just complete the next word

1436
01:05:24,699 --> 01:05:29,100
uh this iterative process. So this is where academia has, it has its strongest.

1437
01:05:29,110 --> 01:05:31,469
So except that one pre training block,

1438
01:05:31,879 --> 01:05:34,489
I think academia is in a nice position to contribute

1439
01:05:35,459 --> 01:05:36,830
e everybody else

1440
01:05:37,590 --> 01:05:38,570
and then you have

1441
01:05:38,939 --> 01:05:40,570
national laboratories

1442
01:05:40,850 --> 01:05:41,439
uh

1443
01:05:42,000 --> 01:05:43,370
who are very well placed

1444
01:05:44,169 --> 01:05:46,350
uh for certain parts of,

1445
01:05:46,639 --> 01:05:48,669
you know, this, this this ecosystem.

1446
01:05:49,560 --> 01:05:50,379
Um

1447
01:05:50,729 --> 01:05:51,659
how do

1448
01:05:51,840 --> 01:05:53,820
companies come in

1449
01:05:54,800 --> 01:05:55,340
that

1450
01:05:55,739 --> 01:05:58,459
complicates things a little bit because

1451
01:05:59,379 --> 01:06:00,429
mostly

1452
01:06:00,620 --> 01:06:01,800
these are for profit

1453
01:06:03,909 --> 01:06:07,770
and so they have their own goals and some of those goals, thankfully,

1454
01:06:07,780 --> 01:06:10,760
I think overlap with some of these broader ambitions we have

1455
01:06:11,949 --> 01:06:13,330
and certain companies,

1456
01:06:13,679 --> 01:06:16,419
it is in their best interest to sell more GPUs.

1457
01:06:17,070 --> 01:06:17,969
So I'm sure I'm

1458
01:06:18,189 --> 01:06:22,929
sure they can help in, in, in, in addressing part of the research and the

1459
01:06:23,770 --> 01:06:26,820
and the computing resource kind of spectrum.

1460
01:06:27,429 --> 01:06:29,199
Uh So,

1461
01:06:29,649 --> 01:06:30,610
so I think

1462
01:06:31,590 --> 01:06:33,229
all these organizations

1463
01:06:33,929 --> 01:06:35,689
can have a very clear

1464
01:06:35,800 --> 01:06:36,719
role in

1465
01:06:36,969 --> 01:06:39,750
this entire process, entire platform.

1466
01:06:40,300 --> 01:06:41,469
And that is again,

1467
01:06:42,139 --> 01:06:45,010
very exciting to me, right? Even even companies that are,

1468
01:06:45,679 --> 01:06:46,179
that have

1469
01:06:47,270 --> 01:06:49,530
different goals than academia

1470
01:06:49,909 --> 01:06:50,760
and national labs,

1471
01:06:50,770 --> 01:06:53,250
which have slightly different goals than academia and companies,

1472
01:06:54,409 --> 01:06:54,550
I

1473
01:06:54,739 --> 01:06:57,459
think there is still so much common ground

1474
01:06:58,020 --> 01:06:59,729
that maps to like one

1475
01:06:59,870 --> 01:07:01,669
part of the platform or the other.

1476
01:07:03,260 --> 01:07:07,169
I don't know if you have this similar thought, but I do often wonder, I mean,

1477
01:07:07,179 --> 01:07:08,280
I don't envy

1478
01:07:08,520 --> 01:07:10,770
the people at the top of Siemens or Dassault

1479
01:07:11,120 --> 01:07:13,149
or Ansys or Cadence

1480
01:07:13,439 --> 01:07:16,550
who are probably faced with a a complication.

1481
01:07:17,090 --> 01:07:17,909
Do I

1482
01:07:18,929 --> 01:07:22,750
bet the House essentially on, you know,

1483
01:07:22,760 --> 01:07:27,550
really going in on AI and machine learning and this idea of

1484
01:07:27,939 --> 01:07:30,229
scientific foundation models because

1485
01:07:30,870 --> 01:07:31,489
you know, a

1486
01:07:31,629 --> 01:07:34,050
lot of scientific discovery is done using

1487
01:07:34,770 --> 01:07:37,479
their billions of dollars worth of software packages, you know,

1488
01:07:37,489 --> 01:07:39,080
throughout engineering and science

1489
01:07:39,729 --> 01:07:40,469
or

1490
01:07:41,040 --> 01:07:41,870
do we

1491
01:07:42,189 --> 01:07:43,219
continue

1492
01:07:43,560 --> 01:07:48,300
and make even better these higher fidelity, you know, physics based simulators?

1493
01:07:48,439 --> 01:07:49,810
Do you know to me that like

1494
01:07:50,290 --> 01:07:51,510
there was not,

1495
01:07:52,879 --> 01:07:55,659
well, I guess there is in large language models, there were companies

1496
01:07:57,239 --> 01:08:00,679
already doing things and I guess Adobe and Microsoft and others are, are in there.

1497
01:08:00,689 --> 01:08:01,699
But just give my point,

1498
01:08:01,709 --> 01:08:05,570
what role do you see for these massive companies already in the

1499
01:08:05,699 --> 01:08:10,889
simulation space who arguably have the most to gain or lose from

1500
01:08:11,179 --> 01:08:13,090
um these new methods?

1501
01:08:14,320 --> 01:08:14,850
Um

1502
01:08:15,050 --> 01:08:18,100
I'm glad you just asked, you just restricted your

1503
01:08:18,229 --> 01:08:19,279
domain to

1504
01:08:19,490 --> 01:08:20,009
simulation.

1505
01:08:21,069 --> 01:08:23,339
So that's easier to answer.

1506
01:08:23,859 --> 01:08:27,529
If they bet the House on foundation models, I think it would be foolish.

1507
01:08:27,540 --> 01:08:30,540
I think if they bet the house purely on AI, that would also be

1508
01:08:30,660 --> 01:08:31,299
foolish.

1509
01:08:31,959 --> 01:08:33,410
But if they ignore this,

1510
01:08:33,589 --> 01:08:34,979
that's even more foolish.

1511
01:08:35,899 --> 01:08:38,580
So, so that is, I think a happy medium where

1512
01:08:39,879 --> 01:08:43,790
whatever you've been doing and whatever you are very good at

1513
01:08:44,899 --> 01:08:49,240
those can be agents. Remember, agents don't just have to be AI agents, right?

1514
01:08:49,580 --> 01:08:51,310
An agent could be your

1515
01:08:52,189 --> 01:08:54,250
uh beautiful

1516
01:08:54,770 --> 01:08:55,569
solver,

1517
01:08:56,040 --> 01:08:56,450
right?

1518
01:08:57,430 --> 01:08:58,839
And it may

1519
01:08:59,000 --> 01:09:01,200
maybe there is an AI angle to it,

1520
01:09:01,319 --> 01:09:02,729
but it doesn't matter. But I think,

1521
01:09:03,950 --> 01:09:05,410
you know, I mentioned earlier about

1522
01:09:07,370 --> 01:09:08,109
uh

1523
01:09:08,560 --> 01:09:10,000
probing an evaluation

1524
01:09:10,459 --> 01:09:12,959
if there is something that's fairly trustworthy

1525
01:09:13,160 --> 01:09:15,600
like your solver that you've developed for 20 years,

1526
01:09:16,459 --> 01:09:19,709
that can play an enormously useful role in this

1527
01:09:19,819 --> 01:09:21,149
evaluation process.

1528
01:09:21,600 --> 01:09:22,319
You know, the

1529
01:09:22,600 --> 01:09:26,680
foundation model can generate a prior that can be evaluated or fine tuned,

1530
01:09:27,120 --> 01:09:29,220
right and brought back in. So

1531
01:09:29,430 --> 01:09:33,229
whatever you have been good, I don't think AI fundamentally changes

1532
01:09:34,069 --> 01:09:37,109
many things that we have been doing. In fact, I think it can

1533
01:09:37,589 --> 01:09:38,129
help

1534
01:09:39,149 --> 01:09:41,169
drive more value

1535
01:09:41,629 --> 01:09:45,890
also because AI on its own has many shortcomings that cannot be addressed

1536
01:09:46,709 --> 01:09:47,229
using

1537
01:09:47,470 --> 01:09:47,529
data

1538
01:09:47,770 --> 01:09:48,640
alone. So,

1539
01:09:49,209 --> 01:09:49,959
so yes,

1540
01:09:50,479 --> 01:09:52,620
I mean for those simulation companies,

1541
01:09:53,089 --> 01:09:54,729
um you know, being

1542
01:09:55,839 --> 01:09:56,669
uh

1543
01:09:57,759 --> 01:09:59,979
cognizant of some of the developments

1544
01:10:00,470 --> 01:10:01,259
and then

1545
01:10:01,740 --> 01:10:04,620
even taking open source models and

1546
01:10:05,339 --> 01:10:07,819
maybe you don't want to spend enormous amount

1547
01:10:07,830 --> 01:10:10,359
of computer on on this pre training stage.

1548
01:10:10,370 --> 01:10:10,680
But

1549
01:10:11,819 --> 01:10:15,160
uh collaborating with institutions like I mentioned

1550
01:10:15,459 --> 01:10:16,759
earlier, right, if we have

1551
01:10:17,479 --> 01:10:19,339
an Open science foundation model,

1552
01:10:20,540 --> 01:10:21,240
then

1553
01:10:21,470 --> 01:10:22,270
maybe

1554
01:10:23,089 --> 01:10:24,069
Stevens

1555
01:10:24,319 --> 01:10:27,990
can provide you shouldn't name particular names, but

1556
01:10:28,250 --> 01:10:30,390
the particular companies can provide

1557
01:10:30,729 --> 01:10:31,810
services

1558
01:10:32,109 --> 01:10:34,470
in the context of the tools that they already provide,

1559
01:10:35,490 --> 01:10:36,020
right?

1560
01:10:36,029 --> 01:10:39,140
So I think there are a lot of opportunities there at the intersection

1561
01:10:39,149 --> 01:10:42,430
of what you're already good at with all of these things happening.

1562
01:10:43,149 --> 01:10:45,169
And of course, some other companies that

1563
01:10:45,629 --> 01:10:48,450
I think certain certain simulation companies that

1564
01:10:48,459 --> 01:10:50,810
are already part of a bigger conglomerate.

1565
01:10:51,819 --> 01:10:55,490
I am absolutely sure every company has this in their vision

1566
01:10:55,919 --> 01:10:57,270
and they may have their own

1567
01:10:57,689 --> 01:10:59,750
context specific uh

1568
01:11:00,000 --> 01:11:01,120
models that

1569
01:11:01,299 --> 01:11:02,720
will be interface. So

1570
01:11:03,069 --> 01:11:07,490
the short answer to the question is. Yeah. Don't bet the house fully on this. But,

1571
01:11:08,089 --> 01:11:08,629
uh,

1572
01:11:09,399 --> 01:11:12,040
it's not something that can be ignored either,

1573
01:11:12,589 --> 01:11:14,819
but at the same time I'm sure there is completely

1574
01:11:15,100 --> 01:11:18,100
unreal, unrealistic expectations all over the place from

1575
01:11:18,250 --> 01:11:19,319
customers and

1576
01:11:20,299 --> 01:11:20,930
management.

1577
01:11:21,600 --> 01:11:21,830
Yeah.

1578
01:11:21,839 --> 01:11:25,740
It's kind of interesting if I speak to some of my friends who work in industry and

1579
01:11:27,040 --> 01:11:30,180
it's hard to know because sometimes you don't know whether you're in the bubble,

1580
01:11:30,310 --> 01:11:33,100
you know, by being in a certain thing. But

1581
01:11:33,310 --> 01:11:38,049
they are almost to the point where they're sick of people pitching stuff to them.

1582
01:11:38,330 --> 01:11:39,859
And, you know, it's almost,

1583
01:11:40,529 --> 01:11:42,310
you know, the classic response is, well, you know,

1584
01:11:42,319 --> 01:11:45,709
we've had reduced order models for decades, you know, you're just rebranding this.

1585
01:11:45,720 --> 01:11:45,970
So there

1586
01:11:46,450 --> 01:11:50,549
is, I think the danger of some of the hype of AI in general

1587
01:11:51,430 --> 01:11:55,290
and if you go in too early into a field, promising the world,

1588
01:11:55,540 --> 01:11:58,890
it can sometimes burn trust or it's a do,

1589
01:11:58,899 --> 01:12:01,410
do you see that sometimes in your discussions?

1590
01:12:01,770 --> 01:12:04,649
Yeah, for sure. And, and, and I don't know if, you know, I

1591
01:12:04,790 --> 01:12:06,529
have a start up as well and

1592
01:12:06,740 --> 01:12:07,319
it's called

1593
01:12:07,609 --> 01:12:09,529
Geminus.AI, we've been going for about

1594
01:12:09,729 --> 01:12:10,680
five years

1595
01:12:10,879 --> 01:12:15,770
and even before the foundation models, we, we had the same kind of challenges,

1596
01:12:15,779 --> 01:12:15,959
right?

1597
01:12:15,970 --> 01:12:16,770
People think

1598
01:12:18,020 --> 01:12:18,729
it's like

1599
01:12:19,100 --> 01:12:21,879
very little data, you can create magic.

1600
01:12:22,339 --> 01:12:23,959
Uh But I think

1601
01:12:24,410 --> 01:12:28,029
what has worked for us right from the beginning is by

1602
01:12:28,529 --> 01:12:30,319
setting these expectations

1603
01:12:30,770 --> 01:12:31,919
and saying hey,

1604
01:12:32,779 --> 01:12:33,459
this can,

1605
01:12:34,140 --> 01:12:38,560
I mean, just be very clear about what it can do well and what it could do

1606
01:12:39,200 --> 01:12:40,569
and what it cannot do

1607
01:12:40,930 --> 01:12:42,259
and being very clear

1608
01:12:43,140 --> 01:12:48,279
and you know, being honest about it, I think, I think there is a, there is a way here,

1609
01:12:48,930 --> 01:12:50,520
but you're right. I mean, there is

1610
01:12:51,140 --> 01:12:54,560
we have faced some customer scenarios where

1611
01:12:55,439 --> 01:12:56,660
understanding of

1612
01:12:57,209 --> 01:13:00,000
the capabilities of this technology is so unrealistic.

1613
01:13:00,529 --> 01:13:01,140
Um

1614
01:13:02,080 --> 01:13:03,500
but it's an iterative process

1615
01:13:03,660 --> 01:13:04,640
ultimately.

1616
01:13:05,100 --> 01:13:06,419
If you're able to show

1617
01:13:07,169 --> 01:13:07,629
that

1618
01:13:08,529 --> 01:13:12,430
there is something that can be done at lower cost or, you know, in,

1619
01:13:12,439 --> 01:13:14,819
in quicker time with reasonable accuracy,

1620
01:13:15,640 --> 01:13:17,939
then that iterative process, you know,

1621
01:13:18,180 --> 01:13:21,779
I think will take hold ultimately, right? Nobody's going to argue against

1622
01:13:22,240 --> 01:13:25,299
actual utility, right? It's the perceived utility

1623
01:13:25,680 --> 01:13:27,540
and hype that

1624
01:13:28,509 --> 01:13:29,879
uh cloud things up.

1625
01:13:30,669 --> 01:13:31,410
Yeah. Yeah.

1626
01:13:31,859 --> 01:13:33,939
Um One of the things

1627
01:13:34,399 --> 01:13:38,410
I've tried it in all the episodes where possible is

1628
01:13:39,529 --> 01:13:42,870
a bit of, well, career advice, I guess. And

1629
01:13:43,620 --> 01:13:44,279
you've

1630
01:13:44,700 --> 01:13:49,990
rose to be a very prominent and successful professor at leading university.

1631
01:13:50,000 --> 01:13:51,750
You have your own start up. Your

1632
01:13:52,910 --> 01:13:56,870
many people look at people like you and they often wonder, well,

1633
01:13:56,879 --> 01:13:58,390
how did you get there?

1634
01:13:58,399 --> 01:13:59,689
How can I get

1635
01:14:00,120 --> 01:14:00,740
there?

1636
01:14:00,970 --> 01:14:01,450
Um

1637
01:14:02,560 --> 01:14:05,080
This is a difficult question to ask because I

1638
01:14:05,089 --> 01:14:07,709
know it's very dependent on people's roots and everything.

1639
01:14:08,109 --> 01:14:11,810
But what have you learned along the way? If people are wanting to go

1640
01:14:12,899 --> 01:14:16,040
um into academia, they're wanting to rise, they think. Oh,

1641
01:14:17,970 --> 01:14:19,799
would you do the same as you did and the

1642
01:14:19,810 --> 01:14:22,509
paths that you would take differently as advice you would

1643
01:14:23,189 --> 01:14:24,109
give

1644
01:14:24,799 --> 01:14:27,529
if you, OK, let me frame it. You're a PhD student. Now,

1645
01:14:28,419 --> 01:14:31,220
you're already doing a PhD, let's say, in fluid dynamics.

1646
01:14:31,229 --> 01:14:34,120
And you would like to become a full professor one day.

1647
01:14:34,129 --> 01:14:36,299
What, what would be your advice to that

1648
01:14:36,689 --> 01:14:37,629
PhD student?

1649
01:14:37,990 --> 01:14:38,799
So that's

1650
01:14:39,080 --> 01:14:39,089
a

1651
01:14:39,509 --> 01:14:42,290
question to answer then, the more broader question, although

1652
01:14:42,390 --> 01:14:44,009
I have been asked that question

1653
01:14:44,149 --> 01:14:45,970
quite a few times, right? So,

1654
01:14:46,220 --> 01:14:46,939
so I think

1655
01:14:47,410 --> 01:14:51,890
I did the first part very briefly and then get to the second part, right? For me,

1656
01:14:52,810 --> 01:14:54,589
the only thing that has

1657
01:14:54,910 --> 01:14:56,990
driven me is curiosity

1658
01:14:58,290 --> 01:15:00,770
and, you know, not defining a

1659
01:15:01,220 --> 01:15:04,310
control volume, saying this is what I'm interested in

1660
01:15:04,799 --> 01:15:09,149
but being intensely curious and then slowly growing the control volume

1661
01:15:09,660 --> 01:15:10,479
to come

1662
01:15:10,600 --> 01:15:11,660
for certain things. And

1663
01:15:11,950 --> 01:15:12,600
none of,

1664
01:15:13,129 --> 01:15:16,779
I mean, if you say I'm successful, then in your view, I'm successful.

1665
01:15:16,790 --> 01:15:17,990
But I don't define

1666
01:15:18,250 --> 01:15:21,229
things that way. But what has worked for me is

1667
01:15:21,390 --> 01:15:23,750
just pursuing the curiosity

1668
01:15:24,180 --> 01:15:27,640
but also not forgetting what needs to get done

1669
01:15:28,740 --> 01:15:31,120
on the site, right? To keep the trains running on time.

1670
01:15:31,850 --> 01:15:35,859
Uh But yes, what advice would I give my PhD students? Uh

1671
01:15:38,350 --> 01:15:42,750
Maybe I'll, I'll interject another question. I get asked a lot, right?

1672
01:15:42,759 --> 01:15:43,970
When people ask about,

1673
01:15:45,459 --> 01:15:46,509
I get asked this, I mean,

1674
01:15:46,520 --> 01:15:51,520
I've given about 20 talks over the past few months just on these AI topics and

1675
01:15:51,799 --> 01:15:53,060
this question comes up,

1676
01:15:53,279 --> 01:15:55,640
you know, I want to specialize in AI for science

1677
01:15:56,439 --> 01:16:02,140
or my group ones. Uh So where should we start? And what should we be focusing on?

1678
01:16:02,540 --> 01:16:04,169
My first answer always is

1679
01:16:04,330 --> 01:16:06,180
get really good at the science

1680
01:16:07,149 --> 01:16:10,500
because the AI is not going to solve any scientific problem

1681
01:16:11,009 --> 01:16:14,259
to a greater degree than what the expert solves. But

1682
01:16:14,720 --> 01:16:16,620
if you have a very good understanding of,

1683
01:16:17,040 --> 01:16:18,259
of the domain

1684
01:16:18,660 --> 01:16:22,410
and you do not lose any rigor there and then you pick up the essentials of AI,

1685
01:16:22,799 --> 01:16:24,129
then it can help you

1686
01:16:24,459 --> 01:16:26,029
be much more productive

1687
01:16:26,310 --> 01:16:27,379
and uh

1688
01:16:27,620 --> 01:16:30,200
you know, hopefully like leverage it in the right way. So,

1689
01:16:30,430 --> 01:16:33,240
so I think I would give the same kind of advice.

1690
01:16:33,890 --> 01:16:34,709
Um

1691
01:16:34,830 --> 01:16:35,379
You know,

1692
01:16:35,689 --> 01:16:38,509
the foundations are the most important pieces

1693
01:16:39,620 --> 01:16:41,350
you're, you know, losing,

1694
01:16:42,319 --> 01:16:46,959
I mean, without losing rigor in, in, in, in, in mathematics, physics,

1695
01:16:47,500 --> 01:16:50,580
chemistry and biology, where, where, where it matters.

1696
01:16:51,240 --> 01:16:51,770
Um

1697
01:16:52,620 --> 01:16:53,830
then every

1698
01:16:54,069 --> 01:16:56,870
AI and all of these techniques that come about, right,

1699
01:16:57,020 --> 01:16:57,479
are

1700
01:16:58,350 --> 01:17:02,470
basically layers that are built by combining these different different nodes.

1701
01:17:03,529 --> 01:17:04,209
And

1702
01:17:04,720 --> 01:17:08,750
um I think for those who have that perspective

1703
01:17:09,189 --> 01:17:13,890
and have a solid foundation, I think moving into new areas is, is, is, is,

1704
01:17:14,620 --> 01:17:15,629
is much easier.

1705
01:17:16,799 --> 01:17:17,290
But

1706
01:17:17,459 --> 01:17:19,819
I think your question certainly has

1707
01:17:19,930 --> 01:17:23,979
one important aspect that I want to highlight, right? So

1708
01:17:24,279 --> 01:17:25,359
if you think about

1709
01:17:25,649 --> 01:17:27,430
how academia has changed

1710
01:17:27,899 --> 01:17:31,020
or even nature of research has changed in the past two decades.

1711
01:17:31,640 --> 01:17:32,330
Um

1712
01:17:33,720 --> 01:17:35,410
when I was a graduate student,

1713
01:17:35,500 --> 01:17:40,490
everybody pretty much in my line of work would write a solver from scratch,

1714
01:17:40,759 --> 01:17:42,410
right. So that has changed completely,

1715
01:17:43,430 --> 01:17:45,950
you know, one PhD would be running an le

1716
01:17:46,089 --> 01:17:47,060
properly.

1717
01:17:47,549 --> 01:17:48,140
That was, I

1718
01:17:48,529 --> 01:17:50,379
think at the

1719
01:17:50,569 --> 01:17:51,290
Stanford

1720
01:17:51,790 --> 01:17:53,020
would say that even like

1721
01:17:53,410 --> 01:17:54,680
10 years ago.

1722
01:17:54,790 --> 01:17:56,339
So all of that is changing.

1723
01:17:57,120 --> 01:17:57,819
So,

1724
01:17:58,040 --> 01:18:03,060
and to be a faculty member, to be a, a professor at the university like Michigan,

1725
01:18:03,310 --> 01:18:05,419
you were the expert in your domain

1726
01:18:05,939 --> 01:18:08,540
and you had a big impact in the field that has changed

1727
01:18:08,959 --> 01:18:10,540
AI or not, right? Because

1728
01:18:10,750 --> 01:18:12,250
globally, there is just

1729
01:18:13,299 --> 01:18:16,479
lot of researchers, you know, looking at these topics. So

1730
01:18:17,850 --> 01:18:21,009
if your foundations are very strong and this is not just for academia,

1731
01:18:21,020 --> 01:18:23,859
this is for industry and national labs as well.

1732
01:18:23,870 --> 01:18:27,189
If your foundations are very strong and then you're curious then

1733
01:18:28,520 --> 01:18:29,979
uh without losing that

1734
01:18:30,629 --> 01:18:32,330
core aspects,

1735
01:18:33,479 --> 01:18:37,430
being able to navigate into newer and newer areas and adapt and be flexible,

1736
01:18:37,439 --> 01:18:38,569
I think that becomes

1737
01:18:38,870 --> 01:18:39,569
easier.

1738
01:18:40,240 --> 01:18:42,490
And my PhD students,

1739
01:18:43,069 --> 01:18:44,609
you know, II I

1740
01:18:45,399 --> 01:18:48,709
I called the PhD program at least in my lab as a,

1741
01:18:48,720 --> 01:18:51,859
as a training exercise to equip you with the skills

1742
01:18:51,970 --> 01:18:52,930
rather than

1743
01:18:53,109 --> 01:18:58,060
just to like solve a problem from end to end. So gain as many skills as possible.

1744
01:18:58,549 --> 01:19:01,379
But the most important thing is, is, is the rigor that

1745
01:19:02,819 --> 01:19:03,430
um

1746
01:19:03,830 --> 01:19:06,140
makes you see things and connections

1747
01:19:06,589 --> 01:19:08,890
and makes you navigate those

1748
01:19:09,240 --> 01:19:09,750
as

1749
01:19:10,299 --> 01:19:12,919
seamlessly as possible, very rarely meet

1750
01:19:13,379 --> 01:19:15,850
academics at the top of that profession who aren't

1751
01:19:16,169 --> 01:19:20,750
genuinely excited and interested and, and feel almost like,

1752
01:19:22,000 --> 01:19:23,750
you know, they're running their own start up.

1753
01:19:23,759 --> 01:19:26,410
I know you also have a start up but, you know, you, you kind of,

1754
01:19:26,799 --> 01:19:28,520
compared to an industry,

1755
01:19:28,529 --> 01:19:33,270
I speak to far more people who are a little bit miserable and sort of depressed

1756
01:19:33,370 --> 01:19:35,910
and I know academic academia has many problems

1757
01:19:35,919 --> 01:19:37,770
but normally when I speak to an academic,

1758
01:19:37,779 --> 01:19:39,140
they are still genuinely

1759
01:19:39,549 --> 01:19:43,100
reasonably positive um about at

1760
01:19:43,290 --> 01:19:46,009
least the flexibility they have and the ability to set

1761
01:19:46,279 --> 01:19:47,979
sort of their directions.

1762
01:19:48,290 --> 01:19:51,910
Yeah, it is like I never left grad school or sometimes I've never

1763
01:19:52,060 --> 01:19:52,290
left

1764
01:19:52,430 --> 01:19:55,209
kindergarten, right. I get to play with these amazing

1765
01:19:55,549 --> 01:19:59,259
tos and tools and get to collaborate with people. It's

1766
01:19:59,729 --> 01:20:01,339
just an amazing place to be, but

1767
01:20:01,620 --> 01:20:04,450
it's not without its challenges, right? Especially the first

1768
01:20:05,120 --> 01:20:09,640
seven or eight years in academia can be extremely stressful for some people,

1769
01:20:09,649 --> 01:20:10,419
not everybody.

1770
01:20:11,310 --> 01:20:13,020
And of course,

1771
01:20:13,319 --> 01:20:14,609
the move from postdoc

1772
01:20:14,850 --> 01:20:15,319
to

1773
01:20:16,660 --> 01:20:18,160
assistant professor

1774
01:20:18,520 --> 01:20:20,339
is also very competitive.

1775
01:20:20,689 --> 01:20:21,439
Um

1776
01:20:22,759 --> 01:20:26,660
and it's maybe not for everybody. But I think for like you said, if you are

1777
01:20:27,299 --> 01:20:31,600
genuinely driven by curiosity and passion and you are um

1778
01:20:33,569 --> 01:20:34,439
you know, you, you're,

1779
01:20:34,669 --> 01:20:36,459
you're paying your bills

1780
01:20:36,709 --> 01:20:40,229
for things that you have to pay your bills. I think it's incredibly rewarding

1781
01:20:40,509 --> 01:20:40,560
career.

1782
01:20:42,129 --> 01:20:43,470
So maybe as

1783
01:20:43,779 --> 01:20:49,069
a final question, as a bit of an outlook to the, to the future

1784
01:20:50,399 --> 01:20:52,549
and I know these are always hard. So maybe more

1785
01:20:52,680 --> 01:20:54,600
thematically where, where you see it

1786
01:20:54,709 --> 01:21:00,060
if you were to fast forward now and you're doing your symposium in five years time,

1787
01:21:01,229 --> 01:21:04,049
what do you think will have been the big achievements by them?

1788
01:21:04,060 --> 01:21:07,700
What do you think will be, will there still be the gen AI hype?

1789
01:21:07,709 --> 01:21:10,990
And it will keep going will have there been in the winter of AI and

1790
01:21:11,290 --> 01:21:15,729
people will be moving on to new things like quantum or where do, where do you see

1791
01:21:16,669 --> 01:21:17,899
in five years time though

1792
01:21:18,470 --> 01:21:20,169
the academic world around,

1793
01:21:21,399 --> 01:21:23,620
let's say, let's restrict it to fluid dynamics just,

1794
01:21:23,629 --> 01:21:25,410
just to make it a little bit easier to

1795
01:21:26,029 --> 01:21:26,689
predict.

1796
01:21:26,910 --> 01:21:30,959
Yeah, first of all, I mean, even next year, it won't be my symposium.

1797
01:21:30,970 --> 01:21:32,319
It could be the community symposium.

1798
01:21:32,580 --> 01:21:32,589
I

1799
01:21:32,689 --> 01:21:34,080
think a whole bunch of

1800
01:21:34,259 --> 01:21:36,439
us together, we come together to organize it.

1801
01:21:37,009 --> 01:21:38,330
Yeah, but, but uh

1802
01:21:38,709 --> 01:21:40,000
you know, II I,

1803
01:21:41,370 --> 01:21:44,379
especially in AI, right, predictions never

1804
01:21:44,770 --> 01:21:46,549
go to plan.

1805
01:21:47,000 --> 01:21:50,490
You know, I think in 1971 of the most famous AI DS such as ever,

1806
01:21:50,500 --> 01:21:53,669
Marvin Minsky of MIT who started their AI lab,

1807
01:21:53,890 --> 01:21:55,709
he said in, in 3 to 8 years,

1808
01:21:56,069 --> 01:21:59,180
you'll have a machine with general intelligence of an average human being.

1809
01:21:59,839 --> 01:22:00,479
And

1810
01:22:00,649 --> 01:22:02,759
I think in 1965 they said within

1811
01:22:03,129 --> 01:22:04,149
six years,

1812
01:22:04,799 --> 01:22:05,850
we will have

1813
01:22:06,029 --> 01:22:09,220
a computer that will be the best human chess player.

1814
01:22:10,080 --> 01:22:10,779
Um

1815
01:22:10,890 --> 01:22:12,220
So in that sense,

1816
01:22:13,080 --> 01:22:13,810
uh

1817
01:22:14,310 --> 01:22:17,899
I would say, I mean, I think these things will happen at some point.

1818
01:22:18,410 --> 01:22:18,759
But

1819
01:22:19,740 --> 01:22:22,240
what I, what I have learned

1820
01:22:22,399 --> 01:22:24,509
from listening to many party leaders,

1821
01:22:24,520 --> 01:22:27,240
including Jeff Hinton and all of these people.

1822
01:22:27,879 --> 01:22:30,339
Yeah, thought leaders tend to compress,

1823
01:22:31,069 --> 01:22:34,740
compress time frames a lot, right? So I will not

1824
01:22:35,169 --> 01:22:35,930
give you

1825
01:22:36,950 --> 01:22:39,959
uh precise answers on what will happen in five years.

1826
01:22:40,899 --> 01:22:42,200
But uh

1827
01:22:42,740 --> 01:22:43,520
I feel

1828
01:22:44,620 --> 01:22:45,149
uh

1829
01:22:45,870 --> 01:22:48,209
there may be one big leap

1830
01:22:48,750 --> 01:22:49,240
in

1831
01:22:49,379 --> 01:22:51,009
the capabilities of

1832
01:22:51,399 --> 01:22:53,129
modules like GPT-4.

1833
01:22:53,839 --> 01:22:54,580
Um

1834
01:22:56,100 --> 01:23:00,060
But I don't know if it would be as big as going from zero to GPT-3.5.

1835
01:23:00,470 --> 01:23:00,919
But,

1836
01:23:01,089 --> 01:23:02,620
but there'll be one big leap there.

1837
01:23:03,379 --> 01:23:06,589
Um But in, in, in more restricted domains like D

1838
01:23:06,850 --> 01:23:07,720
modelling, I think

1839
01:23:09,169 --> 01:23:10,859
again, maybe this is more of a hope.

1840
01:23:11,370 --> 01:23:13,700
There is no two camps, there is only one camp

1841
01:23:14,859 --> 01:23:18,439
and because it's a turbulence modeling, there should only be turbulence modelers

1842
01:23:18,770 --> 01:23:22,350
who use these tools better. There is no like AI people and

1843
01:23:22,720 --> 01:23:23,919
traditional modelers.

1844
01:23:24,140 --> 01:23:26,200
So I think it's already been

1845
01:23:26,339 --> 01:23:28,950
slowly like coming together because

1846
01:23:29,430 --> 01:23:31,209
I mean, I'll be honest, right?

1847
01:23:31,220 --> 01:23:34,839
In 2012, I could write anything and I could, it could be published

1848
01:23:35,520 --> 01:23:38,700
um in this topic because it was just new.

1849
01:23:39,770 --> 01:23:43,060
And then slowly uh people had to show

1850
01:23:44,089 --> 01:23:44,589
that

1851
01:23:44,750 --> 01:23:46,129
I'm not just doing a periodic

1852
01:23:46,290 --> 01:23:47,870
learning, I'm being more consistent

1853
01:23:48,640 --> 01:23:52,259
and then almost every paper that people are working on now,

1854
01:23:52,270 --> 01:23:55,259
it has to show some level of generalization

1855
01:23:55,390 --> 01:23:56,819
which I was not doing

1856
01:23:57,240 --> 01:23:58,220
in 2013,

1857
01:23:58,549 --> 01:24:02,120
I was training and testing on same or very similar things.

1858
01:24:02,509 --> 01:24:04,740
So in that sense, the field is moving

1859
01:24:05,859 --> 01:24:07,209
but I still have not.

1860
01:24:07,310 --> 01:24:07,919
And

1861
01:24:08,100 --> 01:24:10,879
because of the nature of how funding works,

1862
01:24:11,990 --> 01:24:13,089
you know, if you get like

1863
01:24:14,580 --> 01:24:16,560
$500,000 from NSF,

1864
01:24:16,569 --> 01:24:19,830
you're addressing a very specific question and you will publish papers

1865
01:24:20,009 --> 01:24:22,000
to address that very specific question.

1866
01:24:22,500 --> 01:24:26,930
You know, we need a broader thing like in the US, we have things called new

1867
01:24:27,180 --> 01:24:29,200
multi universities admissions.

1868
01:24:30,180 --> 01:24:31,680
So it's like

1869
01:24:32,089 --> 01:24:34,279
order of 77 $8 million.

1870
01:24:34,290 --> 01:24:36,580
So there now you're getting different kinds of

1871
01:24:36,589 --> 01:24:39,029
people together and not just developing methods,

1872
01:24:39,339 --> 01:24:42,669
you want to develop some solutions and that still has not happened

1873
01:24:45,029 --> 01:24:48,339
again, maybe that's an excuse, you know, my group has had it

1874
01:24:49,470 --> 01:24:53,319
enough different projects that maybe we could have done something like that, but

1875
01:24:53,720 --> 01:24:56,930
to have more unified uh

1876
01:24:57,669 --> 01:25:00,250
projects and collaborations where

1877
01:25:00,620 --> 01:25:04,129
the goal is to actually create better models rather than

1878
01:25:04,390 --> 01:25:05,569
showing what

1879
01:25:06,060 --> 01:25:07,669
your particular contribution

1880
01:25:08,000 --> 01:25:08,709
is doing.

1881
01:25:09,189 --> 01:25:10,549
So, so I think that

1882
01:25:10,930 --> 01:25:12,410
I think that will happen because

1883
01:25:13,569 --> 01:25:16,910
even in terms of publishing and even in terms of funding so far,

1884
01:25:17,709 --> 01:25:20,169
you being able to show a few things

1885
01:25:20,779 --> 01:25:25,009
has been good enough because the field didn't quite exist or mature.

1886
01:25:25,350 --> 01:25:28,330
But as with any field, once some of the lower

1887
01:25:28,490 --> 01:25:29,649
hanging fruit have been picked,

1888
01:25:30,540 --> 01:25:31,200
then

1889
01:25:31,330 --> 01:25:36,240
the demand, the market would, would desire actual progress.

1890
01:25:36,890 --> 01:25:37,509
And

1891
01:25:37,859 --> 01:25:39,799
I'm pretty sure in five years,

1892
01:25:41,220 --> 01:25:44,350
at least the mark. It's very clear what the market desires

1893
01:25:44,750 --> 01:25:48,830
and I think it will drive it towards the models. I don't, I still don't think

1894
01:25:50,399 --> 01:25:51,890
data driven or not. Uh

1895
01:25:52,890 --> 01:25:53,549
uh

1896
01:25:53,870 --> 01:25:56,339
general generalisable turbulence modeling

1897
01:25:56,439 --> 01:25:58,029
model exists because of

1898
01:25:58,290 --> 01:26:01,430
many things I go through going in that paper that you have to

1899
01:26:01,589 --> 01:26:01,819
tell you.

1900
01:26:02,939 --> 01:26:05,790
But I think for particular domains and flow regimes,

1901
01:26:07,629 --> 01:26:08,319
he just needs,

1902
01:26:08,470 --> 01:26:10,049
I, I think the methods are there,

1903
01:26:10,779 --> 01:26:13,029
the community, the community is big enough

1904
01:26:13,819 --> 01:26:14,629
uh

1905
01:26:15,029 --> 01:26:18,390
People just need to come together under the same umbrella

1906
01:26:19,060 --> 01:26:19,950
and then

1907
01:26:20,129 --> 01:26:22,439
some, some people have to come together and

1908
01:26:22,680 --> 01:26:24,359
I think you'll already see a lot of benefit.

1909
01:26:25,220 --> 01:26:25,620
Yeah.

1910
01:26:26,629 --> 01:26:28,799
Well, I won't test you in five years time

1911
01:26:29,120 --> 01:26:31,189
on that, but I'm pretty sure you're,

1912
01:26:31,660 --> 01:26:35,169
you're right that it seems to be one of those ones where um

1913
01:26:36,020 --> 01:26:37,799
there's enough swell

1914
01:26:38,089 --> 01:26:41,470
of people working on it that it seems

1915
01:26:41,770 --> 01:26:45,189
irreversible now in terms of momentum. Um

1916
01:26:45,490 --> 01:26:47,049
it doesn't seem

1917
01:26:47,549 --> 01:26:49,000
like quantum

1918
01:26:49,129 --> 01:26:50,520
where there was this

1919
01:26:51,029 --> 01:26:52,160
bit of a buzz.

1920
01:26:52,399 --> 01:26:53,200
But then it

1921
01:26:53,990 --> 01:26:56,979
uh I know that's a whole other topic how machine learning is sort of, you know,

1922
01:26:56,990 --> 01:27:02,129
push quantum aside a little bit from a lot of people's minds, but it feels like with,

1923
01:27:02,140 --> 01:27:02,910
with machine learning.

1924
01:27:02,919 --> 01:27:03,669
AI, there's

1925
01:27:04,200 --> 01:27:07,529
almost every person you speak to in the community is doing in some form.

1926
01:27:07,540 --> 01:27:10,430
Um So it, it feels like it can't really turn back. But

1927
01:27:10,640 --> 01:27:11,620
yeah, whether there'll be a

1928
01:27:11,990 --> 01:27:12,729
um

1929
01:27:12,930 --> 01:27:16,959
I guess in the weather domain, things like FourCastNet and GraphCast

1930
01:27:17,120 --> 01:27:19,319
have had their big moments.

1931
01:27:19,779 --> 01:27:22,640
Uh I guess what you're referring to is maybe uh

1932
01:27:24,959 --> 01:27:27,700
there needs to, if, if there is one in the

1933
01:27:28,069 --> 01:27:29,160
CFD, for example,

1934
01:27:29,169 --> 01:27:33,509
domain that would be enough of a catalyst to then convince industry as well

1935
01:27:33,649 --> 01:27:36,640
where it seems maybe there hasn't been that GraphCast,

1936
01:27:36,649 --> 01:27:38,740
FourCastNet sort of moment yet.

1937
01:27:38,910 --> 01:27:41,509
I would argue in CFD,

1938
01:27:43,089 --> 01:27:46,279
I would say that it's not that the moment hasn't existed.

1939
01:27:46,970 --> 01:27:48,750
The question has not come up.

1940
01:27:49,470 --> 01:27:50,450
The

1941
01:27:50,680 --> 01:27:53,209
the weather, weather prediction is uh

1942
01:27:53,490 --> 01:27:56,129
is a sweet spot, right? For these techniques,

1943
01:27:56,459 --> 01:27:59,209
lots of data, lots of good quality data.

1944
01:28:00,240 --> 01:28:03,459
And then the question you're asking is you take one system

1945
01:28:05,209 --> 01:28:06,430
and you're saying,

1946
01:28:06,709 --> 01:28:07,270
predict

1947
01:28:08,660 --> 01:28:09,879
few days or

1948
01:28:09,990 --> 01:28:10,720
two weeks,

1949
01:28:11,950 --> 01:28:13,649
that's the only question you're asking.

1950
01:28:13,879 --> 01:28:15,410
And I think for that question,

1951
01:28:16,229 --> 01:28:17,169
I would say

1952
01:28:17,500 --> 01:28:22,370
the tools are fairly good, I say fairly good because

1953
01:28:23,729 --> 01:28:24,729
in my opinion,

1954
01:28:25,020 --> 01:28:27,689
these models don't need to be as sophisticated

1955
01:28:27,700 --> 01:28:29,859
and as expensive and they can still do a

1956
01:28:30,390 --> 01:28:31,870
nice job, right? But you're right.

1957
01:28:32,040 --> 01:28:33,930
So that's to me, that's more like

1958
01:28:35,120 --> 01:28:37,899
the sweet spot problem existed.

1959
01:28:38,649 --> 01:28:41,390
And the question was narrow enough

1960
01:28:42,049 --> 01:28:46,180
that you could. And those are the problems that I think AI

1961
01:28:46,950 --> 01:28:47,919
has been

1962
01:28:48,390 --> 01:28:49,930
really doing well, right?

1963
01:28:50,140 --> 01:28:50,720
Like

1964
01:28:51,229 --> 01:28:56,479
alpha fold is the prime example, very specific, very narrow question.

1965
01:28:56,950 --> 01:28:58,890
Now, if you say, can we go from

1966
01:28:59,839 --> 01:29:01,100
two weeks to

1967
01:29:01,620 --> 01:29:02,419
climate,

1968
01:29:03,029 --> 01:29:03,859
then

1969
01:29:04,430 --> 01:29:06,660
all of the work that's been doing then

1970
01:29:07,890 --> 01:29:11,529
needs a lot more, a lot more work.

1971
01:29:11,540 --> 01:29:13,189
So, to me, the,

1972
01:29:13,200 --> 01:29:17,049
the turbulence problem is more like the climate problem rather than the weather.

1973
01:29:18,479 --> 01:29:21,910
Yeah. Which is why I guess I was originally asking like, the,

1974
01:29:22,220 --> 01:29:28,680
maybe we, we need to set our standards a bit lower or our site a bit sort of narrower on,

1975
01:29:28,689 --> 01:29:28,959
like,

1976
01:29:29,259 --> 01:29:32,629
you know, even if you could develop something that proved you could design

1977
01:29:33,549 --> 01:29:35,330
a commercial aircraft,

1978
01:29:36,100 --> 01:29:36,689
that

1979
01:29:37,020 --> 01:29:38,220
is the thing where I

1980
01:29:38,410 --> 01:29:44,290
guess the picture has often been by, by some like all of CFD, which just seems almost

1981
01:29:45,770 --> 01:29:46,379
like

1982
01:29:46,950 --> 01:29:50,859
that's like predicting the weather on any planet in anywhere in the universe,

1983
01:29:51,399 --> 01:29:51,479
you

1984
01:29:51,669 --> 01:29:51,779
know,

1985
01:29:52,640 --> 01:29:53,430
by definition.

1986
01:29:54,509 --> 01:29:58,779
But yes, I think every organization within their, their confines,

1987
01:30:00,009 --> 01:30:00,029
I

1988
01:30:00,430 --> 01:30:04,790
think even full aircraft design is a bit too big for now.

1989
01:30:05,040 --> 01:30:05,450
But

1990
01:30:05,589 --> 01:30:07,029
yeah, it can change.

1991
01:30:07,479 --> 01:30:09,569
But even if your, if your thing is not,

1992
01:30:09,830 --> 01:30:11,589
you know, the weather prediction problem,

1993
01:30:12,229 --> 01:30:14,209
it's end to end AI. Right.

1994
01:30:15,319 --> 01:30:15,970
So that's like

1995
01:30:16,859 --> 01:30:21,109
really clean, I don't think end to end AI for something like aircraft design,

1996
01:30:21,120 --> 01:30:22,660
I don't think that will ever happen.

1997
01:30:23,399 --> 01:30:23,950
But

1998
01:30:24,549 --> 01:30:26,109
in the end to end spectrum,

1999
01:30:26,439 --> 01:30:27,799
I think already like

2000
01:30:27,919 --> 01:30:27,930
a

2001
01:30:28,060 --> 01:30:30,959
few gaps can be like closed

2002
01:30:31,660 --> 01:30:32,890
and because there are

2003
01:30:33,319 --> 01:30:35,359
uh human experts in the loop,

2004
01:30:36,129 --> 01:30:37,549
this can be beneficial.

2005
01:30:37,979 --> 01:30:38,899
But, but

2006
01:30:39,479 --> 01:30:40,709
otherwise,

2007
01:30:40,950 --> 01:30:41,129
yeah,

2008
01:30:41,140 --> 01:30:43,569
these tools are not useful without that human

2009
01:30:43,580 --> 01:30:45,629
expert being in the loop and guiding them.

2010
01:30:46,350 --> 01:30:47,270
But I think

2011
01:30:48,160 --> 01:30:49,379
as time progresses,

2012
01:30:49,390 --> 01:30:54,930
I think more of these gaps can be addressed and more productivity benefits can be,

2013
01:30:55,240 --> 01:30:57,359
can be achieved with not

2014
01:30:57,870 --> 01:30:59,939
much more sophisticated tools. It's just

2015
01:31:00,819 --> 01:31:02,120
having the

2016
01:31:02,229 --> 01:31:05,029
expertise to like integrate them and

2017
01:31:05,379 --> 01:31:06,729
guide them and do

2018
01:31:06,890 --> 01:31:06,910
the

2019
01:31:09,319 --> 01:31:11,720
well, I feel we could probably talk for hours on this,

2020
01:31:11,729 --> 01:31:15,910
which is the danger of AI and sort of science, it sort of becomes one of these,

2021
01:31:16,180 --> 01:31:18,589
you know, drinks over a beer in a pub.

2022
01:31:23,470 --> 01:31:24,580
Yeah, thank you so much.

2023
01:31:24,589 --> 01:31:26,750
And I really appreciate it and I definitely

2024
01:31:26,759 --> 01:31:29,270
would recommend people to follow you very closely.

2025
01:31:29,279 --> 01:31:29,310
I

2026
01:31:29,660 --> 01:31:31,419
genuinely everything you've done,

2027
01:31:31,689 --> 01:31:35,439
you seem to have a very good way of whether you know it

2028
01:31:35,450 --> 01:31:38,750
or not sort of predicting the future or predicting the path of science.

2029
01:31:38,990 --> 01:31:39,629
So, uh,

2030
01:31:39,640 --> 01:31:41,750
because I didn't know anything about the foundational thing and as

2031
01:31:41,759 --> 01:31:43,250
soon as I thought I saw you put it out,

2032
01:31:43,259 --> 01:31:43,750
I was like,

2033
01:31:44,000 --> 01:31:49,069
so he's done it again. He's, he's definitely sees where, where it needs to go.

2034
01:31:49,080 --> 01:31:50,180
So that, yeah,

2035
01:31:52,600 --> 01:31:54,750
having some curiosity and then having

2036
01:31:55,319 --> 01:31:56,589
the right people around me.

2037
01:31:57,990 --> 01:31:59,470
Ok. Yeah, very modest. But

2038
01:32:00,109 --> 01:32:00,120
a

2039
01:32:01,189 --> 01:32:01,350
group

2040
01:32:01,790 --> 01:32:02,049
of people.

2041
01:32:03,629 --> 01:32:04,720
Well, thank you again.

2042
01:32:04,729 --> 01:32:05,310
And, uh, yeah,

2043
01:32:05,319 --> 01:32:08,970
hopefully we get a chance to see each other at some conference uh in the future.

2044
01:32:08,979 --> 01:32:10,609
But yeah, for now, thank you very much.

2045
01:32:11,029 --> 01:32:13,490
See you around Neil. Thank you very much for this,

2046
01:32:36,359 --> 01:32:36,410
that