Hi, and welcome to the Neil Ashton Podcast.
In each episode, we explained some of the fascinating ways
that science and engineering are changing the world around us.
We talked to leading engineers from elite level sports like
cycling and Formula One to some of the world's top academics to
understand how fluid dynamics, machine learning, supercomputing
are bringing in a new era of discovery.
We also hear some of their life stories, their career advice,
the lessons they've learned on the way that I hope will be
helpful to you too. So sit back and enjoy this
episode. Hi, and welcome back to the Neil
Ashton Podcast. So today's guest is Professor
Ricardo Vinu Isa. Ricardo is currently a Professor
of Aerospace Engineering at the University of Michigan, where he
leads research at the intersection of machine
learning, fluid mechanics, turbulence, and artificial
intelligence. Before joining Michigan, he was
a professor at KTH Royal Institute of Technology in
Stockholm, one of Europe's leading engineering
universities, and he built a internationally recognized
research program focused on turbulent flow control and
data-driven methods for fluid mechanics.
He's definitely established himself as one of the leading
researchers looking into this. And one of the topics that we
bring on today, of course, is around like explainable AI,
causality, reinforcement learning, reduced order
modelling, and of course the topic of the moment foundation
models for fluid mechanics. He's a long people like Steve
Brunton. He's done a lot to really
educate the community, also the public in terms of these fluid
mechanics and how machine learning can be used for.
It's really, he's got some great YouTube videos and explanations.
And what I particularly enjoy about his work is he doesn't
just ask whether AI can make predictions faster.
He's looking at the question, can AI help us to understand the
mechanics of turbulence and to ultimate accelerate scientific
discovery itself. So, you know, in this
discussion, which as with any of them, is never long enough to
fully cover everything, you know, we dived into some of the
bigger questions facing CFT today.
So you know whether fluid mechanics will have its ChatGPT
moment, how close are we to foundation models can be
generalized across different flow problems, and why he's
actually more optimistic about that possibility than than ever
before. As I mentioned before, the
explainable AI causality, understanding where they come
from. Can you explain how AI is
getting to it? And one of the things that we
also look about is focusing on the role of reinforcement
learning in terms of optimization and control and not
just using AI as a pure predictor.
And then towards the end, we, we sort of zoom out and talk about
the future of the field, agentic AI, autonomous scientific
discovery, and also how universities should be adapting
education given that AI is changing the whole landscape.
So I, I think this is one of these conversations that, you
know, I learned a lot from it and, and I hope you do too.
So sit back and enjoy this episode with Professor Vinu Isa.
Thanks very much for agreeing to do this.
You are definitely on the list. Of people who everyone says, oh,
you should be speaking to him. You know, I read the papers, I
see stuff. And you're the sort of person
where when I read the paper, I was like, yeah, I really should
have thought of that. That's a really good idea.
So, yeah, thank you. No thanks for having me.
It's a real pleasure. And yeah, looking forward to the
conversation. So maybe we could get straight
into it, which is the, I guess the hot conversation now.
Which has only got more intense. Is this foundation models?
You know, can we somehow make a model that could?
You know, compute any fluid flow, whether it's, you know, a
geometry variation, A boundary condition.
Where do you think we are along that journey and how close?
How close do you think we are? To to get in there.
Yeah, I think few years ago I would have probably given a
different answer, but things are changing quickly in a in a good
way. And I think we're getting closer
to having and it all depends on the type of question that you
want to answer, right. What type of predictions and
what level of accuracy do you want to use the systems for
design, Do you want to use the systems for, for scientific
insight? And of course what quantities do
you want to get with what level of accuracy?
So there's many possible questions and directions to go
into. But in general, I think that
we're getting pretty close to having systems that can perform
very well even for quite complicated quantities and quite
some nuanced phenomena on on our system.
Especially because depending on how you define your model, and
depending on the type of question that you may be
interested in answering, you don't really need to mimic all
the physics of the renal system. You might just focus on a subset
of questions or a subset of mechanisms that could be
represented encapsulated in a smart way.
And that's basically what your model does, and not necessarily
all the intricate interscale mechanisms that give rise to
those particular phenomena. So yeah, I think we're getting
closer to having systems that truly can give us the right
accuracy, the right performance in in quite challenging
problems. Yeah.
So I'm intrigued to know what's changed your mind.
We said a few years ago you would have given a different
handset. Can you pinpoint anything that
has, yeah, changed over the past few years that's changed that?
There's a couple of things. The first one is we realised
that we could identify those key mechanisms pretty well in quite
complex systems using causality, explainability.
So not just in idealised versions of the whole flow or in
reduce order representations, but in really full fidelity high
order versions of the system, we could really interrogate the
data, identify what are the mechanisms that are the most
important and focus on those for control, for modelling.
So I think those almost surprising capabilities of this
explain ability and causal frameworks to really
characterize the systems has been one big pillar.
And the other one, the performance of some of the
generative deep learning methods, especially in the last
few years, diffusion, flow matching, conditional later
diffusion, these systems are really capable of generalizing
to an extent that I mean not so long ago I would say that it was
not really possible. There's still quite some more to
do and there needs to be some smart way of formulating these
problems in order to achieve real generalization, but I would
say that that we are really in a stage where things are pretty
impressive. Maybe you could explain, pardon
the pun, what you mean by explainable AI.
Yeah, that's a, that's a good a good question.
So what we mean with the systems and what constitutes an
explanation and even the, the, you know, the, the grammar and
the definition changes a little bit depending on the, on the
field you have explainable AI, explainable deep learning
interpretability. So there's different kind of
flavours to it. But the way that I use this term
explainable deep learning would be kind of assessing for a
particular model what features of your input matter the most to
the prediction of that model. So really identifying kind of
like in a feature attribution sense, what, what, which ones of
those components of the input really matter to your output.
And why is that interest in the context of high dimensional
chaotic engineering systems? Because if you create a model,
an input output model, where I'm just going to give you an
example. If I have the wing of an
aircraft and I really want to see what flow motions are really
affecting the most the drag, well, then I can create a model
which given the flow, it just predicts the drag on the wing.
And that's a predictive model that hopefully performs
reasonably well. But if I use this explainability
methods on that model, then I can go point by point and really
identify which points are contributing the most to the
drag on that wing. And that idea can really allow
us to identify volumes of importance of importance for
that particular question, for the drag in this case.
And what we have found is 2 interesting things.
One is by using these explainability tools, we realize
that the classical approaches to study turbulence and people have
been looking at vortices, streaks, Rhino stresses, many
other quantities, right? In fact, when people want to
show off a bit and they do a very nice CFD visualisation,
very colourful visualisation, they will show you Lambda 2,
right, or something similar. They show you vortices because
it's very spectacular and it's very turbulent.
And what we realise with our methods is that the vortices
don't matter so much actually, at least for the friction and
for the drug. They matter for other things,
but not for the stuff that you are trying to show them for.
And the reason is the fact that you can see something does not
mean that it's important. You simply can't see it.
And a lot of the turbulence, a lot of the turbulence research
has been a bit misled by what you could see in experiments,
which of course it has been very helpful and it has been very
illustrative. But sometimes you need to step
away a bit from the physical intuition and just simply
interrogate the data in a more diagnostic way and let the data
tell you what matters and what doesn't matter.
So what we've found is the classical approaches to
turbulence research. They were only telling a part of
the story. So in different regions, close
to the wall, the the Renaissance stresses were important.
Farther away, the streets were important.
Farther away there were all the rhino stresses that were
important. So the classical views were not
wrong because obviously, you know, there's a lot of time and
effort. They brought it to it.
They were just telling part of the story, right?
Like the three blind men and the elephant.
They were, you know, giving different partial views of what
the elephant look like. But all the classical
perspectives together is what actually comes close to what we
identify with our explainability methods.
And why are we so sure that these methods are giving us
something that is interesting and physical?
Because when we device control mechanisms, when we manipulate
the flow to diminish the presence of these mechanisms
that we identify, that's when we get the highest track reduction.
So it's not just that purely we can find nice colourful volumes
of extractors is that those extractors in a causal way are
actually affecting the drug in this case the most.
And they are tackling those extractors is really looking at
the root cause of the drug, so the disease and not the symptom.
And that's why we think that this is really a good way to
look at these mechanisms and not only from kind of controlled
perspective or from a satisfactory knowledge point of
view, also from a modelling point of view, because those
mechanisms are the ones that contain the crucial information
if you want to be a good model for that phenomenon.
So how do you see the the challenge that I still be not
got my head around which is the following.
If we assume that for a foundational model or you know,
surrogate model, we're assuming that we have lots of data of
wings of cars, of buildings, etcetera.
At least today, you know, most of those have, you know, a
decent error to what is the real turbulence, what is the real
flow? And you're asking the model to
learn it. And sometimes the data comes
from different sources. You know, the wing may have been
done with this turbulence model, The car may have done with this.
So how do you see the foundation model?
Is it ultimately yeah. Do we have to train on DNS?
But then we can't train on DNS because it will just be too
expensive. Do you need some DNSI?
Still feel like the foundation model is ultimately going to not
be as good as people think. Yeah, because no, I totally see
your your point. And I think here the key is
going back a bit to the one of the first questions, what do we
want this model for, right? And I mean, I don't think that
we are trying to build these models to replace DNS, right?
I mean we're not going to because that computationally
would not really make sense, right?
So I think that we're trying to build these models to either
accelerate design and optimization and or achieve some
sort of scientific insight in our system.
So how can we handle data from different fidelities and
different modalities? That's something that in what
we're building in my group, we're trying to embed the
uncertainty quantification with active learning loops in a sense
that we know that not all all our data is of the same
fidelity. When we are trying to generalize
and we're trying to produce data for new cases, the system will
tell us, look for this particular case that you're
trying to produce data in some latent representation, your
uncertainty is very high. So you should probably go back
and if you know, if you're trying to do a helicopters,
well, your helicopter data is actually pretty bad.
So you should be there, have higher fidelity or, you know,
run more experiments or really improve your model there.
And I think that's one strength actually that we can
progressively keep improving our model as new data becomes a
while. Now your data for helicopters is
pretty bad for what, right, Because it depends on what
you're trying to do. And here is when we need to be a
bit a bit realistic about the metrics and about the targets
that we want to hit. So if we are trying to get lift
and drag, right, that's one type of accuracy and one type of
model. If we're trying to get the
spectrum right and all the interscale mechanisms, right,
then that's a different type of model, right?
So I think in that sense, depending on the type of
approach and the type of figurative that we want to
achieve, we may have to resort to different data sets and
different ways of assessing the error and the uncertainty of our
systems. Yeah, no, I, I agree.
I think the there'll be probably one side of the community that
happily uses, you know, Reynolds averaged or even panel methods
or or or lower and are, you know, quite comfortable with
models that are not of DNS accuracy.
But yeah, yeah. On the other hand, I, I, I can
imagine just as companies spend a fortune constantly trying to
get to higher and higher fidelity CFD that maybe the
models will, you know, go with it.
One question, because I think some of your background, you
know, was on, I guess, would it be fair to say like reduced
order modelling and and some of that side 2, maybe a more like
lay audience? I'm not saying everyone
listening to this is a lay audience.
That's quite the opposite. But there are some people who
are maybe not as grounded in all the latest common question I get
is, well, machine learning is just reduced order modelling.
How would you chart the evolution, you know, from like
what people would call ROMs before to machine learning?
And what is it about modern machine learning that separates
it from reduced automobile? Unless you classify them as to
say. Yeah.
No, that's a good question. Depends a little bit on what
you're trying to do with your machine learning, right, Because
also machine learning is a quite broad area.
So, I mean, and we have some work that we've done before on
machine learning for CFD. So how should you or what are
the areas where machine learning can help CFD?
And this is work that I did with my friend Steve Branton some
years ago where we found three areas.
We found accelerate DNS, improved models, both runs and
LES and improve reduce order models or saturate models.
And, and, and, and that's if we are thinking of machine learning
for CFD, something that we made quite clear at the time, this is
4 years ago was that machine learning in principle would not
replace CFD. It's not about, and I think that
many people when they think machine learning for CFD,
they're thinking automatically replacing CFD, right.
I, I don't think that it's really about replacing CFD, but
rather complementing and helping.
So reduce order modelling would be just one area within all the
spectrum of possibilities for machine learning can help just
within CFD, but we can also think about control and
optimization where this fantastic reinforcement learning
work, for example, on really finding new controller
strategies. So it's very broad, but I would
say that if we focus on on surrogate models, traditional
models, one key has been the possibility of hiring well,
nonlinearities in your in your model development, right in your
surrogate. And of course PODDMDI mean the
mostly linear, although you can have nonlinearities in there.
But having neural network based compression at scale, which is
what you can have now with more, more than out in core
architectures that really allows you to have very impressive
compression rates, right. So quite some capability to to
distill the essential physics of your system.
And something that we found also in some of our work is in, I
mean, out on corners are good at compressing, right?
Compressing anything, images of cats and dogs and images of tool
and flows if you wish. But tool and flows, their
images, their flow fields, they contain much more information.
They're much richer than cats and dogs, right?
There's a spectrum, there's a range of scales.
So you should somehow, when you compress, you should somehow
disentangle the latent representation, because if you
do that and you can use beta AES, you can use hierarchical
priors. There's different ways to do
that disentanglement, but that really allows you to encapsulate
in different latent representations and different
latent vectors, different physical phenomena of your
problem. And that's going to be more
effective from the interpretability point of view
of your Stargate system, but also from a predictive point of
view. If you want to make predictions
in in the latent space, that disentanglement is actually
going to be quite, quite helpful.
So, yeah, I mean, I guess it was a bit of a long answer to your
question, but machine learning is not just reduce order
modelling. Let's say that there's many
other things that one can do and within reduce order modelling
that, you know, capability of really having out in colors at
scale and disentangling the latent representations has been
quite critical. And also Transformers, that's
another, well dimension let's say, or another topic, which
also has helped in building the temporal dynamics of these radio
solar systems. I mean, where do you see?
At least my impression has been that even though the turbulent
modelling started off as being one of the the big focuses of
the fluent community, it does seem to have slightly petered
out a little bit and the focus has shifted a little bit more to
the surrogate modelling, to the sort of reduced auto modelling.
Is that something that you've also noticed?
And do you think that's just because of a lack of progress?
Do you think it's because the, you know, the improvements?
I'm just wondering if how you've seen that?
Yeah, no, that's another good point.
I think that so turbulence modelling can be done in
different ways. And One Direction that has been
adopted by many people has been to start where more classical
approaches to turbulence model modelling start.
Basically to which in a way is just making an empirical
assumption of how the, you know, the Renaissance stresses need to
behave. To some extent.
There is some empirical assumption, more or less
sophisticated and then try to use machine learning systems to
continue from there. So in, in other words, to fit
coefficients on classical models, right?
And well, that's not maybe the most revolutionary thing, right?
Because if you do that, and that's what many people have
done. And I think that's why a bit of
the ML community could have got a bit disappointed with ML for
fluids, because they were using ML to adjust coefficients in two
large modelling models that have been around for decades and
which have based on assumptions that are a bit crude, perhaps,
right. But for a reason, they were
crude because there were not other approaches to, to
modelling that were, you know, kept possible at that time.
So I think that could be why there has been a bit of slow
down. But at the same time, I've seen
some promising approaches. I mean, I'm, I'm a bit of a fan
of reinforcement learning. I've, I've been developing many
methods within reinforcement learning for control, for
optimization and also for modelling.
We have some, some projects on reinforcement learning for
modelling and there's other groups who are doing great
things in this space. And I think that the idea is why
start with a model that has been around for, you know, 20305060
years and try to adjust those coefficients when we can maybe
take one step back. So instead of asking my machine
learning model to say for the Smolensky model, what should be
my coefficient, Can you find it to, to, or even say to my
machine learning model or just find the subway scale tensor to
be able to fit the statistics of this this particular channel or
particular flow? Maybe we can be a bit more
abstract and say, look, I don't know what the mean flow or the
fluctuations will look like in this particular case.
I, I don't know what happened. I've never run this wing or this
aircraft, but what I know is that turbulence is characterized
by certain energy transfer mechanisms across scales, and
that energy goes in this election and there is an inverse
cascade. And there's certain phenomena
that need to be true from a spectral point of view.
And from a mechanistic point of view, whatever your model is and
whatever the forcing term that you get each step from your
reinforcement learning or whatever optimizer that you're
using, just respect those physical constraints.
Because my flow needs to be physically correct.
And physically correct does not mean this is your mean flow.
You need to adjust to this mean flow because that's a circular
argument, then you need the mean flow, right?
But rather not only a circular argument, it's some more, it's a
much more constraining goal to fit the statistics right.
But something a bit more flexible, such as making sure
that the energy fluxes are correct step by step is
something that is less impose imposing.
And it's also something that is a bit more manageable for a for
a system for an optimization system to to be able to adapt
step by step such that those fluxes are OK.
So I think the second direction of making statements that are a
bit more general and still correct from a physical point of
view, that could be more promising to develop more.
I mean, one of the things that I've started contemplating, and
I'm interested to see if you've been the same is for for a while
I was only thinking about, you know, the machine learning,
either in the context of learning what the total and
viscosity should be or, or more broadly, what a surrogate model,
you know, should be in terms of a transformer based or some
neural operator or, or whatever. And the criticism has been that
it's, how can it match the, the brains, you know, the thought
process that, that, that we've had.
And I saw at an event, I was at the British Library, there was
an AI, somebody organized UKTC, the turbans consortium in the
UK. And it was Luca Magri from
Imperial was organizing with some people.
And there was a Professor, Michael Leschiner, who, you
know, is a sort of pioneer of turbans modelling developed with
Brian Launder, you know, all these renal stress models.
And, and he rightly said to me that he was, you know, gently
wondering how could these AI models, you know, know, all
these things that, that, that they've done.
And then I started to think, is this really, and I don't know if
the capabilities today where the agentic thing comes in because
in some ways is it more realistic to ask an LLM to
reason through the process of how you would develop a turbans
model and have links in to software tools to explore it and
go through that reasoning process and reinforcement
learning. Then just say, hey, try and find
me the the latent representation of this.
Do you know what I mean? Do you think almost the
scientific discovery that piece is the better route rather than
just expecting machine that is? So I don't know if I've
explained that, but. That makes perfect sense.
I think it's, it's a very fair question and we are trying to
well to really first of all understand the potential of
decisioning systems and 2nd, we are learning to ask the right
questions to these systems. I think before getting into
that, I think that a preliminary point is what is the right way
to represent the data that we as engineers and scientists in you
know, higher order chaotic complex systems.
What is the right, the right way to represent the data and such
an exploration from these agents or whatever or, or any
scientist, what's the right platform to represent the data,
right? Because there's quite some work
on LLMS to do this and to try to achieve discovery, whatever we
define by discovery. But you know, if we think of the
of turbulent flows, of course, this is these are systems that
are having the broadband in terms of the spectrum, they have
multiple skills, multiple energy fluxes.
Perhaps text is not the right way to represent this very
complex data, right? And perhaps using LMS directly
and text as a platform to try to explore complex questions in
this space might not be the the most suitable approach.
And what we have been thinking and exploring is what if we can
create latent representations that are particularly attuned to
represent important properties of these two water flow systems.
And, and that's an approach that we have been developing in our
group where we are trying to build a foundation models, kind
of latent representations of very broad ranges of cases such
that we can use those latent spaces to explore the, the, you
know, the, the complexity of these, of these two water flows.
One guiding principle has been information, information fluxes,
causality. So what if we can create latent
spaces where causal relations are maximized or where the
disentanglement of the variables is such that I can very easily
explore physical mechanisms through that latent space.
And I believe and again, this is stuff that we are very excited
about because we are we're developing it with pretty
promising results that using a genetic systems in that latent
representation might be a good way to achieve discovery.
So in, in other words, I have a data from wings aircraft flows
in cities a compressors and these are very different fluid
mechanics problems. But if we find smart way to
compress all that data and express it in the same latent
representation, so now suddenly I don't have a wing and a city,
but everything is expressed in the same language on the same
table. And then I can allow this
agentic systems to explore this latent space very efficiently.
Because of course this AI systems can't really find one
side compress everything so much.
I can find patterns across cases that might not be obvious in in
the physical space for us humans and neither for agents, right.
Because finding a connection between a wing and a city, I
mean, yeah, maybe if I look like that, I see a vortex that looks
similar. But you know, it's, it's a bit
more anecdotal, anecdotal than anything.
But when I express everything in that space and then I can do
that automatic and autonomous exploration with these agents,
then I can try to get inside. Then I can try to find causal
relations that are key in that later space.
And then our job with explainability tools is to
express that insight in the latent space, which is
completely non understandable for us back in the physical
space and say, look, this is what insight looks like in the
latent space. That means that this vortex from
the sheer layer of the wing is actually very similar to this
vortex from the sheer layer in the separation of that building
in the city. So that's actually a common
mechanism. That's interesting.
And I can find that through compressing, expressing in the
common space, A autonomously exploring and then going back to
the physical representation. Yeah, that's, I really like the
I've you've hit the really good explanation there because this
is something I've been thinking in my head is and I I'm not, I
need to look into it more to be completely honest.
But I almost see 2 slightly different approaches
traditionally, which is 1, which is kind of what we wrote in this
paper. We put out recently this idea
that, oh, you, you know, we're going to run lots of simulations
of planes and cars and cities and turbo machineries and
basically every single possible geometry and band of condition
that we see in real life, which feels in some ways the more
logical and achievable way. It's just like you just have to
have a lot of them. But then the other side is that
actually it doesn't matter whether it's a plane or a city
or whatever, It's just you're just trying to learn the flow
physics. And actually you may massively
have way too much information by trying to do all possible ones.
Which then I thought, couldn't you achieve this?
Do you really need wings? Couldn't you just create lots of
random shapes with lots of random boundary conditions?
And as long as you take every possible consequence of shear
flow and pressure gradients, it doesn't actually matter if it
looks like a plane because you're just trying to learn the
physics. That's a very good question, I
would say. I mean, that can be helpful
because what you want is to be able to explore such a latent
representation very broadly, you know, and that you can really
investigate many configurations and calculations and so on.
At the end, well, we want to revert back to something that we
can understand, right? And then probably having some
random shapes with some random pressure gradient and curvature
distributions would be helpful for the exploration because that
would allow us to identify different mechanisms that
probably we haven't seen just because they were not ideal
dynamic, but maybe they were helpful for other things.
But probably in our data sets, we want to have shapes that
we're familiar with and applications that we're familiar
with just for the kind of interpretation point of view, so
such that we can connect it. But from the latent space
generation, yes, the most the more diverse and the more crazy,
the better. In fact, what what we are doing,
what we are building is systems that the agent not only does the
exploration but also generates new geometries by itself.
So we have this problem that we like very much and we're having
a pre print coming out very soon on this.
So it's basically 2 cylinders to the keep it simple for now, but
it's 2 cylinders with different radii and different separations.
And we just want the our system agentic AI with a foundation
model system to to explore the wake of this tandem cylinder
arrangement and just look at the recovery of the wake.
And then what this does is, well, in order to do that, and
this is trained on few cases, right?
It's not trained on many cases, but I need to explore the whole
space, right? So these agents, what they do
is, oh, but I need to look at this configuration and calculate
my weight characteristics and internal quantities and you
know, displacement thickness, momentum thickness, okay, I know
this. But then I need to look at a
different case and a different case and a different case.
And then it starts to sequentially understand what the
scaling of the wake recovery looks like as a function of the
parameters of your problem. And you haven't not had to
create hundreds of cases. You only have to create a few
cases. But it's the foundation model
combined with the agent that is doing that autonomously is doing
the discovery for you by identifying what regions are
important and creating and generating new data in those
regions. In those regions.
I think that's where we are going so that we can even find a
discovery of cases that we don't even think of because, because
we have been looking at patterns influenced by the past history,
right? And kind of like the need of
applications, but autonomously exploring the whole space as
agents can do. They can really multiply the the
the insights and new solutions and new mechanisms that we can
actually discover. And and that's why I'd feel and
I'm guilty of this myself, that we are still probably too, too
much splitting the fluids task or the data generation task from
the machine learning task. And so, you know, I'll be
transparent for the data sets that I've typically generated,
they have been kind of segmented a little bit for practical
reasons, for people reasons. So you're like, right, Well, you
know, you have a bunch of meetings, you decide what you're
going to do. You then kick off the data
generation exercise, you generate lots of data, and then
you train the model. And where, as you say, what you
really want to be doing is constantly them being in a loop
together and only generating the data it needs to generate.
But I don't know if you found this in your work, but one of
the problems is that even the machine learning frameworks are
typically not even written in the same language that the data
generation is. And doing that on a big HPC
machine and training and translating information, it's,
it's quite challenging and maybe brings the topic that, you know,
we briefly discussed before we started recording, which is this
fluid community, computer science, ML community.
How have you seen those two communities come closer?
Still not close enough, You know, how how have you assessed
this? And, and, and to be fair, you
have done a great deal. And to be fair with people like
who you collaborate, Steve Brunton of of trying to, you
know, bring a little bit together.
But yeah, how how do you see those two communities at the
moment? Well, thank you.
Thank you so much. First, I think the problem that
has happened for a while and still present, although probably
getting a bit better, is that they have been a bit
disconnected in really understanding the the depths of
the problems in fluid mechanics. And perhaps the fluid mechanics
community, when trying to adopt MMM methods, has been sometimes
a bit naive in just. Adopting very vanilla
architectures and maybe giving up a bit too quickly without
fully understanding. So, you know, if the mechanics
community is guilty in part, and I think what has happened also
from the CS community is that, you know, I mean aerodynamics
from I am at the aerospace engineering department right
here in Michigan. We train engineers for many
years to understand the mechanics, aerodynamics,
instructors and many other things.
But of course fluid mechanics is a big part of it.
So you can't understand all the complexities of fluid mechanics,
you know, in the course of 2-3 months, right?
Because there is a lot to look at.
These are very entangled problems and sometimes the
questions and the problems that we have in fluid mechanics are
not what typically the CS community has looked at
necessarily. Because you know, the type of
data that has been used in the CS community to develop methods
is not necessarily of the same characteristics as the fluid
mechanics data. And this is very rich data with
a very broadband Spectra multi scale.
This is really, really complex physics, right?
So maybe looking at the MSE is not the right metric, right?
Because this only tells you a very, very partial view of the
story. And maybe even having a model
that predicts the MSC, well, that might be useless in some
cases, right? We maybe want other things.
Maybe we just want to look at, you know, systems that predict
separation, right or that predict certain aspects of the
mixing or maybe, you know, certain structures, certain
mechanisms as I was mentioning before for, you know, for a drug
producing mechanisms. So it could be that for my
purpose, if I want to reduce drug, having a good MSE is
actually counterproductive because the MSE is going to
drive me to certain events, which may be very energetic and
they may be contributing a lot to the MSE, but not so relevant
towards the drag generation, which is the stuff that I want
to, you know, focus on in my application.
Therefore, a model with a much worse MSE but getting the error
right in the right mechanisms is way better, right?
So I think the this type of this type of interaction is what is
really going to help us build more useful models, at least
from the fluid mechanics community.
And of course, this is not just for fluid mechanics.
This is applicable to any area of, of science, right?
But of course, I can speak, you know, with more depth on the
fluid mechanics problems because what you want, and this is
another question, you know, let's build a foundation model.
Let's build a big ML system for fluids, OK.
But for fluids for what, right. I mean, within fluids and within
aerodynamics and within combustion, I mean, there are so
many questions that we can look at and probably the model that
you build is going to be quite different for all those
questions, right? So I think that's really
understanding in more depth what the fluid mechanics problems
require. What are the questions that we
have? What are the questions that we
don't have? How to build models that can
really effectively tackle those questions, that can be a much
more effective use of everybody's time, basically.
Where do you sit on the debate around a data-driven versus
physics driven models? You know, again, the usual
criticism from people who are maybe from a fluid background
is, you know, any of these data-driven approaches cannot
guarantee mass conservation, energy conservation.
They can't guarantee, you know, we should be implicitly or
explicitly imposing, which I guess is where some of the pins,
you know, things started. But then the successes seem to
have been not as strong in the sort of physics informed,
inspired, conditioned. So yeah, where where do you sit
on that side of the fence? That's, well, that's, that's a
big question, right? Of course you need to, you need
to use any physical information that you have about your system
when building your models, right.
But if you just use the equations and nothing else, then
you're building a numerical solver, right, with a
data-driven numerical solver. And if you just use data, then
you are a problem, you know, be being very wasteful because
there's a lot of information that you have about your system
that you need to relearn. Like, you know that your system
has certain symmetries, certain conservation properties.
If you don't embed that in your system, well, first of all, your
mother will be wrong because you will never be, you know,
conservative exactly right or as exactly as possible.
But second, even if it's a very, it's a system that is very
conservative, you have to learn that right.
So you need to use data and compute to get to that point.
And of course, that's something that you knew from the
beginning. So it's not very smart to to
waste that information. So like with everything in life,
right, no extreme is going to be optimal.
And I think that both extremes have been explored and we have
learned a lot from it. To me, being able to use
symmetries, conservation laws in your problems.
If you're trying to solve for several velocity components, I
mean, do you know that there is the flossing compressible, Do
you know that you can use incompressibility to get the
velocity component or use that in the way that you're building
your system? But also going back to an idea
that I like very much, this whole explainability, causality
mechanisms. If we know these things, why
don't we try to build systems that are really focused on this?
And one example is, let's imagine that I want to create a
system that predicts very well the acoustic feel of an airfoil
or a drone, right? Because drones can be very
noisy. I want to be able to control the
acoustic feel, the noise produced by this drone.
Why don't I use some of these causality explainability methods
to identify and pinpoint the mechanisms producing the noise?
And by mechanisms, I'm talking about flow regions interacting
with each other in physical space.
And then when I build a predictive model, I don't look
at the, you know, trying to get neither just the governing
equations like in a pinch way or just use data to predict the
flow fields. But rather, why don't I try to
predict just those extractors, those mechanisms that produce
the acoustic field in that way, you're really encapsulating in
those structures a lot of information.
And that's where I think that one can have computational
savings with respect to, you know, doing a much more resolved
simulation. Because those structures
producing the acoustic field are the result of a lot of, you
know, interscale energy interactions that you need to
resolve when you do ADNS. But the model, I mean, once that
DNS is run, those structures that are producing the acoustic
field and so on, that the result of those interactions, right?
So you don't need to re simulate them all the time.
Once you know what the mechanisms are, you can try to
develop models that target those physical mechanisms precisely
and not everything else. Of course.
That's why I repeat, sometimes I want a model for what?
If your model is for the acoustic field, then my advice
would be, well, identify what are the mechanisms producing the
acoustic field and try to get those very well, right?
Because probably you will not get a perfect model, but you
will get something that is pretty accurate and pretty
efficient because you don't need to solve everything.
But if your goal is to have something that gets the acoustic
field very well and the gusts and the turbulence and the heat
transfer well, then you need a multi physics DNS, right?
Then, then you need to solve everything.
So it depends a little bit on what you want and of course,
embedding that physical insight in your model, not as equations,
but as mechanisms that you can then, you know, recreate with
your system. I think that that's a pretty
promising way to go. And how from a, you know,
you're, you're obviously a professor at university.
What do you think? What's the role of academia in
this, focusing more on like the students in the education side,
you know, do you try and team up more with the computer science
departments? Is it a matter of swapping
people over like, but but then equally, I guess you don't want
to not teach some of your existing content.
So how do you balance getting rid of some content, bringing
new ones in? That's that's a key question.
And we're having many discussions around this.
For example, here in Michigan, I mean, we're having a lot of
conversations around our new curriculum and how to well adapt
to AI era, LLM base era. So we're having many
conversations around this. So first of all, we have a very
good interaction with Los Alamos National Lab.
We actually have a bunch of Los Alamos scientists sitting on our
campus and working with them daily and interacting with them
constantly. So we really get access to that
synergy and that cooperation with computer scientists, which
are world class, right? So with with really incredible
computational facilities. So I think such an interface and
such a close connection is, is essential.
But the other thing is from the educational point of view, we
need to be of course aware of the fact that students have
access to LLMS, right? And and that, you know, the way
that we teach, the way that we assess is different from what it
used to be. But in a way, and this is
conversations that we're having with my colleagues and I think
that there's quite some interesting, I'm promising
directions in here. We, we don't need to necessarily
see this as a disadvantage, but rather as an opportunity to use
these systems to encourage students for more exploration,
for deeper, for, for deeper questions and deeper assessments
of the content almost in a customized way, right?
So, so I think that rather than thinking that students are going
to be lazy and just look up their answers and not think,
rather use these these systems to help students think in a way.
But do you feel that? I guess flicking around the
other way, do do computer science students need to learn
more about the sciences? You know, there's a lot of like
AI for science. If the engineers are all
learning some of the computer science being, then what are the
computer scientists doing? Is there, is there an equal
thing on the computer science say hey you all need to be
learning some science domain because AI is going to do the
some of the core CS stuff, you know?
I was, well, I'm, I can tell you that there is, I mean, there is
an increase of enrollment of students in, in areas like
aerospace engineering and, and part of it comes from computer
science, although of course the computer science degrees,
they've been excellent at adapting to, to new technologies
and new well, developments within AI for science.
But there is more and more overlap and more and more, you
know, joint educational programs across systems and across
engineering disciplines. And yes, I would, I would agree
completely that if computer scientists are going to be the
developing AI for science or AI for engineering methods, then
they're going to need more background in these, in these
topics. And that's something that is
really that's multidisciplinary wealth and and richness is
what's going to bring really stellar research in the next
years, right? That 1 is really capable of
finding completely new directions like the foundation
model and the agent are finding patterns across data sets that
you would not imagine, right. When we start to interact in
disciplines at a different level, we're going to find such
connections that maybe we did not expect at the beginning.
So one of the things that, you know, you mentioned a few times
is on the like control and optimization side of things.
And I feel at least myself a little bit guilty that I, I'm
always thinking of, you know, the, the impact of machine
learning or reduced order models in terms of a predicted
capability. You know, we can predict this
flow much faster. But I guess for industry or even
for scientific discovery, but particularly for industry, it's
almost always a kind of optimization problem in a way
isn't it's like I need to have the best design or the lowest
drag or the it's never just a prediction.
So, So what progress have you seen with the sort of automatic
differentiation, the the ability of machine learning to maybe
help advance that optimization process?
Yeah, I I think that prediction is important.
It's interesting. It's not the most impactful
application of machine learning. It's in optimization and
control. That's where we can really have
the, the biggest, the biggest impact.
So as I mentioned before, I, I worked a lot on reinforcement
learning. So that's an area where we have
been able to control cases that were not possible three or four
years ago or at least with this degree of control authority.
And not only. So I would say not just from a
purely design point of view of reducing drag or enhancing
mixing. I would argue that for
discovery, for really understanding the mechanisms and
the building blocks of these physical systems, you can use
optimization and, and, and reinforcement only.
For example, I mean, imagine a tumulant flow where a made-up
turbulent flow where the energy fluxes are restricted to certain
scales and to certain work numbers dynamically, right That
you do it as the flow is running and, and, and you want to do
that in physical space. So you want to really affect
certain instructors with a certain forcing.
Well, you are probably going to have to do it with some
optimization technique. And reinforcement learning is
good for optimizing systems that are changing dynamically.
So you you're going to end up with a with a modified system or
a system where production is minimized and dissipation is
maximized. How does a flow like that look
like? And let me study, let me study
that new flow, right? I mean, I have created a
completely made-up flow thanks to optimization.
And now I can look at the Spectra, I can look at the
structures and learn something about my problem, right.
So yes, these capabilities with optimization are key also for
discovery and so for getting insight.
And going back to your previous point about how new systems and
differential solvers, I mean, we, we are really having the
chance of solving problems at scale of this optimization type
of system, which was not possible before.
And inverse problems where we can, you know, optimal sensing,
we can really look at optimal initial conditions and optimal
perturbations really for very, very challenging configurations.
All of that is possible now thanks to the scale that we can
achieve with these systems. So I think that's, yeah, I mean,
prediction is, is good. It's interesting.
It's only part of the problem. It's really in the control and
optimization and in the explainability of those
predictions. Like, OK, I have predicted very
well in my system, but why are those predictions happening?
And what are the really important physics and effects
that I can observe with my system?
That's that's the stuff that I think is is is actually
valuable. So maybe a sort of final
question or, or topic looking towards the future a little bit.
You, you said right at the beginning that if, if I had
asked you a few years ago, you'd have said that we're, we're
actually, you know, further away.
So if you look forward in time and let's say to 2030, so 4
years out, yeah, four years out, where do you think we'll be at?
Do you perceive there being a a sort of ChatGPT moment in fluid
or do you think it'll be just more incremental improvements?
Yeah, that's a tough question. I think one key to go towards
ChatGPT moment in fluids was to realise that we need a good
latent representations for our problem.
And I think that now there's several groups of several people
working in that direction. So I think we're probably in the
right path. I would not.
So I don't know if we want to aim at being having a ChatGPT
situation in fluids, maybe something beyond maybe something
more than what chat PT can can do.
What I think that we are aiming at, and I don't know if that
would be in 20-30, but I think that it's actually reasonable to
think that it should happen in the next years, is more
autonomous discovery, a more automatic discovery.
So one, I like the, the one example of how big changes in in
the paradigm of physics have taken place and of course, well,
relativity departing from more classical mechanics.
I mean, it takes, if you think about it, a lot of a lot of
coincidences to line up in order for that to happen.
And you need someone who is smart enough and also crazy
enough to propose something very different.
And of course, Einstein lacked all the math background to be
able to, you know, develop the theory of relativity properly.
So he had to learn the math and interact with the right people
to be able to do this properly. And then, you know, to have the
right experiments of the of the solar eclipse to really look at
the deviation of the sand beams. And there's some fun stories
about how those measurements took place because there were
several groups doing the experiments at the same time,
and they had different types of problems to really get the
measurements that eventually corroborated that light was, you
know, displaced by the sun in a way consistent with relativity.
There's a lot of certain Dipion discovery, a lot of coincidences
lined up, and a lot of human try and error to achieve truly
transformative discovery in science.
And I think that the sort of systems where we can line up on
the same space, the data from different disciplines and having
agents systematically probing and analysing these data sets,
that can really be the way in which we can accelerate that.
But it's this acceleration of discovery, which is something
that, you know, many people are talking about with different
meanings. And I think it's a little bit
shallow the way that it's been portrayed sometimes.
To me, what it means is being able to probe the data and find
patterns in a more systematic way than what it takes us as
humans to take those leaps. Because we're it's really
decades and decades and centuries to line up all those
coincidences necessary for that leap to happen.
And if we can do this in a way that is much more autonomous and
much more systematic in that sense, that can be accelerated,
right? And this sort of systems I, you
know, I think that we are really getting there actually.
And, and, and this is not really about replacing human insight or
human input. This is really about
accelerating the exploration on the capability of coming up with
mechanisms with hypothesis with a really patterns that are non
trivial across across disciplines and across cases.
So yeah, I think that we're heading in that direction of
really being able to make autonomous discovery thanks to
the possibility of compressing data and interrogating it
systematically in a way. Yeah, it does seem that there is
this, I would say inflection point now where I would largely
agree with you, you know, a few years.
Ago when I. Started to think about some of
like a foundational model, you know, it it did for various
reasons just felt quite Yeah, just like impossible almost.
It would, you know, may still be, but I I also get the feeling
that there is a, a general movement of replicating the sort
of chat chi BT success, you know, the the set.
And this is broader than just fluids across science, across
physics, across all, you know, possible disciplines that, you
know, it worked for that which and, and I'm not I haven't
studied a a great deal, but I, you know, I know enough to know
that many people thought that it couldn't be possible, right.
You know, people weren't saying, oh, chat chi BT Oh yeah, we knew
that was going to come. You know, it was quite a big
impact and, and it, it was done at a scale that was larger than
anyone thought possible. And I, I do have a feel that
made people more ambitious, you know, and, and think, well, if,
if it could work for that, it, it could work.
But I do like your argument on that.
Sometimes you need to look at things a bit differently because
I have a funny feeling that the approach that ultimately makes
it may not be the one that is mainstream today.
You know that that like in all in science, there's always, it's
always the person who takes a little bit of a, an odd
direction at the time which turns out to actually be the
one, you know? An agent to come up with that
direction, maybe. But that's what I meant.
Like the, I can totally see how with humans sort of in the loop,
you know, in, in, in terms of guiding things, that the maybe
it is ultimately the power of ChatGPT to find the next
ChatGPT. You know, that those LLM
technology and the agents and the way they work together with,
of course, the way that you structure the data, you know,
may help. And, and then maybe that's
artificial general intelligence. And then we can all just, you
know, go and retire and, and, and chill out.
But yeah, I really appreciate having the chance to, to, to
chat about this. What I'm going to do for people
listening or watching is to put a bunch of links into the, the
comments because there's a few papers that you alluded to that
are really good, you know, great reads that talk in more detail
about some of that explainable AI and some of the work that,
you know, you've done. So I'll, we'll put them in the
link and highly recommend that people read through them to go
even deeper than we discussed now.
But yeah, I, I really appreciate it.
And I think your work is going to be seen as a major
contributing factor to hopefully achieving that ChatGPT mode.
Well, thank you very much. I appreciate it.
Maybe that's a bit optimistic, the money work, but I appreciate
that much. And at the end we, we have fun
with what we do. I think that's kind of like the
key, the key idea and we keep learning.
So that's that's pretty much it. Exactly, exactly.
Great. Thank you so much.