Hi, and welcome to the Neil Ashton Podcast.
In each episode, we explained some of the fascinating ways
that science and engineering are changing the world around us.
We talked to leading engineers from elite level sports like
cycling in Formula One to some of the world's top academics to
understand how fluid dynamics, machine learning, supercomputing
are bringing in a new era discovery.
We also hear some of their life stories, their career advice,
the lessons they've learned on the way that I hope will be
helpful to you too. So sit back and enjoy this
episode. Hi, and welcome back to the Neil
Ashton Podcast. So today I'm speaking with
Professor Mike Giles, somebody who has had a enormous influence
on the CFD community, but more broadly in, in maths, in high
performance computing. And someone that, as I say to to
Mike at the beginning of the of the chat is his name often comes
up when I speak to, to other people.
And I was always really intrigued to, to speak to him,
particularly because when I was at Oxford myself, he was always
a figure that was, that was mentioned and was influential in
so many bits. Probably for most people, he is
most known as the, I guess you'd call it the lead developer
instigator of the Hydra Rolls Royce CFD code that is used
stilted today by I would assume, thousands of engineers around
the world to design the the jet engines that they produce.
And, and I think what makes really interesting is his pivot
also to them work in finance, computational maths in in as
applied to, to, to, to finance, where he is equally made in
prayer. And that's what's incredible for
someone to make an impression in two quite different fields, CFD
and and finance being quite different shows the level of the
person, the intellect, the the sort of innovation potential to
switch to a different field and then still make an impact.
Now I only come from the first field, so my appreciation of the
second in the sort of quant side of the world.
But I've read enough to know and you only have to look at do a
quick Google search to see that he's his work on what's called
that multi level Monte Carlo methods has been very impactful
in that community. And the third thing that I guess
he is really pioneered in some ways, and one one was one of the
early adopters for is on the high, high performance computing
side and particularly around GPUs.
I should state obviously for transparency, I do now work at
that company NVIDIA, but this conversation was in no way
arranged to to promote NVIDIA. This was organized completely
separately. And it just happens to be that
I, I, I don't I would there. So you'll hear some mentions of
it, but please trust me, this is not some sort of, you know,
product placement. And and he was actually working
on this and I think he said that he was maybe the number 2 or
like the second ever CUDA fellow and was looking at this in 2000
and six 2007. You know, well before sort of
everybody knows the name of video and GPU.
So he, he is always, and he says in the discussion, had an
interest in high performance computers throughout the time
and, and still today teaches classes and, and programs and
does everything, which is, I really love that when you see
someone who's gone through their whole career and it's completely
fine if you do change. And some people as they
progress, become more senior, they get less hands on, you
know, and then they're more about enacting a vision for what
they want to do. And there's nothing wrong with
that. But I always have a special
appreciation for, for people who are, you know, one of the
world's leading professors and they, they're still a hands on,
you know, keyboard. So yeah.
Mike is a professor of numerical analysis at the Maths Institute
at the University of Oxford. As we mentioned, beautiful
building, lovely location. I'm very jealous.
And he was at the University of Cambridge that was rated as an
undergraduate. And we mentioned it, he was a
senior Wrangler, which if you look it up on Wikipedia,
basically means the person who graduated top of class for the
whole university in terms of maths as an undergraduate.
And then he went to MIT, he was Kennedy Scholar, taught there,
came back to Oxford. Now we go through all of that,
but one of the things that's probably worth mentioning and
congratulating him on is that actually very recently he was
elected a fellow of the Royal Society, which is one of the
highest honors that can be bestowed on somebody.
So he's had an amazing career, still has an amazing career.
And like any of these episodes, when I talk to something like
that, there are so many things that I realized I didn't ask him
after I finished the episode. And I don't think we would have
time anyway to, to to go through stuff.
I will just note that if you Google his name and go to his
personal website, I'll put it in the chat for the YouTube side of
things. He has a great link to lots of
presentations courses that he's done.
So I would definitely look at there on that to yeah, to to
find out more. But yeah, I really hope you
enjoy this conversation. I genuinely did.
I hope you can see it from my face if you're watching it.
I was learning and, you know, interested throughout the whole
2 hours. So yeah, please sit back and
enjoy this episode with Professor Mike Childs.
Thank you very much. Really appreciate.
As I said just before your name comes up a lot in people I speak
to in the in the CFD and HPC world.
And so I was, yeah, really wanting to speak to you to find
out a little bit more how this, you know, connections started
out. But maybe we could start, you
know, I guess towards towards the beginning.
Now you're a professor at, you know, one of the top
universities in the in the in the world and maths department,
but obviously with an engineering background.
Did you always want? Was your interest in maths?
Engineering? Were you a person with planes
and cars? More reading?
Physics What? What was your early days like?
So I would say in school my interests were maths and
physics. And so I actually, you know,
went to Cambridge to study maths, intending after the first
year to transfer into theoretical physics.
But then, you know, enjoyed the maths, stayed, stayed in the
maths. So, so as a child, that's kind
of the direction I suppose I saw myself going in, but I also
wasn't sure, you know, what I would want to do after my
studies. And so in terms of CFDI mean
really the, the pivotal thing for me was the fact that in in
going to Cambridge in those days, you did the entrance exam
in November, December and then you had from January to October
to do something else. And some people have travelled
then in my case, I went and worked at Rolls Royce for that
period. And that, that was really to
learn well, well, to see what engineering was like, see if
that's something which interested me.
So at age 18, I was an Rolls Royce undergraduate engineering
apprentice. That was my, my job title.
And normally they wouldn't, wouldn't have taken a
mathematician on, you know, usually it, it was engineers,
maybe people in materials. But I was actually third
generation Rolls Royce. My, my mother was a programmer
before I was born. She was a programmer with Rolls
Royce in Derby. And my grandfather worked,
worked for the company for more than 25 years in Glasgow.
So back in those days, you know, the application forms asked if
you had any family members working in the company.
I mean, these days that, that that would be nepotism that's
strictly forbidden. But back in those days that that
was viewed as a positive thing. And so I think because of that,
they, they, they took me on as an apprentice because I was a
mathematician. They weren't exactly sure what
to do with me. So, so I went through a lot of
the standard training with, with the engineers then, whereas the
engineers had to be moved around different parts of the company
to satisfy, you know, requirements for chartered
engineer status. You know, later on, you know, I
had, I had more flexibility as, as to what I did.
And so I got into various assignments which involved
programming in in various forms. Oh wow, that's interesting.
So this is before you did your undergraduate?
Yes, this is the period after school.
Before undergraduate and then each summer while I was at
Cambridge, each summer I went back to Rolls Royce for, for
another two months. So it was after my second year.
So that would be 1980 that I joined what was essentially the
CFD group, you know, only a few months after it was first
created, you know, so it was called the theoretical sciences
group. And that that summer I was doing
2D grid generation using conformal mapping.
So directly using, using my, my, my coursework in complex
variable theory. That was actually a paper, I
think it was written by Bob Nee at Pratt and Whitney.
I don't know whether you recognise that, that name, but
Bob's a really, you know, senior figure of that of that era.
Oh. Wow.
OK, so you did. So you're going between, but in
the during your undergraduate, did you already therefore get a
sense that you wanted to go down the more CFD route because you
were doing in Rolls Royce? I imagine because you were doing
maths you didn't touch. Well, I guess would you do fluid
dynamics in? I guess you would do numerical
methods, but would would fluids come up?
So. So we did lots of theoretical
fluid dynamics. So I can remember thinking, you
know, on on the one hand here, here I am taking a course
learning about invisid incompressible 2D flow over a
cylinder. And on the other hand, here I am
at Rolls Royce looking at these phenomenally complex, you know,
engineering devices. You're clearly paper and pencil
cannot take you very far. So I was really sold on the idea
of numerical simulation at a very early age.
You know, a very, very early stage.
I mean, already by the time I went to university I'd done some
amount of programming. So I got into programming pretty
early in, in, in part through, through my mother.
I mean she, she was IT support at University of Sterling.
So in the holidays I actually did a little bit of programming
on one of the big academic systems down in Manchester.
Where was that in the big building?
When was that in Manchester? I mean, I, I didn't go there.
I mean, this was remote access from Sterling.
Yeah. So this was in, in, in the
sense, you know, the equivalent of what's now the Edinburgh
Apparel Computing Centre. Back in those days, it was the
Manchester centre. I don't remember what I did on
it. Nothing very significant.
But also while, while that Rolls Royce for those eight months
before going to Cambridge one one day a week, they sent us
along to what was then called Darby Tech to sort of keep our,
our brains ticking over doing various classes, including
programming. And so I, I, I worked in an IBM
system there, I, I wrote a code to do project critical path
analysis, which was great fun. So so I've always enjoyed
programming. What?
What languages would it be? I'm sorry if I'm asking a stupid
question, I'm trying. To.
Get back to like. So I think that must have been
Fortran. It was punch cards.
So that was my experience with, with, with punch cards that that
time at Darby Tech, it was, it was a cast off the IBM machine
from Rolls Royce. They, they, they donated it to
Darby Tech. That's, that's my recollection
anyway. So yes, I think that must have
been Fortran in, in Cambridge, we had the whole teaching lab of
desktop machines and that was basic, I think that we used.
So there was a, there was a numerical projects course in the
third year, which was very good, you know, and so that, that
really sort of solidified my, my, my interest in, in, in
computational methods. And then what about so I, I, I
have to ask you one, one question.
When I was doing a bit of research, I hadn't come across
this term before, but it's quite an esteemed senior Wrangler.
Am I pronouncing it? Correctly.
I guess you've looked at the Wikipedia.
And then I was like, then I started to look down and I read
fantastic stories about people being paraded around.
I don't know if that was the time.
So this was the top undergraduate of maths, is that
correct? Which which you you've got.
Yeah, there there was no fanfare in my game.
So yeah, I was just told afterwards by by my tutor, yeah.
OK, well, it's it's still it shows, I guess your your
ability. Were you, did you enjoy the, the
sort of Cambridge life, the collegiate life?
Was, was that something that you, I mean, now obviously you
know, you're at the other place, but was that something that you
made a, a strong impression on you and, and sort of motivated
you later on to, to ultimately stay in or, or go back to
academia? I mean, I, I enjoyed my time at
Cambridge. I spent a lot of time doing
orienteering while I was there. So orienteering was something I
did as a child, you know, from the age of about 12, you know,
for. Yeah, for, for those listening
to this who don't know about orienteering, it's effectively
cross country running, using a map to guide yourself through
forests and over hillsides and stuff.
So, yeah, a lot of my time at Cambridge was spent going off
the orienteering at weekends. I would say that I was as a
diligent student, but I wasn't particularly hard working.
Let's let's let's put it that way.
You know, I seem to remember my, my maths as being sort of a, a
nine to five activity five days a week.
And then, you know, weekends I was away.
I didn't really start working until I went to MIT.
OK, that, that that sort of sums up the difference between MIT
and Cambridge also, you know, grad student life and undergrad
life, I guess. Yeah, Yeah.
So you said the beginning that you'd, you weren't sure.
You, you were debating around theoretical physics, maths I
presume. Then as you went towards the end
of your undergrad and particularly with the summers in
Rolls Royce, you'd, you'd put aside the theoretical physics
and you were more moving towards, I guess the engineering
or applied mathematics. Would that be fair?
I mean, I think by the time I finished that initial 8 months
at Rolls Royce, I think probably the physics ideas had to a large
extent dropped out. Although I think equally I knew
I didn't want to work in industry.
So yeah. So maybe things were still
somewhat open. Yes, I I do remember, I'm trying
to think whether it would be end of my second year or early in my
third year. There was a talk in college by
an academic, A researcher just talking about the joy of being
an academic researcher and that that did strike me.
So, so that's one of those moments that I think, you know,
confirmed me in going as an academic path or, or at least
taking it further and doing, doing a PhD.
But you know, the move to MIT, my, my tutor told me about the,
the Kennedy scholarship scheme and encouraged me to apply, you
know, and so, you know, initially I went to MIT on, on
this one year scholarship thinking it would be a chance to
see the world. And then I would think about
what to do next. Having gone out to MIT, my, my
supervisor there found funding for my second year to, to finish
up the masters. And in doing that, I also came
up with a, what turned out to be a good idea for a PhD project.
And then, then he got more, more research funding for, for me
actually from US Air Force to, to carry on and do do the PhD.
So it wasn't. It wasn't the master plan by any
means, but yeah, yeah, I went out initially for one year and
ended up staying for 11. So what was the Kennedy
scholarship like? Because I spoke to somebody
else, Anthony. Oh, sorry, I'm confusing.
Oh, got a brain fog. Now the have to rise.
Now I'm just interested because I wonder if you'd actually come
across. Probably you haven't.
My my memory for names is terrible, but there wasn't
Anthony about my time. Yeah, that's why I'm.
Just. Wondering if you've in In
general, most of the Kennedy scholars were at Harvard.
There were very few at MIT and, and, and these days it's really
quite the exception to have anybody at MIT, which is a bit
of a shame I think. Yeah, Tony Pannell.
Did you ever come across Tony? Yeah, that that that name does
sound familiar. So what?
What? What's he doing now?
Yeah, So Tony, who I actually also interviewed, he's a really
great guy. So he he won the Kennedy
scholarship, went to MIT and I think it was in 80, mid 80s, so
similar time. And then he went to he then went
to work. He created his own company, but
he ultimately ended up in Formula One running the what is
now the Red Bull team. But he's now a professor at
Cambridge. Does all the aerodynamics and
CFDII wonder whether? Yeah, so, yeah.
So I was a Kennedy scholar 81 to 82.
Your and the name does sound familiar.
Basically, we we didn't really hang out with each other.
I didn't hang out with the other Kennedy scholars.
So yes, yes. I didn't really have have have
those connections, but it was a part time experience.
Oh, oh, yes, yeah, yeah. So, so, you know, it was Kennedy
scholarship that took me over to MIT.
And as it happened, and this this was really sort of
coincidence, my supervisor at MIT had Rolls Royce funding.
This was in the days when Rolls Royce was starting to sell
engines to the US Marine Corps for the Harrier, and they wanted
to be viewed as more of an international company.
And so as part of that, I think almost out of their marketing
budget, maybe there was a whole pile of research funding to be
spent to MIT. And and so some of this was was
going to my supervisor guy guy by the name of Tilt Tompkins.
So although my going to MIT and, you know, landing up with Tilt
as a supervisor was completely independent of Rolls Royce,
there was still that sort of accidental background
connection. So then my, my, my, my graduate
history at, at MIT is a bit curious.
So I did did the masters in 18 months, which is a bit faster
than normal, but not exceptional.
I then did did my PhD in 2 1/2 years, which is highly unusual
for MIT. So, So what happened there was I
was basically two years into my PhD and you know, it had gone
very well. I mean, this was, you know, the
project that me and Mark Drella did, you know, you know, the ICS
code, you know, 2D airfoil design code.
So the project was going ahead very well, but I got called into
the head of departments office about two years into my PhD and
told that my supervisor hadn't got tenure and would be leaving
in six months time and would I like his job.
Wow. So so I then had to finish up
really quickly that that I wasn't.
Expecting that. Wow.
OK, so. Yeah, yes, that, that, that,
that last six months was, yes, exhausting.
So then, so your your pH. What was the?
What was the end title of your thesis then for the?
Something like 2 dimensional transonic aerodynamic design
method. So did you work with or be
inspired, I guess by, by Professor Jameson?
Was there, was there any sort of link into?
I just, I always find it interesting when there's sort of
Brits going over to the US. Yeah, yeah, they go and work on
things. But it was independent.
There was no he didn't have a. Connection.
No, there was no connection at all there, although he, he, he
was aware of me. He was aware of my master's
thesis and actually told, told tilt type, you know, something
along the lines of, you know, they should have given me a PhD
for it, which it was flattering. But but no, my, my, my master's
thesis was using some mathematics WKB analysis to
understand some numerical wave propagation on grids.
And what it shows you is that if you have something like the
convection equation, if you have a poorly resolved wave, it can
actually travel in the wrong direction and you can get these
weird wave track wave trapping phenomenon.
Anyway, it, it, it, it, it was something that intrigued
Anthony. And so, so he was aware of me
already at at that point. I mean, I was obviously aware of
him, but the, you know, the, the stream tube idea that I had,
which was the basis of the ICS code, that was completely
different to anything, you know, in, in, in CFD at that time, you
know, worked beautifully in 2D. It had had no natural 3D Ext,
but, but for, for 2D wing design it, it, it was really ideal.
And so Mark, Mark Drella and I teamed up on that.
And in fact, for a while we were going to do a joint thesis.
We're actually going to write it up as a single document between
the two of us. And our thesis committee was
perfectly happy with this. And since our thesis committee
included the head of department, I assumed that this was all
fine. And then later on in the
process, the central university somehow, you know, learnt about
this and said there is no precedent for this, there shall
be no precedent. You know, you're not allowed.
And so very late in the process, basically, you know, Mark and I
had to sort of carve our work up into two separate pieces so that
we could write up two separate documents.
You know, so in, in, in the end, I, I finished up early because I
had to take over Tilt's position.
Mark, Mark finished up a year later then, you know, he did, he
got an academic position as well, you know, so we were both
hired. I mean, there was never any
question of one of us freeloading off the other one.
I mean, our, our thesis committee were perfectly content
on that point. So yes, so it's a curious
situation, but it meant that I, I took over Tilt's office, PA
software engineer, almost all of his students and all of his
research contracts, including the Rolls Royce research
contract. So, so at that point I was then
back into the Rolls Royce family.
And how old were you then? You couldn't have been that old.
25. OK, that's quite, that's quite
young then. In in the US system and
particularly at MIT, they they do hire a lot of people straight
from PhD into assistant professor positions because
given the tenure system if they decide that they made a mistake,
they just flush you out after seven years.
Whereas whereas in, in the British system, you know, we,
we, we, we like to see people get a good bit of experience
under their belt, you know, before we'll, we'll, we'll hire
them in, in Oxford. Yeah.
Very occasionally we'll we'll take people straight from PhD,
but it's very, very rare. Wow.
So you're 25, you're an assistant professor at MIT,
you've got APA, all that, and and now you're taking over the
Rolls Royce. So what what?
How do things progress from from there?
So looking back, actually the, the, the first thing is it took
me about six months, I think, to recover from burnout from having
finished up the PhD so quickly. But, you know, there were, there
were a certain number of plans that were already in place that
I sort of carried on supervising students.
But I guess during that first six months, I was thinking
about, you know, what, what was the first new thing I wanted to,
to, to do with Rolls Royce funding.
And you know, I want, I wanted to do something different.
I, I can't remember how much I talked to them to understand
what their concerns were at the time.
So that what, what, what they needed.
But what I decided to do was to develop a 2D unsteady CFD code.
I think that they had had some engineering challenges in I
think it was a military engine where in military engines
there's a smaller gap between the stators and rotors.
So they're they're they're more closely coupled stages.
And as that as such, that means you get a larger level of
unsteady forcing on on on the blades.
And I think they, that there had possibly been some engineering
project where they had major difficulties with that and they
really needed tools to analyse that.
I mean, this, this was still, you know, fairly early days for,
for, for CFD, you know, so, so I think at that point, you know,
people like Bill Dawes and John Denton had developed steady 2D
CFD codes, possibly even 3D, but but not anything unsteady.
So, you know, so I, I, you know, my first code for Rolls Royce
was one called Unsflow, which was 2D unsteady.
Initially it was wake rotor interaction.
So you were passing in the wakes through upstream boundary
conditions and then going into doing stator rotor interaction.
So you've actually got the moving blade rows, you know,
moving relative to each other that that was initially
envisited. I later made it viscous.
I can't remember now what I did for a turbulence model.
It's probably an algebraic turbulence model in those in
those days. Mid 80s, Yeah, yeah, I guess,
yeah. So the the the sort of unique
thing about onsflow was this thing called the time inclined
plane. So one of the difficulties in in
doing unsteady analysis in turbo machinery is the number of rotor
blades is different to the number of stator blades.
So you want to do a simulation that just has one blade passage,
but if you do it the natural way, that doesn't work.
You know, you don't have the right periodicity to, to to do
that. So the time inclined plane
involved, you know, usually you know, when you're at time level
N, you know, all the grid points are at the same physical time.
The time inclined plane, you know that, that, that that time
was inclined. So that, and you could incline
it in such a way that you then set up the right periodicity
condition to cope with this arbitrary blade count.
So yeah, that that, that that was kind of the the unique
point. And then, you know, there's also
some maths I did on non reflecting boundary conditions
that when you're doing these unsteady interactions with the
boundaries very close, you want the outgoing waves to go out to
not be artificially reflected from the boundary.
And so there's a whole piece of research on on, on that.
Steph, you take for granted now in a commercial solver.
Yeah, Yeah. So I mean really I was one of
the first people doing that in, in the context of compressible
flow CFD yes. I mean these, these these days
you, you've got things like absorbing boundary methods,
which is probably what you would use if you're doing far field
acoustics and electromagnetics. But that actually wouldn't work
well in in this context of closely coupled stages with very
close in boundaries. And were you always hands on?
Were you, you know, at that time, were you always that sort
of person who was programming it yourself?
You had students, but you were still very much yes and and sort
of a hands on programmer. Yes, that was hands on.
I mean generally the students were writing their own other
codes. So, so with Ansflow there was
this software engineer Bob Haynes.
So I don't know whether you recognise the name Bob Haynes.
He, he he was responsible for developing our visualisation
software. So visual two, Visual three.
Yeah. Name names from the past.
Bob's Bob's still at, at MIT, I think he still hasn't retired.
You know, he must be about 10 years older than me, something
like that. So, yeah.
So I did most of the onsflow development, but Bob will have
helped me with, with, with bits of that as well as doing, you
know, he did, he certainly did all, all of the visualisation
because if you're doing a, you know, 2D calculation, you know,
you want to have some nice, nice visualisation.
It's true then, true now, isn't it?
So this was, this was in, in the early days of Silicon Graphics
and there was a company called Stellar which was based just
outside, well, in the Boston suburb.
And I, I sort of got involved with, with them.
And so we, we, we had a couple of their, their machines,
beautiful machines, you know, multiple courses I recall as
well, which was unusual. I mean, it was a very early days
of parallel computing. And back then were you, you
know, as much as you were developing the code for accuracy
and the physics side, did you, did you still have a strong
interest in the sort of high performance computing side?
Were you were you always excited to try out different machines or
have access to the machines or did that come later?
No, I was always interested. So I mean, even even while I was
doing, you know, masters and PhDs.
So I think, oh, I'm trying to remember.
So it was so probably soon after my master's, I spent the summer
at NASA Langley at at at ICASE and down there I think I was
doing programming on oh gosh, what, what what would it be?
My mind's gone blank. There was there was Crane and
then there was the other company.
What was the other one called? So it was, it was before, you
know, the ETA 10. Oh, Cyber 205.
Yeah, that sounds, sounds maybe right.
So I guess coming out to CDC maybe so.
So I I had my first experience of supercomputing at at I case.
I don't think I did a lot there. Who was there at that time?
Because I've heard other people mention about this I case they
don't do it anymore, I don't think, but it was this wasn't
it. So the person who led it, I
think when I was there was Milt Rose, but I think he may have
retired not long after I was there.
And then for many years it was led by somebody whose surname is
Hosseini. I'm trying to remember what his
first name is. Possibly Youssef.
No, I'm not sure. Yes, yes, I case.
I mean, you know, there are lots of academics there.
Who else do I remember? Eli Turkel.
I remember he was, he was there, I think, you know, the summer
that I was there. And then, you know, we sometimes
saw the people in, in the CFD group there and oh, there was a,
there was a great, great person who headed up the CFD group.
Oh, gosh, my, my, my memory today is, is is poor.
Yeah. So, yeah.
So I had, I had experience with
this, I think it was a cyber 205 at at Langley at MIT.
I did a little bit of work on on the Thinking Machines CM5, you
know. That I remember at the time, you
know, there was a lot of discussion about, you know, this
is the future of massively parallel computing, you know so
I think it was 64,000 processors, but each of the
processors was incredibly elementary.
And then it got blown away by risk computing, you know, and,
and people putting together PC clusters, you know, so I think
DARPA basically bankrolled it for maybe five years, eight
years. But then, yeah, it wasn't
capable of of sustaining itself. And then and then, you know,
Cray, Cray really got established, you know, And so
Cray, Cray was the winner. Yeah.
I so Rolls Royce at one stage had the Cray.
So I don't remember the time scales.
So I don't remember whether Onslow was ever run on the Cray
or not. I think it's possible it was.
And were you travelling back when you were MIT working for
Rolls Royce, Did you come back to the UK, to Derby to sort of
have meetings with it or was it slightly sort of separated?
I mean, I probably came back twice a year at most.
So. And, you know, to some extent I
would be coming home to come home and see people.
And then while I was here, I would visit Rolls Royce.
And to some extent, you know, a trip might, might be motivated,
you know, more more primarily because of Rolls Royce, you
know. So, yeah, I guess probably twice
a year was the norm in those days.
So how did things progress then at the MIT?
So you were working on the Rolls Royce projects that, Yeah.
How did, how did, how did things evolve during that time?
I mean, as well as working on the Rolls Royce projects, I had
some amount of funding from US sources.
Not a lot, but things I think. Office of Naval Research Air
Force Oh, we, we, we bought the stellar machines with a grant
from DARPA. So again, DARPA was really
active in funding new technologies to see whether, you
know, these really were useful or not.
That that is probably the worst proposal I have ever written.
But but, but my excuse is by the time I submitted the proposal,
my temperature was 102 or 103. I was going down with glandular
fever. Oh, I've had that.
That's bad glandular fever, isn't it?
Yeah. So in in, in the way I was lucky
mine was sufficiently bad that I had to be admitted into
hospital. And so they then pumped, pumped
me full of, you know, various antibiotics and stuff.
And so I recovered well. I was ill for like 6 weeks or
something, it was horrible. Yeah, I was.
I was probably in hospital for a week or two and then I was sent
home and told to stay at home and and recuperate for like 2
months or something. And and So what I did was to
write up a 60 page document documenting all of onsflow so
that that that was the best documented code I ever wrote it.
It's always this, this rule that people hate writing detailed
documentation and, you know, just just laying out all the
details of the numerics, all of the things that your future
people modifying the code need to do.
You know, it's very hard getting getting students to do that.
So anyway, I yeah, yes, I'm trying to trying to remember
exactly when, when all these different things happened.
Yeah. So.
So you're progressing at MIT, your assistant professor.
Did you feel that you would always stay there or what
started to get into your head about coming back?
To so I, I, I had a good, good job.
Obviously they're, they're at MIT.
I was part of the gas turbine lab and there were wonderful
experimentalists there. And so, you know, we, we, we did
lots of good works of comparing numerics with experiment.
And there's one paper we, we, we have an unsteady heat transfer
where in the sense neither the experimentalists nor me had
great faith in our own research and yet the results matched
wonderfully. We were delighted.
I, I did feel to some extent, not quite a fish out the water,
but really I am at heart a mathematician.
You know, you, you asked me at, at the beginning, you know, was
I the kind of kid that tinkered with devices?
No, I wasn't, you know, so I, I, I really wasn't by nature an
engineer in that sense, by by nature, I'm an applied
mathematician who enjoys mathematics and enjoys seeing it
being useful in the real world. I'm not fundamentally at heart
an engineer. And so in that sense, I did feel
a little bit constrained being in the aeronautics and
astronautics department, which is, you know, where I was at
MIT. And that that I was sort of most
aware of when doing things like the non reflecting boundary
condition theory, because that was very much maths theory, but
publishing it in in engineering CFD, So journals, you know, AI
AA journal. So that yeah, I think also, you
know, getting married and thinking about where, where you
want to have a family and raise kids and things like that.
You know, that that that was part of it.
And then the other part of it. And I forget the exact sequence
of all of this was on one of my trips to Rolls Royce, I was
called in to see the chief engineer who said, oh, by the
way, please let us know whenever you want to come back to this
country and we'll sort it. So I had that standing offer
from Rolls Royce that they would organise it for me or that I
could basically choose where I came back to.
So yeah, so I decided, yes, I wanted to, to, to come home and,
and, and I chose Oxford rather than Cambridge as the place to
come to partly. Well, you see in, in Cambridge
I'd have probably been in engineering, not in maths.
You know, Cambridge maths was always kind of anti numerical
analysis. There was certainly a chunk of
the faculty who felt that you know the computer is what you
used if you weren't clever enough to do it properly.
Old school. Old school, yes, whereas Oxford
really had had embraced numerical methods and had very,
very strong group here and in those days the numerical
analysis group, although they were a group of mathematicians,
they were in the computer science department.
And so there were also people in computer science on the parallel
computing site. And so that that was an
attraction to me as well, you know, to have them as sort of
neighbors. As it turned out that that
didn't work out because soon after arriving in 92, it was
thefirst.com boom and all, all of the parallel computing people
left to set up companies. So anyway, but but that was part
of the motivation of choosing Oxford.
Ah, OK. So yeah.
So, you know, Sir, Rolls Royce kind of organized it.
I mean it, it had to be a properly advertised and competed
for a position. So I was in this strange
position of helping to write the job description, possibly even
the advert for a position that I then applied for.
You know, there are various parts of my career which I, I, I
look back on. And now, now I realise just how
peculiar they were at the time. That's right.
So you technically came into the computer science department?
Yeah. So in those days it was called
the Computing Laboratory. Yeah.
And that was a Keble Rd. always OK.
And so that was like a Rolls Royce.
So it was a Rolls Royce readership in CFD and then and
then they funded me to set up a whole research group.
So I, I have a lot of funding from them for a, you know,
prolonged period. So really from 92 through to
2008 is when I moved to maths, you know, so, so, so for those
sort of 15 years, you know, I, I had a lot of Rolls Royce
funding. So this is the bit that I was
wondering about. So you came into computer
science for CFD and is this the time then when I guess what most
people recognize as the Hydra code is, is this where it begins
or did it actually begin even when you were MIT?
When, when would when would you say was the start of that?
Journey. So I would say that Hydra proper
started in about 96, but there was there were various bits of
research that in the sense laid the foundations for Hydra.
So while I was still at MIT, you know, so I was thinking about
what to do after onsflow. And so my plan after onsflow was
that we really wanted something that would be a design tool for
doing complete engines, you know, so no no longer single
stage, really looking multi stage.
I was also interested in the idea that the unsteadiness could
be done from a linear perturbation point of view
rather than doing non linear unsteady.
So so also linearize the unsteady equations, look at, you
know, harmonics. And so doing this for both
flutter and forced response. The key technical issue there is
whether it was legitimate to do linearised harmonic analysis of
shock capturing. And so I had a student who did a
project at MIT to prove that, show that that was was a
legitimate thing to do that as long as you so slightly smeared
the shock over a few grid points that that the linearised
analysis did do the right thing and did, yeah, you know, you
could do it on that basis. So that was the precursor work
at MIT. And then early on in Oxford, I
don't think I'd started this coding before I moved.
I did a code called Slick SLIQ. So steady, linear and quadratic.
So the idea was you did you know the non linear steady state, you
did a linear perturbation analysis for the unsteady
effects and then the quadratic was to it's kind of a formal
asymptotic expansion to get the mean flow changes due to the
second order quadratic effects. So, yeah, so I developed slick
early in, in in my Oxford days and and you know, I had a
student who who, who worked on that with with me.
So that was one piece of work. Another piece of work was back
in the 9293 era, there was funding.
I think this came from, you know, the UK Department of Trade
and Industry DTI. In those days there was an
initiative of setting up parallel application centres in
various parts of the country. And so I, I had a colleague on
the computer science side in Oxford, Bill McCall, who'd
applied for that funding even before I arrived.
And so had funding for an IBM machine, something called an SP2
and and also there was funding there for, for research and also
also match funding. I think so.
So I had 5050 matching from Rolls Royce and so did the
project there on developing. So in the sense of support layer
for doing distributed memory parallel computing.
So this was something called, you know, we called OO plus
Oxford parallel library for unstructured solvers.
Although although orally O plus doesn't doesn't sound right, but
written down it looks good. So, yeah, so, so I think Bill,
yeah, must have been in Oxford after I arrived from maybe two
or three years before he left in that.com boom.
He was one of the people I hoped to work, worked with a bit more
on the parallel computing side. So, so we got a whole pile of
funding half and Rolls Royce half from DTI to develop this
parallel application framework. There was another code that at
the postdoc Paul Crompton, who actually did most of the
development of that software and wrote the CFD code as basically
a test bed to check everything worked correctly.
But that was never intended as ACFD code for Rolls Royce.
So then that's what led into Hydra.
So basically there were the the ideas of doing steady post
linear perturbation tested out in slick and there was the
parallel framework in O plus that we'd developed thoroughly
tested out. And so Hydra then what was built
on those foundations now. So Hydra it dropped the idea of
doing the quadratic piece. So, so it's non linear steady,
or at least in its original incarnation, non linear, steady
linear perturbation for flutter forced response and then
adjoints of all of those for for the design optimization, both
steady and and the unsteady aspects, you know, so the
adjoints was completely motivated by the work that
Anthony Jamieson was doing, you know, on on, you know, the
aircraft side, except that I chose a different technical
approach in as much as Anthony always viewed the adjoint being
developed at the PDE level, you know, formulating the adjoint
PDE and then thinking about how to discretize it.
Whereas I followed the the so-called discrete adjoint
approach, where where you take the non linear discrete
equations, you linearize those sort of element by element and
then you take the transpose of the matrix to define the the
discrete adjoint. You know and.
Yes. How did you, just out of
interest at this time, maybe also MIT, but Oxford, how active
were you in, you know, the AI AA, the the sort of turbo
machinery conferences were you, were you sort of always, were
you at these events and saw these people or was this more of
a like direct industrial engagement?
I'm always interested, like now I go to the AI AA conference.
I'm always, just always wondering what it was like, you
know, before and was that still the main venue I guess to go.
Yes, yes, that, that, that, that was the main venue.
I, I sometimes went to the ASMEIGTI conference, but I went
more, more to the AI AA conferences.
You know, I think there was more discussion of CFD at at AI AA
and in, in, in particular, you know, the AI double ACFD
conference. I mean that that was really my
home, you know, sort of during this period.
I would say that, you know, the ASME was more on the application
side. Yeah.
So I, I, I, I can remember going to two or three of those, but it
was the AI AA ones which really were.
Where was was this Reno? No.
Where was the CFD? So Reno was the January
conference, the CFD conference, what was in the summer.
So I I remember well, I remember one in Snowmass in, in Colorado.
Yeah. Beautiful venue.
Yeah. Good, good.
Orienteering, right? Well, yeah, yeah, except it's
high, high enough up that you really wouldn't be wanting to
run at that altitude. No, Yeah, yeah.
I mean, this is this is a ski resort that in the in the
summer. You know, they, you know, are
used for conferences because there aren't that many people
who want to go hiking. But I remember Snowmass.
I remember Hawaii. I remember there was 1 in LA.
Yeah. I mean, it just hopped, hopped
all over the place. Yeah.
So. Yeah.
So I was really mainly in those days going to engineering
conferences, not so many, I guess some maths conferences,
but, but in those days, yeah, most of my publishing was in in
engineering journal still at that point.
But did you struggle? I'm interested because I,
because you have such a strong maths background, did you ever
struggle getting accepted or being in that blur of what
journal, what conference is the right place for maths?
And then I always find this interesting that, you know, it's
something too applied or too fundamental.
Did you find a sweet spot or was there still a little bit of a
straight frustration that your deep maths wasn't understood by
everybody? Do do you know what I'm getting
at or was it not an issue then? I mean, I, I think I was
probably in those days still viewed more as an engineer than
than the mathematics, you know. I mean, you know, if you have a
PhD in in aeronautics from MIT, you know, you're, you're an
engineer. Yeah.
So, I mean, there was never a question of the engineering
community not accepting me. You know, I think, you know, my,
my evolution has, has been one of the of the mass community
accepting me. OK, OK.
Yeah. So, yeah.
And that, that, that I guess, you know, happened more once
once I moved over into the Maths Institute.
But yes, I guess I felt, I mean, having made the, the move to the
numerical analysis group in Oxford, then in a sense I felt I
was back amongst mathematicians. But but I was still very much at
the engineering end of of of the group.
You know, I think I was probably 40 before I wrote my first paper
that had a theorem and a proof in it, you know, So I mean that
that's kind of needed to be a mathematician.
So what was the link to the engineering department at that
time like because there's was, was it the case that that was
more experimental work and the sort of CFD was mainly computer
science? Is, is that sort of, I always
found that unique in Oxford that, you know, like I said, MIT
engineering where like, yeah, I guess Oxford's a bit different.
So in in Oxford at that time we had three UT CS.
So the UT CS are the university technology centres that, that
the Rolls Royce set up. And so I had mine in CFD, there
was an experimental one in heat transfer in, in, in engineering
science. And then there was one in
materials which I didn't didn't have any interaction with except
one of the profs. There used to be an orienteer
back in my case. So there was.
A UTC just for CFD. Yes, yes, yes.
So this was my, my own little UTC.
I was the only academic in it. So and, and possibly within the
Rolls Royce family, that's slightly unusual to have a UTC
that only has one academic. Generally they're they're bigger
than that. But it was a funding mechanism,
I guess, and you were doing the work.
Yes, the funding mechanism it it, it involved me in all the
UTC directors meetings. I mean, you were part of the
family. And I mean that that's kind of
important in the sense that Rolls Royce really knew how to
work well with academics, that you were part of the family in
the sense that you knew all the problems as well as, you know,
the, the achievements of, of, of Rolls Royce.
You know, they, they, they, you know, they didn't hide anything
from you so that you could think about what you might potentially
do to, to, to help them address some of their challenges and
things. So, you know, it really was a
very good collaborative experience when, when you're
working with industry, you know, it's important that both sides
realise that what the other wants out of the relationship
is, is different, you know, and, and, and so you're always
looking for this sort of win win arrangement, you know, the, you
know, so they understood that, you know, for us, it was
important for the students to publish papers, to write
dissertations, you know, that there would be times when they
would be utterly focused on writing their dissertation and
not doing any more research, you know, but equally, I, I
understood what Rolls Royce needed out of the relationship,
which was primarily software, but occasionally if, if there
was a particular engineering thing to be investigated, you
know, we may occasionally do some, no special calculations
just for them, as it were, rather than as, as part of the
research. Yeah, I always found that
interesting that some it takes a special company to understand
the value of academic engagement and have the patience and the
long term vision that it's not just cheap labour.
Yes, you know that requires you. You have to be very careful to
make sure it's never just cheap labour.
I think Rolls Royce was possibly slightly disappointed that they
never ended up being able to hire any of my students.
You know, that's, that's, that's the other thing that, you know,
Rolls Royce would ideally like from a UTC is as a source of,
you know, people to be employed, you know, and that that never
quite happened. But, yeah.
But certainly they, they, they, they got their money's worth in
terms of CFD codes. So, yeah, and I guess like in
today's world probably, you know, you'd create a start up or
something, write a code, you know, I guess at that time there
was a more traditional link to the to the company, right?
You know, in terms of. Yeah.
I mean in, in, in this area, it would be tough to do a start up.
I mean, I guess 11 Alonso's done, done the startup.
I haven't talked to him recently as to how, how that's going.
I mean it, yeah. At, at, at one point in parallel
computing, I, you know, once I moved out of CFD into into
mathematical finance, I, I tried setting up a spin off.
No, it's just more hard work. Yeah, I'm, I'm, I'm, I'm
fundamentally an academic, not, not a start up guy.
The reason I say it is because, you know, speaking now to
yourself or, or to Anthony, and I guess that like now CFD has
become very dominated by these huge multibillion dollar
commercial companies. I guess in the 80s, that was
before the time, wasn't it? It was before the, the Fluence
and, and, and the, the open phones and it was still, you
wrote your own code. I assume that's old.
Was that one of the reasons for the hydrogen development that
they couldn't just buy something off the shelf that wasn't a
company that could just sell them a capability they felt they
needed? Yes.
And even today I don't think there's a company that could
sell them what they need because the turbo machinery requirements
are really very specific. So I mean I've I've not kept up
with the discipline. So I don't know what ANSYS
Fluent has as a capability these days, but you know, the ability,
for example, to have flutter calculations being performed on
a single blade passage with an inter blade phase angle between
the passages. You know, that's such a unique
requirement of turbo machinery that I'm not sure the answers
views the market as being big enough to develop that
capability. And then, and then you get into
things like real gas effects, you know, so, so, so Hydra
doesn't assume a fixed gamma. You know, it, it, it has a
general, you know, energy temperature relationship in
there. So again, I mean, I guess things
like that answers could add in, but there will be add insurance
for particular customers and, and they would charge
accordingly. I mean, one of the reasons I got
out of the CFD business is it's not clear to me long term, you
know, what Rolls Royce will do for their next so CFD code.
So I mean, Hydra's just celebrated its 25th anniversary
at, at, at Rolls Royce. I would say it'll remain the
corporate, you know, primary code for at least another 10
years because it takes a long time to, to introduce a brand
new code. And you know, they, you know,
they're doing work with Spencer Sherwin on his Nectar code with
the thought that they will use that in, in applications where
they want to really do DNS or or at least high resolution LES,
but, but that won't be part of their sort of standard, you
know, design process. So I think it's, it's harder to
see now. Yeah, companies funding a brand
new code development. That's kind of what I was
hinting at, that it does seem as if that that generation in the
80s and the 90s throughout aerospace was a time of real
innovation and development. It was.
Themselves. Yeah, I mean when, when, when I,
you know, joined Rolls Royce's first CFD group in 1980.
You know, most of the design methods were one-dimensional
design methods, you know, with sort of mean, mean line, sort
of, you know, I mean really mathematical models rather than
numerics. So, yeah, I mean it, it, it's
been wonderful just being part of that whole process of, you
know, you know, the development of computational engineering
and, and seeing it go all the way from 1D2D3D, you know, in
viscid to viscous steady to unsteady and then the adjoints
and everything. You know, it is interesting.
So looking back over your lifetime and looking at the
progress in computational engineering and then of course
the progress in parallel computing and the high
performance computing, I mean. Both of them, yeah.
It's really stunning looking back over the years.
Yeah, that's what I was wondering about is so you, so
you came into Oxford doing the computer science.
I was just looking at the and I guess this is where we have a
slight shared connection into some of the people.
How did and you mentioned also about the government setting up
some, you know, parallel computing.
I guess how did that evolve the HPC angle, Oxford, maybe some of
the E research ideas you'll move into mass.
When did how did that time? And I guess that probably aligns
to also when you had a declining interest in CFD and more into
the computing and other areas. Is that fair to say?
Were they aligned a little bit or were they sort of separate?
OK, so yeah, yeah, let's let let's.
Try to. I've asked you like 3 questions
in one. Yeah, yeah, yeah.
So get get the timeline straight in my mind.
OK. So the main Hydra research
period was sort of 96 to maybe 2004, 2006, something like that.
And the Hydra code was built on top of this O plus parallel
layer that we had developed earlier as part of the DTI
funded activity. And so incidentally, both O plus
and Hydra, you're the IPR belongs to Rolls Royce.
So O plus was very much developed in the time of single
core CPUs, risk risk based CPUs, you know, so, so there, you
know, there was good performance there, but single core.
And then what was happening over time was that CPUs were, were
going multicore and, and then we had GPUs that came along.
So, so again, I've, I've always been interested in the latest
computing technology. So I think it must have been
sort of late 2006, early 2007. An ex colleague, Mike Rodger, I
don't know if the name means anything to you.
He, he set up a company spin off from Warwick actually, which
sold parallel, you know, systems, you know, parallel
clusters. And he was the one who said to
me, there's this new thing called the GPU and this, this,
this new language CUDA. Actually, I'm not even sure if
he told me about CUDA. He may just have told me, you
know, there's this new hardware called GPUs.
It's really impressive. You know, you ought to have a
look at it. And at about the same time there
was a company based in Bristol called Clearspeed, which had
people in it who I think had the background coming from the in
MOS transputer days, you know, so, so I first tried the the
Clearspeed card and I was impressed by what it was capable
of, but I wasn't blown away by it.
And then actually I had the visiting student, a Chinese
student, I forget how, how he came to me, but he, he did the
clear speed work. And then I said, well, now, now
let's try this GPU. And he came back to me with it
within like a couple of weeks with the code performing just
incredibly fast. And I was saying, OK, you've
clearly messed up the timing. And so it took took me a week to
convince myself, no, I mean it really was performing that that
well. So, so this was right at the
beginning of CUDA. So I think we, we started with
the naughty .9 beta release or something like that.
And, and so I was just stunned at, you know, the power of, of
the GPU. Now this was when I was already
starting to pivot into mathematical finance.
So I was actually looking at it for doing Monte Carlo
simulations in, in finance, which is why NVIDIA then got
interested in me because they saw that as a potential market.
So, yeah, so, so I was pivoting to finance, but I was also
interested in the power of the GPUs.
So after two or three years of that, I'm trying to think
exactly on the timing. Again, I was interested in doing
an upgrade to O Plus which could then incorporate both GPUs and
multi core CPUs. But by the way, just before you
go on, did so did you if it was 2000 and six 2007, did you
interact with Ian Buck at all then on the cooter stuff or was
it on people more in the UK who? You no, no.
So Massimiliano I, I interacted with Ian.
Ian, I probably met in in the same way that I met Jensen at,
you know, drinks things at GTC. So, so I was, I think the number
2 CUDA fellow. I think.
I think the first CUDA fellow was somebody in India.
And then I was #2 like a month later.
So yeah, yeah, I mean, yeah. So I was going along to the GTC,
you know. I have to say to the, I guess to
the people is that a bit of like self-proclaimed interest just
because now the team I'm in has those people in it.
So I'm always intrigued when they're like, oh, you, you know
that person and this person knows this person.
So it's kind of interesting, especially because you were, I
mean, now everybody knows NVIDIA and GP us.
But 2006 seven was really early on, wasn't it?
Yeah, yeah. And, and in fact, some of the
people from Clear Speed moved to NVIDIA in part in influenced by
my feedback to them. Yeah.
They then saw the writing on the wall and moved, moved ship
accordingly. So, but yes, I don't think I,
yeah, I don't remember talking to Ian Buck at all.
Massimiliano was was was one of my early contacts in in in the
Bay Area. Yeah.
So, yeah, yes, I sort of came back then to do this upgrade
from O plus to to OP 2. So that, that was with Gehan,
you know, mutilation and Ishwan regularly that you, you, you
both of them. So, and so that, that had
funding from both Rolls Royce and, you know, the UK EPSURC,
you know, government funding. So that has in a sense, you
know, protected Hydra's future by, by up upgrading the, the,
the underlying computing hardness for, for, you know,
modern systems. But but I haven't really done
CFD as such since about 2/2/2000.
Seven, I think was the last time.
So what was, yeah, maybe that's the elephant in the room.
Why what, why was the, I mean, you sort of hinted towards it
earlier, but what was the reason to maybe move away from CFD and
move into the, you know, the financial side?
Was that the maths angle moving to the maths department sort of
just wanting new challenges? The computing side What?
What was the ingredients to that?
Yeah, I'm just, I'm just thinking what, what, what what
to say. Yeah, let's, let's let's be be
open about it because I have been been open about this, you
know, in, in various settings. I got, you know, more than burnt
out developing the hydrocode. You know that there were lots of
aspects of the hydrocode that worked very well, but they had
major problems early on with numerical stability which got me
horrendously stressed out to to the point of serious
consequences. So, you know, for about 6 months
it took to, to recover from all of that.
And I decided at that point that it's a got to a situation where
in a sense, too much perspiration and too little
inspiration. It, it, it's not being fun, you
know, managing a large software project is, is tiring, you know,
and, and, you know, I'd, yeah, I'd, I'd had enough.
So, you know, I sort of finished up the things that needed to be
finished up, transferred things to Rolls Royce and they've,
they've continued, you know, the development subsequently.
So I decided I had to had to do something fresh different.
I had to get out of big codes. Yeah.
So in, in engineering style, I, I thought, OK, where's the money
in terms of where, where's the research funding?
What do, what do people care about health and wealth?
So, so computational biology and computational finance, those,
those were the two areas I contemplated moving into.
And computational biology, I, I know no biology.
So I'd have been starting from Ground Zero on that.
And I also wasn't convinced there was the research funding
in in that area, although I was proved wrong on that, you know,
you know, there is actually plenty of funding there.
Computational finance, there was a very good mathematical finance
group in the maths department but they didn't have a lot of
numerical expertise. So, so for me that that was a
natural fit. And So what I did was I actually
transferred from computer science into maths to join the
mathematical finance group. And then a couple of years later
the rest of the numerical analysis group kind of moved,
moved across behind me into maths as well.
And so now, now I'm, I'm head of the, the numerical analysis
group now, you know, so, so in a sense in, in, in the last few
years I've sort of transferred back from the finance group into
the numerical analysis group. As which ultimately sort of came
with you in the fullness of time from the computer science into
the into the. Maths, yeah.
When when when maths got its new building, there was the
opportunity for the numerical analysis group to move and it
was a take it to leave it kind of opportunity.
And you know, they, they, they, they moved.
And for people listening, it's a lovely building.
It's a very nice. It's a very nice.
Building also, you know, when I joined computer science in 92,
the numerical analysis group was half of the whole department,
you know, Wow yeah. So what happened over time was
the computer science side grew and the numerical analysis side
didn't. And so by by the time the
numerical analysis group moved over in 2910, yeah, they were
basically pushed out effectively, you know, you know,
you know, that it didn't make sense for them being in computer
science any longer. I mean, historically they were
there because that's where the computers were, you know, and,
and that, you know, that is a history that has happened
elsewhere. I mean, you know, numerical
analysis and Stanford for a long time was based in computer
science. And I think, again,
historically, it's because that's where the computers were.
I do always find this interesting and that's why I
just wanted to, you know, get the I get the quick history of
like places like the E Research Centre and others because in
some ways I understand the logic that where does some of these
places fit? You know, there's numerical
analysis, there's computers, there's high performance
computing, there's an engineering application, there's
a pure application was the was was that sort of E research
centre and I believe there were others around the country was
the initiative to try and bring them together to in a more
collaborative way Was that was that sort of the initiative?
I know that they've subsequently largely folded into other
departments now, but. Yes, I mean, it it there was a
massive funding initiative to fund this E science and yeah,
so, so Tony, hey was was the person in charge of that.
And I'm not sure what exactly the intent was.
How, how explicitly they wanted it to be an interdisciplinary
effort. Maybe.
Maybe they did so in Oxford. Yeah.
When, when, when they round up the usual suspects for high
performance computing. I was, I was one of the usual
suspects. And so I was one of the four
people who put in the Oxford bid, you know, to get OERC
initially, I mean, at that point and Trefethen was Tony's deputy
and then she later, you know, joined Oxford and and and became
head of OARC. So I think certainly computer
science in those days in Oxford was very theoretical, especially
after the panel computing people like Bill McColl had left
duringthe.com era to some extent in Oxford there were tensions
between engineering and computer science as to where some of the
more applied computer science activities should go, should
should go. And then OERC was, was just,
yeah, another, another location to have such things, you know,
so, so eventually it made sense for OOERC to be merged into
engineering. I think at the time computer
science expressed a strong view that they they did not want to
be the destination. So.
Anyway, OK, but maybe on to then more the maths side.
I mean, I'm, I'm intrigued because maybe to explain at more
of a higher level, if you can, what what are the similarities
between some of the mathematical finances and maybe CFD?
What are the, and obviously you're known for which, you
know, I only understand the basic basics of it, but the sort
of multi level Monte Carlo, which I as soon as I saw the
description, always like a multi grid.
It always makes me interested that there's some of these, you
see it with AI today that some of these CFD solutions to
problems are now being applied to new areas.
So what? Yeah, what are the sort of high
level things that makes the link between CFD and maths, I guess.
So in in mathematical finance, there's basically two kinds of
methodology. You can approach it from a PDE
point of view, where in in one sense you you have APDE that
describes the evolution of the probability density function
for, you know, a stock having a certain value at the time in the
future. And there's a corresponding sort
of adjoint of that to give the value of of various financial
options or there's the Monte Carlo approach where you
simulate lots of these different possible future trajectories of
of the stock and then and then say, OK, given that family of
solutions, what's the average pay off of your financial
option? So when I initially moved into
finance, I was focused on the PDE site because it's basically
convection diffusion Pdes. And I thought, hey, you know,
this is, this is a no brainer. I can just bring all the CFD
techniques over and, and, and do things here.
What I quickly found was other people have beaten me to it in
terms of moving over from CFD. So particularly Peter Forsyth,
University of Waterloo in Canada, who I think Peter had
come out of the oil reservoir CFD area if I remember
correctly. And so he he did a lot of the
pioneering work in terms of PDE methods, you know, numerical
methods for for finance. And so there wasn't, as it
happened so much, you know, left for, for me, as I had maybe
thought. And then I mean this, this was
over a relatively short period of time.
I thought that I would probably have to teach a course on Monte
Carlo methods, you know, because, you know, we, we, we
had still have an MSC in mathematical and computational
finance. And I, I teach numerics on that.
And you know, we would need to teach them about both sides.
And so I took a 2 day course put on by a couple of professors
from Columbia University in, in, in the US, put on in, in London
for, for London finance. People managed to, to, to
convince the, the, the department to, to pay my fees
for that. They, they, they, they gave me a
50% discount as an academic, but Even so, it was a costlier
course to attend. Anyway.
So I went to this course and they were teaching me about,
about, about Monte Carlo methods and talking about doing
sensitivity calculations. And I went up and talked to Paul
Glassman, the lead guy in in a coffee break and said, well,
this is all fascinating, but I presume that of course you, you
actually use an joint methods to to compute these sensitivities
more more efficiently. To which he said, what?
So yeah. So just pure, pure luck.
I was able to introduce adjoint methods to the financial Monte
Carlo community for doing sensitivity calculations.
So I did a paper with with, with Paul Glasserman in a finance, so
industry magazine really, rather than as a proper academic
journal. Because from my point of view
there was absolutely nothing new mathematically.
This was just a new application. This is a paper that went, went,
went by the title of smoking a joint, which helped helped.
It's it's notoriety. This is a journal or, or trade
journal that liked puns in their titles.
And so I would never dare do that in an academic journal.
But anyway, so yeah. So I so I got known for, you
know, the adjoint work, which is really sort of taken over in, in
in the finance sector. And then the multi level Monte
Carlo. Yes, I mean, you're right to
take, you know, the analogy to multigrid.
I mean, multigrid is such a fundamental part of CFD that it
was natural in getting into Monte Carlo methods to think,
well, is there anything analogous that that that we can
do here? And so, yeah, I came up with
with, you know, the multi level idea.
And it's one of those things that like multigrid itself, I
mean, it's such a simple idea. It really ought to have been
thought of ages before. But because I came in from a
different background, you know, this was, this was part of my
toolkit, you know, it, it, it was a fairly natural thing for,
for, for me to do. And so I've kind of been living
off that and extensions of that, you know, for the last 15 years.
And what about the computing side that the GPU side, you
know, is that is that being just because of, of, of an interest?
How much of that and forgive me for for not knowing, but how,
how much is that shaped Also on the maths side, how, how much is
that acceleration? You know GPUs to CFD is well
known but is it similar ID on the maths side?
So, I mean, most maths research just doesn't need lots of
compute power. I mean, these these days more
and more, you know, our, our students do things in Python
And, you know, maybe some will, will use the jacks package
within Python to get performance.
But a lot of work. No, nobody worries about, you
know, performance, you know, so I'm, I'm kind of an unusual,
yeah. So I, I, I still teach my, my
CUDA course every year with, with Wes.
You know, a few weeks ago I taught in, you know, a one day
open MP, you know, some mini course for, for PhD students
just to introduce them to this forgetting performance.
But there's very, very few students who are particularly
interested in, in that. It is an interesting question.
You know, where, where does that kind of work belong?
You know, to what extent does it belong in computer science or
maths or engineering? You know, I guess I'm somehow a
product of, of my time and it's not clear that there will be a
new generation of people like me, at least not in maths.
Yes. I'm not sure where, where the
next generation of me sort of lives, you know, so, so there's
Wes. Wes is in, in engineering now,
you know, having moved with, with, with OARC.
So, you know, Wes is now kind of my successor within the
university. He's, he's now Mr. HPC, you
know, there will be individuals in departments like physics and
chemistry and biochemistry who, who have their expertise in HPC,
but we don't really, you know, we're not a community as such
now, I would say, and we're still struggling a bit to
organise graduate teaching across the university.
I mean, this is something where Oxford and I think Cambridge
are, are poor compared to our American counterparts, where
you'll, you'll have graduate courses offered by one
department taken by people from across the university.
Yeah. We, we don't do enough of that
Our, our, our, our CUDA course is, is unusual.
I mean, this this year, I think we're currently up to about well
over 100 people signed up, of whom 65 are Oxford people and
another 35 externals, you know, and so the Oxford people do come
from across the university, but that, that's very unusual in, in
the Oxford setup. I'm, I'm, I'm trying to get more
of that happening. You know that we we have a much
more systematic training in, in advanced computing because you
know there are needs across the university.
I mean, I have to ask the obvious question, which is you
know, you've you've pivoted once I guess to the maps were you,
were you? And are you still tempted on the
AI given that it seems your skills are probably absolutely
perfectly aligned, which is strong maps under tech,
engineering and HPC? Yeah, So the quick answer is no.
So it, it, it is curious. I've I've got so three points of
contact with with AI. I've, I've got the, the GP US
I've got, you know, stochastic gradient method is very close to
the stuff I'm doing in Monte Carlo.
And there's the I joint, you know, so.
I mean, certainly at one point in the past, if you looked at
the source code for Pytorch, there were more references to my
adjoint publications than than any other academic, you know, So
it is, it is interesting how, how, how the adjoint stuffs got
got, got picked up. But, but no, no, it, it AAI is
for, for for a new generation. No, I'm, I'm, I'm not doing
another pivot. I'm, I'm I'm happy to keep
things going on on the HPC side. So I will continue doing things
with, with, with GP us I've, I'm, I'm interested in the
potential of Fpgas for doing finance calculations.
It it it's amusing when when I arrived in in 92 in computer
science, one of my colleagues then in in computer science told
me, you know, this is wonderful new technology Mike called Fpgas
is going to absolutely revolutionize everything that
you're doing. You know, you really need to
learn about it, and that's kind of still the message.
Yeah. I was just about to say they
haven't been right in the last 30 years.
Doesn't mean that they may not be right at some point in the
next 30 years. So.
And there's a story for GPUs from that.
It is that, you know, you were there early in the day, But I, I
guess it's Jensen always says that, you know, it's, it's taken
like 30 years to sort of get to this point where there's been
this massive, you know, use because of because of AI, but I
guess it was because of people who are early adopters it like
yourself. It's it's been a slow
progression, but suddenly it's all come together.
Hasn't. It, I mean, it has been
fascinating being sort of in, in, in the company of, of NVIDIA
during this whole evolution. You know, that, you know, I went
to all the GTCS in, in the early days and I, I remember being
there, you know, when Imagenet, you know, one, sorry, not, not
image net one, you know, one won the, you know, the competition,
yeah, for, you know, you know, for image recognition, image
clinic classification. And then the next year, the top
10 competitors won with, with GPUs, you know, and, and you
know, the way that NVIDIA pivoted was, was impressive, but
they were always on the lookout for the killer application.
You know, so right at the beginning they thought, at least
this is my impression that they thought that computational
finance might be the killer application.
And so that's why they were interested in in what I did in
implementing, you know, the first random number generator
on, on, on CUDA. You know, I think in, I think at
that point, the value proposition wasn't sufficient to
persuade the banks to take highly paid quants to, to
rewrite all of the software. And so the adoption rate was,
was, was slow. But yeah, yeah, the company was
always on the lookout for that killer application.
And, you know, when when they saw, you know, those early days
in AI, they thought, yeah, let's let's let's double down on this.
And, you know, first, first on the software side and then
increasingly on the hardware side as well.
Yeah, I mean, it is. Fascinating.
I just find that the CFD in some ways is still a niche industry
at a global level. You know the amount of money and
the amount of people. But I do always find it
fascinating that so many things have originated in CFD.
I mean, Ian Buck's PhD was CFD, right?
CUDA, as far as I'm aware, there's like a fluid dynamics
application, you know, like with yourself with the early days of
CFD. OK, now you've it's.
It's funny how I guess because it was one of the original hard
problems to solve. Yes.
I mean CCFDI guess was a prime driver for HPC for for a long
period, you know, So yeah, I mean, both in terms of as it
were open, open CFD and all, all, all of you know, the
nuclear weapons stuff as well. Yeah, which is sort of related
very closely, you know. So whereas now you know, within
the UK the biggest computers or the most IT spend is in the
banks, you know, you know, So you know, the amount of money
that Rolls Royce spends on compute per year, I would
imagine is, is less than any any one of the big banks in London.
I don't know that for a fact, but no.
Probably. That's the case, you know, of
sense. Yeah.
You know, so these days it it's the money in AI which is driving
the hardware development, you know, clearly.
And so, you know, you know, CFD is no longer that driver.
I guess CFD was also driving a lot of development of numerical
methods within academia, you know, and academia tends to be
at the sort of bleeding edge of the technology.
You know, again, all of those cheap bodies, Yeah.
Bright, bright young minds and well, seeing just how much they
they can squeeze out of this, this new hardware.
But yes, what are the big drivers these days other than
AI? Yeah, Yeah.
I mean that so much is is is based on that now.
Yeah, that's why I find it interesting that the and it's a
bit more of a controversial topic, which is the convergence
or the use of AI techniques for some of these disciplines.
You know, whether just as there's influences of using CFD
or numerical methods for AI, is AI got any use within CFD or or
you know, do you see it in the maths world that the actual use
of AI techniques or is it controversial or not sort of
proven? You know, just as you were
bringing in like techniques from CFDCAI applications in the
finance. So I mean, in terms of AAI
within CFD in general, I would say I'm a sceptic, but but to
some extent that may be that I just haven't spent enough time
to actually see what is is happening.
I think potentially the idea of an AI based turbulence model
might, might make sense. I mean, turbulence modelling is
such a challenging topic and there's so little progress I
think has been made in the last, you know, 25 years that AAI may
may be the answer there. I think in I think, you know,
for detailed CFDI think it it, it's going to need to continue
to be traditional CFD methods. You know that whether AI can
then get trained on the results of a large number of simulations
such that it does a pretty good proxy for the purposes of of
design optimization. Maybe I guess that doesn't
particularly interest or excite me.
But again, maybe I'm, I'm just showing my, my, my age.
I, I am interested in the impact of AI on research more
generally. So, you know, I'm really very
impressed by the capabilities of the latest ChatGPT, you know,
and you know, I think it really has improved hugely, you know,
over the last two years, say, you know, so the pace of
development is, is fascinating, you know, so, so I'm doing quite
a few experiments in different settings just just to understand
what it can do. So, so you know, I mean, just
yesterday I was getting it to, you know, tell me all about, you
know, Brownian motion, spatial white noise, space-time white
noise for, you know, for stochastic modelling.
And so put together a whole lot of information for me to, to, to
give to an MSC student who's doing a project with me, you
know, all all the way to it, you know, telling me about the
stochastic heat equation, providing me with some sample
code, implementing it in, in a simple, you know, finite
difference approximation. It, it tried to set it up to, to
use multi level. So understood about multi level
it it didn't quite get things right, but but it, you know, did
surprisingly well, you know, so I really wonder where we're
going to be in five or ten years time, you know, so.
Yeah. I think it's entirely possible.
I might even go as far as to say likely, that 10 years from now
you will have a proof assistant that is capable of looking at a
theorem and proof given the necessary background
information, and it won't be able to say with certainty that
the proof is correct. But it will on many occasions be
able to highlight bits in the proof that look dodgy or don't
like wrong. So yeah, So I, I, I kind of
think that AI, you know, you should absolutely never trust it
100%. You should always look at what
it produces. But the fact that it gets some
things wrong doesn't matter if it also gets some things right.
And it is, you know, it's very helpful.
I mean, another another thing I did with it a few days ago, it's
always irritated me with programming Intel CPUs that at
times their compiler is very poor at vectorization.
And so if you really want to get performance, you have to work
with the vector intrinsics, you know, the AVX 512 intrinsics.
And it's irritated me the fact that Intel doesn't provide you
with AC plus plus class that has operator overloading, you know,
to go along with all of those intrinsics.
And so I asked ChatGPT about this and it, it produced me
with, you know, it gave me AC plus plus class definition with,
with all of the operator overloading, you know, and I'm,
I'm not a good enough C++ programmer that I could have
done that myself. So just saved me, you know,
incredible amount of time. So things that in a sense are
are routine enough for a large group of people now become
automatically available to everyone else who isn't part of
that, you know, So, so as a productivity tool, you know, it
really is capable. So I'm, I'm doing this, I mean,
partly from, for my own benefit, partly also to just spread the
word amongst my colleagues, you know, to give them a range of
examples of, look, this is what AI can do for you.
Yeah. I mean, I agree.
It's, it's amazing what it, it's and and it's, it almost does
need people to evangelise or show it because if you haven't
seen it, you don't know. But once you see that, Oh my
God, this is. Yeah.
So I'm, I'm, I'm now kind of getting into that evangelism
group. Yes, Yeah.
It it it, it is fascinating. So here's maybe a question for
you as, as we get towards the end of this discussion, which
I'm sure we could carry on for hours, because I, I, yeah,
you've got so many interesting things that we didn't talk
about. But I'm I'm conscious of like
your, your time, but maybe more philosophical question, which is
if you were if you were now a undergrad.
Yeah. In today's world, what would be
your advice on a career trajectory, what to focus on?
Like, I know it's a very difficult question, but seeing
everything you've done, is there anything you would advise now,
somebody who is, yeah, 1819 or doing an undergraduate and is
thinking about, you know, academia, industry, Is it good
to work with industry to get that understanding?
Yeah. Yeah, it's tough because I mean,
I think the main advice is to try to do something that you
enjoy doing. You know, you know, you know,
you know, we don't want everyone to be doing the same things.
Everyone should kind of pursue, you know what, what, what they
like. If, if I mean if it was me again
now, you know, thinking of, you know, where I was as a teenager,
you know, I would, I would still probably be, be heading in into
mathematics. You know, I would.
I think by my nature I was always interested in
applications. So I think as a teenager I was
interested in maths because it helped me with applications in
physics that that was kind of the motivation.
I mean maths now one of the areas of growth I think is in
terms of mathematical modelling in medicine.
So, so I do think if, if, if I was maybe an undergraduate now
and, and thinking about where I wanted to head for PhD, you
know, computational methods applied in, in the sort of
medical area, I think may, may be an area right right now.
It's still challenging with the whole funding situation.
So, you know, we have difficulties linking up medical
departments with science departments and, you know, the
whole way the UK funding mechanisms work, but at least we
don't have the problems that, that, that the US has for us.
So yes, I mean that that's maybe what, what I would end up doing
if, you know, I was, I was 18 again.
Now I don't know. But I think the main thing is,
is to try to enjoy what it is you do.
And, and you know, don't don't worry too much about, about the
future, you know, you know, you know, when I started as, as an
undergraduate at Cambridge, I really didn't know what I was
going to be doing at the end. I think, I think now students
start as undergrads with much more of an idea of what they
want to do at the end of it. And maybe there's.
Pressure. Yeah, there is more pressure
now. I mean, back when I was an
undergraduate, so few people went to university that in a
sense you were, you felt, you know, guaranteed that you you
would get a good job at the end of it.
And people do do not feel that guarantee now, you know, so, so,
so I do, you know, I do understand and appreciate that,
you know, I certainly wouldn't discourage people from doing,
you know, you know, pursuing interests in programming.
I know, I know there's talk about AAI is going to do away
with all these programmers. No, I don't think so.
I think, you know, you, you, you, you still need people with
with programming skills. I think I would definitely
encourage people to develop their AI using skills, you know,
so and engage with these AI tools, you know, because they
they are hugely useful, you know, and yet at the same time,
you have to have the critical skills to look at what they
produce and ask whether it's right or not, you know, so, so
you absolutely still do need to have very strong understanding
of your technical area, but given that, you know, the AI
tools can, you know, make you more productive.
Would would you agree as well that I mean it's easy for both
of us to say this, but that the if you want to guarantee a job
for the next 60 years, having a maths and or engineering
background allows you to turn your hand to almost any
problems. And it's, it's quite a general
skill set that is desirable because you could turn to maths,
you could turn to CFD, you could turn to biology, you could turn,
you know, there's always a need for the sort of mathematical
simulation side of things. I mean, I, I, I think maths is
rightly viewed as, as a subject that has, you know, lots of real
world applications. I think, I think it's viewed
more so now than probably 30 years ago, you know, so you
know, you know, because of data science, because of AI.
You know, the importance of maths I think is much better
understood. We still at some time, you know,
have have trouble convincing politicians of, of, of that from
the point of view of funding. You know, we, we, we do feel a
bit hard done by, shall we say, in terms of supporting, you
know, you know, the underpinning maths that's so important for so
many applications. So yes, I think, you know, a
mass education gives you a very firm foundation for, for life.
But again, I would, I would emphasise doing things that you
enjoy, you know, you know, I think, you know, there are
people on, on the, you know, the creative side where I don't see
AI sort of taking over. I mean, whether it's creative in
the arts or being a chef or being a singer, you know, I
mean, yes, I mean, we'll see where, where, where AI gets to
in terms of, you know, generating your pop songs and
things. I don't know, yes.
I mean, maybe, maybe it'll turn out yeah, but people will, will,
will still want live performances and stuff.
No. So, so I think there's a lot of
stuff on the creative side which will will be important for the
future. But, you know, there's only so
much that AAI can do. You know, I, I think it, I think
it's best to think of it as a productivity tool that it's
important you engage with so that you you have those skills
to be productive. But don't, don't think that it's
going to suddenly eliminate huge numbers of jobs.
Yeah. Well, and you still you need at
least at the moment there's a huge market for people with HPC,
CFD, programming, maths backgrounds to develop these AI
models. So it's actually a a great time
in some ways to have those skill sets.
Yes, I'm not sure that we're, we're developing enough new
people in HPC, you know, and I'm not sure which degree programs
they're coming out of it. It's definitely not maths.
It's definitely not computer science.
Well, not, not Oxford computer science.
You know, places like Warwick maybe have a bit more or Bristol
maybe have a bit more of a focus on on HPC.
Now it's, it's a very actually good question because I guess by
definition, what is HPC is part of the problem.
And I, I would agree with you that.
And again, to link to the AI, it's even more important because
you know, as you know, big AI training clusters are
essentially what people would have called HPC classes before.
There is essentially no difference networking that you
Slurm or the OK now Kubernetes and things.
But but you're right, what course teaches?
Yeah. Yes, I mean, I know some of the
inside story on, on in Microsoft's development of large
GPU clusters and how they in, you know, involved a consultant
who is one of the world's leading HPC experts, you know,
so, so yes, absolutely. You know, doing, doing these
really large systems is is a massive HPC challenge.
It, it, it would be interesting to know, you know, the people
that NVIDIA hires as dev techs, what is their background?
You know, to what extent have they been formally trained in
HPC or to what extent is, has it been a passion?
And they've learnt on the job in various application areas and
they've, they've proved their skills and, you know, been hired
on that basis. That's that's a good question.
And I think maybe this goes not full circle, but half to our
discussion about with some of these OEOERC or these research
software engineers that at least what I see is that people get
their HPC skills often during their PhD when they're using a
HPC utility and they're sort of become best buddies with the
admin because they want to get higher up in the queue where
they have to sort of figure out some courses.
And so they haven't done necessary HPC course, but
they've had to do it to get access to the compute.
But somebody was there to manage the system.
So it's probably those like RSI think they call it.
Is it Rs ES now research software?
Engineers. Are sort of like this lifeblood
who support the people and are probably helping them.
That maybe is an underrated skill or need in the community.
But you're right, I'm I'm not sure of any HPC official
training, but maybe I need to do a bit of digging around to we
sort of take for granted stuff that we have.
We never realized that. What if all those people retire?
I suppose is your point, isn't? It like, yeah, yes, yes, I
wonder yes, where, where, where, where the next generation of
academics is, is, is coming from, but.
Yeah, yeah. But I, yeah, really appreciate
you talking. And I, I will put some links to
the the sort of episode notes because you, you know, I saw
you've got a great website as well where you link some of the,
you know, full list of your papers, some of the courses that
you're doing so that people could maybe read up because we
didn't get into all of your academic papers.
But I know you've done a good job of linking and some of the
presentations and things like that.
So I'll put it through. But yeah, I just want to say
thank you also collectively thank you because all the work
that you did in throughout your career has actually helped
people like me and others who work in industry to be able to
do CFD in an easy way that we take for granted now and don't
have to program 2G grids by hand like like you did.
So thank you from all the CFD people who take it for granted
now. And yeah, thanks for taking the
time to to speak to me. You're very welcome.
It's fun. Fun reminiscing about these
these things from days past. Yeah.
Great. Thanks very much.