All right, let's start a little bit more pragmatic, but then we'll Branch out. So one of the things I think a lot of folks here are interested in is, based off the apis that very large models will create, what are the real business opportunities like?
What are the ways to look forward? And then how, given the, the apis will be available to multiple players. How do you create distinctive businesses on him? Yeah, so I I think so far we've been in the realm where it's. You know you can do like an incredible copywriting business or you can do like a sort of like Education Service or whatever, but we I don't think we've yet seen the kind of like people go after the like you know trillion dollar like take on Google's, and I think that's about to happen. Like maybe it'll be successful, maybe Google will do it themselves, but like I would guess that with the quality of of language models we'll see in the coming years, you know there will be like a serious challenge to Google for the first time for for a search product, and I think people are really starting to think about like how did the fundamental things change? And that's going to be really powerful.
I think that a like a human level chat bot interface that actually works this time around, like I. I think, like you know, many of these trends that, like we all made fun of, were just too early- like the chatbot thing was good, it was just too early. Now it can work and I think you know, having like new medical services that are done through that, where you get great advice, or new Education Services like this- these are going to be very large companies. I think we'll get multimodal models and not that much longer and that'll open up new things. I think people are doing amazing work with sort of agents that can use computers to do things for you, use programs and this idea of like a language interface where you know you say a natural language what you want in this kind of like dialogue back and forth, you can iterate and refine it and the computer just does it for you.
You see some of this with like Dolly and co-pilot in very early ways. But I think this is going to be a massive Trend and you know very large businesses will get built with this as the interface and, more generally, that, like these very powerful models, will will be one of the genuine new technological platforms which we haven't really had since mobile and there's always like an explosion of new companies right after. So that'll be cool. And and what do you? What do you? Things are, given that the large language model we provided as an API service.
What are the things that you think that folks who are thinking about these kind of AI businesses should think about is: how do you create an enduring, differentiated business? So you know they're, they're. I think there will be a small handful of like fundamental large models out there that other people build on, but right now, what happens is, you know, company makes large language model API, other people build on top of it, and I think there will be a middle layer that becomes really important where I'm, like skeptical of all of the startups that are trying to sort of train their own models.
I don't think that's going to keep going, but what I think will happen is there'll be a whole new set of startups that take an existing very large model of the future and tune it, which is not just fine-tuning, like all the things you can do.
I think there will be a lot of access provided to create the model for medicine or using a computer or like the kind of like friend or whatever, and then those, those companies will create a lot of enduring value because they will have like a special version of. They won't have to have created the base model, but they will have created something they can use just for themselves or share with others, that has this unique data flywheel going, that sort of improves over time and all of that.
So I think there will be a lot of value created in that middle layer. And what do you think some of the most surprising ones will be? It's a little bit like, for example, you know a surprise from a couple years ago- and we talked a little bit to Kevin Scott about this this morning as we opened up- which is train on the internet, do code, right. So so what do you think some of the the surprises will be of? You didn't realize it reached that far. I think the biggest like systemic mistake in thinking people are making right now is they're like: all right, you know, maybe I was skeptical, but this language model thing is really going to work and sure, like images, video too, but but it's not going to be generating net new knowledge for Humanity. It's just going to like: do what other people have done and you know that's still great. That's still like brings the marginal cost of intelligence very low, but it's not. It's not going to go like create fundamentally new, it's not going to cure cancer, it's not going to add to the s total of human scientific knowledge, and that is what I think will turn out to be wrong. That most surprises the current experts in the field. Yep, so let's go to science. Then there's the next thing: what are some of the things? Whether it's building on the apis, you know, use of apis by scientists, where, what are some of the places where science will get accelerated and how? So I think there's two things happening now, and then a bigger third one later. One is there are these science dedicated products, whatever, like Alpha fold, and those are adding huge amounts of value. And you're gonna see in this like, like, way more and way more.
I like I think I, if I were, like you know, had time to do something else, I would be so excited to like go after a bio company right now, like I think you can just do amazing things there, the anyway. But there's like another thing that's happening which is like tools that just make us all much more productive, that help us think of new research directions, that sort of write a bunch of our code so, you know, we can be twice as productive and that impact on, like, the net output of one engineer, a scientist, I think will be the surprising way that AI contributes to science. That is like outside of the obvious models, but even just seeing now, like what I think these tools are capable of doing- copilot is an example there's, you know, be much cooler stuff than that- that will be a significant like change to the way that technological development, scientific development, happens. But then so those are the two that I think are like huge now and lead to like just an acceleration of progress. But then the big thing that I think people are starting to explore is- I hesitate to use this word because I think there's one, one way it's used which is fine, and one that is more scary, but, like AI, that can start to be like an AI scientist and self-improve.
And so when, like, can we automate? Like can we automate our own jobs as AI developers? Very first, the very first thing we do, can that help us like solve the really hard alignment problems that we don't know how to solve? Like that, honestly, I think, is how it's going to happen. The, the scary version of self-improvement, like the one from the science fiction books, is, like you know, editing your own code and changing your optimization algorithm and whatever else. But there's a less scary version of self-improvement which is like kind of what humans do, which is if we try to go off and like discover new science. You know that's like we come up with explanations, we test them, we think like we, whatever process we do, that is like special to humans, teaching AI to do that. I'm very excited to see what that does for the total like. I'm a big believer that the only real driver of human progress and economic growth over the long term is the, the structure, the societal structure that enables scientific progress and then scientific progress itself and like I think we're gonna make a lot more of that well, especially science that's deployed in technology.
Say a little bit about how what. I think probably most people understand what the alignment problem is, but it's probably worth four sentences on the alignment problem. Yeah, so the alignment problem is like we're going to make this incredibly powerful system and like be really bad if it doesn't do what we want or or if it sort of has, you know, goals that are either in conflict with ours and many Sci-Fi movies about what happens there, or goals where it just like doesn't care about us that much. And so the alignment problem is: how do we build AGI that that does what is in the best interest of humanity?
How do we make sure that Humanity gets to determine the- you know, the future of humanity? And how do we avoid both like accidental misuse, like where something goes wrong we didn't intend. Intentional misuse, where, like a bad person is like using an AGI for great harm, even if that's what other person wants. And then the kind of like you know, inner alignment problems where, like what if this thing just becomes a creature that views this as a threat? The the way that I think the self-improving systems help us is not necessarily by the nature of self-improving, but like we have some ideas about how to solve the alignment problem at small scale, and we've, you know, been able to align open ai's biggest models better than we thought we'd we would at this point. So that's good. We have some ideas about what to do next, but we cannot honestly like look anyone in the eye and say we see at 100 years how we're going to solve this problem, but once the AI is good enough that we can ask it to like: hey, can you help us do alignment research? I think that's going to be a new tool in the toolbox. Yeah, like, for example, one of the conversations you and I had is: could we tell the the, the agent, don't be racist, right? As opposed to trying to figure out all the different things where the weird correlative data that exists on all the training settings- everything else- may lead to racist outcomes? It could actually, in fact, do a self-cleansing totally. Once the model gets smart enough that you can, that it really understands what racism looks like and how complex that is, you can say: don't be racist. Yeah, exactly what do you think are the kind of Moon shots that, in terms of evolution of the next couple years, that people should be looking out for, in terms of, like evolution of where AI we'll go? I'll start with like the higher certainty things. I I think language models are going to go just much, much further than people think and we're like very excited to see what happens there.
I think it's like what a lot of people say about you know, running out of compute, running out of data, like that's all true, but I think there's so much algorithmic progress to come, that that we're going to have like a very exciting time. Another thing is, I think we will get true multimodal models working and so you know, not just text and images, but every modality you'd like in one model able to easily, like you know, fluidly- move between things.
I think we will have models that continuously learn. So, like right now, if you use GPT, whatever- it's sort of like stuck in time, that it was trained, and the more you use it it doesn't get any better and all of that, I think we'll get that changed. So, very excited about all of that and if you just think about, like, what that alone is going to unlock and the sort of applications people will be able to build with that, that, that, that, that that would be like a huge victory for all of us and just like a like a massive step forward and a genuine technological Revolution if that were all that happened. But I think we're likely to keep making research, progress into new paradigms as well. We've been like pleasantly surprised on the upside about what seems to be happening and I think you know all these questions about like new knowledge generation: how do we really Advance Humanity? I think there will be systems that can help us with that. So one thing I think would be useful to share, because folks don't realize that you're actually making these strong predictions from a fairly critical point of view, not just a, you know, we can take that Hill- say a little bit about some of the areas that you think are current, kind of illusionally talked about, like, for example, Ai and fusion. Oh yeah, so I like one of the unfortunate things that's happened is, you know, AI has become like the Mega buzzword, which is usually a really bad sign. I hope. I hope it doesn't mean like the field is about to fall apart, but historically that's like a very bad sign for you know, new startup creation or whatever.
If everybody is like I'm this with AI and that's definitely happening now. So, like a lot of the you know we were talking about, like are there all these people saying like I'm doing, like these you know RL models for Fusion or whatever, and as far as we can tell, they're all like much worse than what. Like you know smart physicists to figure it out. I think it is just an area where people are going to say: everything is now this plus AI.
Many things will be true. I do think this will be like the biggest technological platform of the Generation Um. But I think it's like we like to make predictions where we can be on the frontier, understand predictably what the scaling laws look like, or already have done the research, where we can say, all right, this new thing is going to work, and make predictions out from that way, and that's sort of like how we try to run open AI, which is, you know, do the next thing in front of us, when we have high confidence and Kate take 10 of the company to just totally go off and explore, which has led to huge wins, and there will be wait, like oh- I feel bad to say this- like I I doubt we'll still be using the Transformers in five years.
I hope we're not. I hope we find something way better, but the transform has obviously been remarkable. So I think it's important to always look for, like you know, where am I going to find the next, the sort of the next totally new paradigm and but but I I think, like that's the way to make predictions- don't, don't pay attention to the like AI for everything. Like you know, can I see something working and can I see how it predictably gets better, and then, of course, leave room open for, like the, you can't plan the greatness, but sometimes it had the research breakthrough happens, yep. So I'm going to ask two more questions and then open it up because I want to make sure that people have a chance to do this. The broader discussion- although I'm trying to paint the broad picture so you can get the crazy aspirations as part of this- what do you think? What do you think is going to happen vis-a-vis the application of AI to, like these very important systems, like, for example, financial markets? You know, because the very natural thing would be is saying, well, let's, let's do a high frequency Quant trading system on top of this and other kinds of things. What, what is it? Is it just kind of being a neutral arms race? Is it? Is it what? How do how?
What's your thought in? Like? It's almost like the life 3.0. Yeah, omega's point of view, yeah, I mean, I think it is going to just seep in everywhere. My basic model of the next decade is that the cost of intelligence, the marginal cost of intelligence and the marginal cost of energy are going to Trend rapidly towards zero, like surprisingly far, and, and those, I think, are two of the major inputs into the cost of everything else except the cost of things we want to be expensive, the status, Goods, whatever.
And and I think you have to assume that's going to touch almost everything, because these like seismic shifts that happen when, like, the whole cost structure of society change, which happened many times before, like the Temptation is always to underestimate those.
So I wouldn't like make a high confidence prediction about anything that doesn't change a lot or that where that doesn't get to be applied. But one of the things that is important is it's not like the thing Trends either, Trends all the way to zero, they just Trend towards there. And so it's like someone will still be willing to spend a huge amount of money on compute and energy. They will just get like unimaginable amount of intelligence, energy. They'll just get unimaginable amounts about that. And so like who's going to do that and where is it going to get the weirdest? Not because the cost comes way down, but the amount spent actually goes way up. Yes, the intersection of the two curves. Yeah, you know, the thing got 10 or 100, thing got 100 times cheaper in the cost of energy. You know 100 million times cheaper in the cost of intelligence. And I was still willing to spend a thousand times more in today's dollars, like what happens then yep. And then, last of the buzzword, Bingo, part of the the future questions: metaverse and AI. What do you, what do you see coming in this?
You know, I think they're like both independently cool things. It's not like totally clear to me, yeah, other than like how AI will impact all Computing. Yeah, well, obviously, Computing, simulation environments, Asians possibly, possibly entertainment, certainly education, right, you know, like an AI tutor and so forth. Those those would be Baseline. But the question is, is there anything that's occurred to you that's I? I would bet that the metaverse turns out in the upside case then, which I think has a reasonable chance of happening. The upside case, the metaverse turns out to be more like something on the order of the iPhone, like a new, a new container for software and you know a new way, a new computer interaction thing, and AI turns out to be something on the order of like a legitimate technological Revolution, and so I think it's more like how the metaverse is going to fit into this, like new world of AI. Then AI fit into the metaverse, but low confidence, the TBD, all right questions, hey there, how do you see Technologies, foundational Technologies like tpg3, affecting the pace of life science research. Specifically, you can group in medical research there and and sort of just quickening the iteration cycles. And then what do you see as the rate limiter in life science research and sort of where we won't be able to get past because they're just like laws of nature, yeah, something like that.
So I think the currently available models are kind of not good enough to have like made a big impact on the field. At least that's what like most like life sciences researchers have told me. They've all looked at it and they're like it's a little helpful in some cases. There's been some promising work in genomics but like stuff on a bench top hasn't really impacted it. I think that's going to change and I think this is one of these areas where there will be these like you know, new 100 billion to trillion dollar companies started those. Those areas are rare but like when you can really change the way that if you can really make like a you know future Pharma company that is just hundreds of times better than what's out there today, that's going to be really different.
As you mentioned, there still will be like the rate limit of like bio has to run at its own thing and human Trials take however long they take, and that's so. I think an interesting cut of this is, like: where can you avoid that? Like, where are the the synthetic bio companies that I've seen that have been most interesting are the ones that find a way to, like, make the cycle time super fast and that that benefits like an AI.
That's giving you a lot of good ideas, but you've still got to test them, which is where things are right now. I'm a huge believer for startups that, like, the thing you want is low costs and fast cycle times, and if you have those, you can then compete as a startup against the big incumbents, and so, like I wouldn't go pick like cardiac disease is my first thing to go after right now with, like this kind of new kind of company. But you know, using bio to manufacture something that sounds great. I think the other thing is the simulators are still so bad, and if I were an a, if I were a, bio means AI startup- I would certainly try to work on that somehow. When do you think the AI Tech will help create itself? Oh, it's almost like a self-improvement- will help make the simulators significantly better. People are working on that. Now I I don't know quite how it's going, but you know there's very smart people are very optimistic about that. Yeah, other questions, and I can keep going on questions. I just want to make sure you guys had a chance. This here. Yes, great, Mike is coming. Awesome, thank you.
I was curious: what, what aspects of Life do you think won't be changed by AI? Sort of did all of the deep biological things? Like I think we will still really care about interaction with other people, like we'll still have fun and like the reward. You know systems of our brain are still going to work the same way. Like we're still going to have the same like drives to kind of create new things and, you know, compete for silly status and, like you know, form families and whatever.
So I think the the stuff that people cared about 50 000 years ago is more likely to be the stuff that people care about, you know, 100 years from now than 100 years ago. As an amplifier on that, before we get to the next whatever, the next question is: what do you think are the best utopian science fiction universes so far? Good question: Star Trek is pretty good, honestly, like I do, like all of the ones that are sort of like. You know, we turn our Focus to like exploring and understanding the universe as much as we can.
It's not, this is not a utopian one. Well, maybe I think the last question is like an incredible short story-hyeah, that was what that came to mind. Yep, I was expecting you to say Ian Banks on the culture, those are great. I think science fiction is like there's not like one, there's not like one sci-fi universe that I could point to and say I think all of this is great, but like the collective optimistic corner of sci-fi, which is like a smallish Corner I'm excited about.
Actually, I took a few days off to write a Sci-Fi story and I had so much fun doing it- just about sort of like the optimistic case of AGI- that it made me want to go like read a bunch more. So I'm looking for recommendations of more to read now, like the sort of less known stuff. If you have anything, I will. I will get you some great some recommendations. So, in a similar vein, one of my favorite sci-fi books is called childhood's End by Arthur Clark, from like the 60s, I think, and the I guess the one sentence summary is: aliens, come to the Earth try to save us and they just take our kids and leave everything else.
So, you know, there's a slightly more optimistic than that, but, yes, I mean there's Ascension into the over mind is is is meant to be more utopian, but yes, okay, you may not read it that way, but yes, well, also in our current Universe, yes, our current situation, you know a lot of people think about family building and fertility and, like, some of us have different people, have different ways of approaching this. But, from where you stand, what do you see as like the most promising Solutions? It might not be a technological solution, but I'm curious what you think other than everyone having 10 kids. You know, like, how do we of everyone having 10 kids, yeah, how do you populate? How do you like, how do you see family building coexisting with you know, AGI, high tech, it's- this is like a question that comes up at open AI- a lot like: how do I think about?
You know, how should one think about having kids? There's, I think, no consensus answer to this. There are people who say, yeah, I'm not, I was gonna, I thought, I always thought I was gonna have kids and now I'm not going to because of AGI, like there's just for all the obvious reasons- and, I think, some less obvious ones- there's people who say like, well, it's going to be the only thing for me to do in you know, 15, 20 years, so of course I'm going to have a big family- like that's what I'm going to spend my time doing.
You know, I'll just like raise great kids and then I think that's what'll bring me fulfillment. I think, like, as always, it is a a personal decision. I get very depressed when people are like I'm not having kids because of AGI. The EA Community is like I'm not doing that because they're all going to die. They're kind of like a techno. Optimists are like, well, it's just like. You know, I want to like merge into the AGI and go off exploring the universe and it's going to be so wonderful and you know just, I want total freedom. But I think, like all of those I find quite depressing. I think having a lot of kids is great.
I, you know, want to do that now more than I did, even more than I did when I was younger, and I I'm excited for it. What do you think will be the way that most users interact with Foundation models in five years? Do you think there'll be a number of verticalized AI startups that essentially have adapted and fine-tuned Foundation models to an industry, or do you think prompt engineering will be something many organizations have as an in-house function?
I don't think we'll still be doing prompt Engineering in five years. I think it'll just be like you and this will be integrated everywhere. But you will just, like you know, either with text or voice, depending on the context you you will just like interface in language and get the computer to do whatever you want and that will, you know, apply to like generate an image, where maybe we still do a little bit of prompt engineering, but you know, it's kind of just going to get it to like: go off and do this research for me and do this complicated thing, or just, like you know, be my therapist and help me figure out how to make my life better.
Or, like you know, go use my computer for me and do this thing, or or any number of other things. But I think the fundamental interface will be natural language. Let me actually push on that a little bit before we get to the next question, which is, I mean, to some degree, just like we have a wide range of human talents right now and taking a look, for example, a dolly, when you have like a, a great visual thinker, they can get a lot more out of Dolly because they know how to think more. They know how to iterate the loop through the the test. Don't you think that will be a general truth about most of these things?
So it isn't that. Why would be? Natural language is the way you're doing it. It will be. There will be like almost an evolving set of human talents about, about going that extra mile 100. I just hope it's not like figuring out to like hack the prompt by adding one magic word to the end that like changes everything else. I, I like what will matter is like the quality of ideas and the understanding of what you want. So the artist will still do the best with image generation, but not because they figured out to like add this one magic word at the end of it, because they were just able to like articulate it with a creative eye that you know I don't have. Certainly what they have is a vision and kind of how their visual thinking and iterating through it. Yeah, yeah, well, obviously it'll be that word or prompt now, but it'll iterate to to better. All right, at least we have a question here.
Hey, thanks so much. I think the term AGI is used, thrown around a lot, and sometimes I've noticed my own discussions like the sources of confusion has just come from people having different definitions of AGI, and so it can kind of be the magic box where everyone just kind of projects their, their ideas onto it, and I just want to get a sense from you what, like how do you think you know, how would you define AGI and how do you think you'll know?
Yeah, it's a great point. I think there's like a lot of valid definitions to this, but for me, AGI is basically the equivalent of a median human that you could, like you know, hire as a co-worker, so, and then they could like say, do anything that you'd be happy with a remote co-worker doing, like just behind a computer, which includes, like you know, learning how to go be a doctor, learn how to go be a very competent coder, like there's a lot of stuff that a media human is capable of getting good at, and I think one of the skills of an AGI is not any particular Milestone but the, The Meta skill of learning to figure things out, and that it can go decide to get good at whatever you need. So for me, like that's, that's kind of like AGI, and then Super intelligence is when it's like smarter than all of humanity put together, thanks.
Just what, would you say? Or in the next 20, 30 years, are some of the main societal issues that will arise as AI continues to grow, and what can we do today to mitigate those issues? Obviously, the economic impacts are huge and I think it's just like if it, if it is as Divergent as I think it could be, for like, some people doing incredibly well and others not, I think Society just won't tolerate at this time. And so, figuring out when we're gonna like disrupt so much of economic activity, and even if it's not all disrupted by 20 or 30 years from now, I think it'll be clear that it's all going to be what like? What is the new social contract like? How do? My guess is that the things that we'll have to figure out are how we think about fairly Distributing wealth, access to AGI systems, which will be like kind of the commodity of the realm, and governance, like how we collectively decide what they can do, what they don't do, things like that, and, I think, figuring out the answer to those questions is, is gonna just be huge.
I I'm optimistic that people will figure out how to spend their time and be very fulfilled. I think people worry about that in a little bit of a silly way. I'm sure what people do will be very different, but we always solve this problem. But I do think, like the concept of wealth and access and governance, those are all going to change and how how we address those will will be huge.
Actually, one thing- I don't know what level of devs you can share that, but one of the things I love about what openai and you guys are doing is when you they think about these questions a lot themselves and they initiate some research.
So you've initiated some research on this stuff. Yeah, so we run the largest Ubi experiment in the world. I don't think that is. We have a year and a half, a year and a quarter left in a five-year project. I don't think that's like the only solution, but I think it's a great thing to to be doing and you know, I think like we should have like 10 more things like that that we try. We also try with different ways to get sort of input from a lot of the groups that we think will be most affected and see how we can do that early in the cycle. We've explored more recently, like how this technology can be used used for reskilling people that are going to be impacted early. We'll try to do a lot more stuff like that too. Yeah, so they are the. The organization is actually in fact these are great questions, addressing them and actually doing a bunch of interesting research on it. So next question: hi. So creativity came up today in several of the panels. You know, and it seems to me that the way it's being used, like you, you have tools for human creators to go and expand human creativity. So where do you think the line is between these tools to to allow a Creator to be more productive in artificial creativity itself? So I I think- and I think we're seeing this now- that tools for creatives, that that is going to be like the great application of AI in the short term. People love it. It's really helpful and I think it is, at least in what we're seeing so far, not replacing. It is mostly enhancing. It's replacing in some cases, but for the majority of like the kind of work that people in these fields want to be doing. It's enhancing and I think we'll see that Trend continue for a long time eventually. Yeah, it probably is just. Like you know, we look at 100 years. Okay, it can do the whole creative job. I think it's interesting that if you asked people 10 years ago about Holly, I was going to have an impact with a lot of confidence from almost most people.
You would have heard: you know, first it's going to come for the blue collar jobs- working in the factories, truck drivers, whatever. Then it will come for the kind of like the low skill White Collar jobs, then the very high skill, like really high IQ, white-collar jobs like a programmer or whatever, and then, very last of all, and maybe never, it's gonna take the creative jobs and it's really gone exactly and it's going exactly the other direction and I think this like isn't there's an interesting reminder in here generally about how hard predictions are, but more specifically about, you know, we're not always very aware, maybe even ourselves, of like what skills are hard and easy, like what uses most of our brain and what doesn't, or how like difficult bodies are to control or make or whatever.
We have one more question over here. Hey, thanks for being here. So you mentioned that you will be skeptical of any startup trying to train their own language model, and it would love to understand more. So what I have heard, and which might be wrong, is that large language models depend on data and compute and any startup can access to the same amount of data because it's just like internet data and compute, like different companies might have different compute, but I guess I see a big players can sell more compute. So how good a large language model startup differentiate from another?
How would the startup differentiate from another? How would one large language model startup differentiate? I think it'll be this middle layer. I think in some sense, the startups will train their own models, just not from the beginning. They will take, like you know, base models that are are like hugely trained, with a gigantic amount of compute and data, and then they will train on top of those to create, you know, the model for each vertical and and that those startups.
So in some sense they are training their own models, just not not from scratch, but they're doing the one percent of training that really matters for, for whatever this use case is going to be those startups, I think they will be hugely successful and very differentiated startups there, but that'll be about the kind of like data flywheel that the startup is able to do, the kind of like all of the pieces on top of and Below, like this could include prompt engineering for a while, or whatever. The sort of the kind of like core base model. I think that's just going to get too too complex and too expensive and the world also just doesn't make enough chips. So Sam has a work thing he needs to get to. So and, as you probably can tell, with a very far, far ranging thing, Sam always expands my boundaries and a little bit unlike the, that when you're feeling depressed, whether it's kids in a house, you're the person I always turn to. Probably I appreciate that. Yes, so anyway, I think I think, like no one knows, like we're sitting on this like precipice of AI and like people like it's either gonna be like really great or really terrible. You may as well like you gotta, you gotta. Like plan for the worst. You certainly like it's not a strategy to say it's all going to be okay, but you may as well like emotionally, feel like we're going to get to the Great future and we'll play as hard as you can to get there and play for it. Yes, rather than like act from this place of like, fear and despair all the time, because if we acted from a place of fear and paranoia, we would not be where we are today. So let's thank Sam for spending dinner with us. Thank you.