Impact Podcast | Can AI determine what and how we perceive value?

KANINA BLANCHARD: Exclusive insights, actionable strategies, and ideas that ignite change. You're listening to the Ivey Impact Podcast from Ivey Business School.

JULIAN BIRKINSHAW: Hello and welcome to Dialogue with the Dean, the inaugural series on the Ivey Impact Podcast. I'm Julian Birkinshaw, Dean of the Ivey Business School. And in this series, I sit down with Ivey's leading faculty to explore their latest research and tackle the pressing issues shaping business and society.

On today's episode, we're exploring a force reshaping industries at an unprecedented pace: Artificial Intelligence. Often viewed as a tool of efficiency, AI is proving to be much more – redefining the way businesses operate, innovate, and compete.

Imagine, for example, being able to predict consumer demand with precision or refine your pricing and marketing strategies based on AI driven insights. This is no longer just a possibility. It's happening right now.

And to unpack this, I'm joined by Fredrik Ødegaard, associate professor of management science at Ivey. His latest research, co-authored with Joshua Foster, introduces a groundbreaking, AI driven approach to demand estimation that could redefine how businesses forecast and optimize sales. Fredrik, welcome to Dialogue with the Dean. It's great to have you here.

FREDRIK ØDEGAARD: Oh, thank you.

And thank you for nailing the last name pronunciation.

JULIAN BIRKINSHAW: You're most welcome. I spent a couple of years in Sweden, so I guess that helps a little bit.

You've had a long career, both in academia, but also a little bit in business, just give a sense of your background and how you ended up here working on artificial intelligence.

FREDRIK ØDEGAARD: I've always been mathematically inclined. And my area of management science is mathematical modeling of business problems or could also be technological problems. And so, there is a natural connection between AI and mathematical modeling because in the end, that’s just basically what AI is – it’s mathematical modeling of various scenarios.

JULIAN BIRKINSHAW: And you also worked a little bit as a business practitioner actually working in supply chain, was it, or operations?

FREDRIK ØDEGAARD:  Correct. So, I did my master's degrees in operational research and in statistics. And then I was working for a supply chain software planning tool called i2 Technologies, at the time. We were providing software for decision making of the largest firms, like Nokia and Siemens and Volvo and so on.

And then from there, I worked for a think tank slash consulting firm slash research place called Institute of Applied Economics. And that's when I then decided I want to pursue a more research-based career. And that's how I entered my PhD studies.

JULIAN BIRKINSHAW: So, before we get into your research, AI is everywhere. I mean, you can't pick up a newspaper today without reading about the latest innovation, and we perhaps will pick up on some of those specific recent things at the end, but why is AI taken off over the last five years or so?

FREDRIK ØDEGAARD:  So, one aspect of this is, of course, the marketing of the latest buzzwords. We've gone in a change. ‘AI’ was a buzzword, I'm going to say maybe 20 some years ago. And then we went into the buzzword ‘big data.’ And then there were buzzwords around ‘machine learning’ and all these types of learnings and then analytics.

And now there's like a renaissance on the term ‘AI.’  So it's not really about, oh, all of a sudden that has it's always been around. And in fact, AI as a concept that's been around for decades and decades, like 50 year plus years. So, what has happened in the most recent years, with the boom of gen AI, is that we used to have technology tools, such as, thermostats or self-driving cars and so on. But now we also have AI also on things like the searches and the pricing, and these online tools such as ChatGPT and Copilot and so on. And so that has had a boom, I think.

JULIAN BIRKINSHAW: Right. And this gen AI, obviously generative AI means it's actually giving you literal text. I mean, we're all experiencing this. Which, whereas before it was a little bit behind the scenes, was perhaps working in the background. But I guess it's the visibility around gen AI, which means it's actually become part of the province.

FREDRIK ØDEGAARD:  Exactly. I mean, I don't think this has happened over the last five years, but over the last maybe 20 years is, of course, the computing power has increased enormously. And the data that's being collected has just mushroomed like crazy. And so, you need a lot of data and you need a lot of computing power in order for these tools to work.

So, these large language models, that is the engine of Gen AI, which is based on these neural networks, those ideas have existed since the 70s. When I was in grad school in the 90s, we used neural networks to solve small little toy problems because the computing power didn’t have much data there, and they were completely useless.

And I must say, it's amazing that the people who were pushing that research frontier had such belief in the potential for it. Because what they were doing at the time wasn't that amazing, so you could only solve small little toy problems at the time. But it's quite impressive that they had the tenacity to think “no, no, this has the potential.” And then, now, as soon as we have the technology and the data, we’re getting these amazing applications.

JULIAN BIRKINSHAW: Right. So, there's been computer scientists working on AI for, as you say, for decades. Suddenly the amount of processing power and the amount of data available to train these models allows for this sort of inflection point, if you like, whereby we suddenly see the exponential increase in the quality of what can be done.

FREDRIK ØDEGAARD:  Exactly.

JULIAN BIRKINSHAW: And of course it's not going to stop. I mean, everyone tells me that the AI we're working with today is going to look quite primitive five years from now.

*MUSIC TRANSITION* 

JULIAN BIRKINSHAW: We're going to dive into a paper that you've written, called Giving Deep Attention to Consumer Preferences with Large Language Models. That's the one with Joshua Foster, our colleague. You're going to have to just take us through this in a really basic way. So, what is the business problem you're trying to solve? And what does the AI then help us to do, which we couldn't do before?

FREDRIK ØDEGAARD:  So, what we're trying to figure out is: from a description, from a pure text description, can you infer what value that customers or consumers have associated with that? So, in economics we have something called hedonic pricing. So, if you have a product, like a house or a car or a piece of software that's built up of different components, then you can provide some attributes on all those components of what is it worth to the consumer. So, when I bundle all this stuff together, I create a car or create a house or create a piece of software together that gives me this value. And historically, when we do hedonic pricing, you have to pre specify the different components but give a specific value if it builds up. So, if we take a car, if you tell me the make, the model, the miles, the year, and the type of transmission – maybe I'll ask for five attributes – then I can say, here's what one is worth in the consumer market.

So, that's great. If you're a car expert, then you might be able to define those five components. But what we wanted to see as well was: what if you just provide the description? How do consumers actually react to a pure text description on the internet? So, if you go to buy a car and you test drive a car, perhaps all of those attributes somehow manifest themselves in: oh yeah, this car feels nice, I like the way this door locks, I like the shift, I like the seat, and so on…. You can experience that. But more and more we are buying things online and have to rely on the text, and that text generates value. And so, what the large language models now can do is go: okay, I'm just going to analyze the text. I'm not going to pre specify what's really important, or the make of a model, and the transmission type. And then I'm just going to train to see if I give you some transactional data, we used to auction data, can you figure out what a car is worth? That is what these machines can do.

JULIAN BIRKINSHAW: So, I think I understand but let me just check. So, I might think that a BMW car has a particular kind of intrinsic value that’s greater than a Ford. And, I might think that having some sort of turbocharger makes it more valuable, and a sunroof makes it more valuable. And that's my intuition. But you're saying that may or may not actually be correct. And if we're trying to get to a more accurate pricing, AI can help us to essentially what, go through thousands, tens of thousands of examples of complex, multifaceted products, look at the prices that were actually charged, and help you to almost link the attributes of the car to the features and prices.

FREDRIK ØDEGAARD:  Yeah. And not only just the attributes, but the way it was described. So, you might have two identical BMWs up for sale, and one just has a very sparse or technical description, such as: BMW 2.3, and so on. And then the other one has some sort of nice prose and explaining, “when you close the door, it goes so smoothly” and that might generate value. And so, you can see how the text description actually influences the consumer value. And that's what these large language models can actually terse out.

JULIAN BIRKINSHAW: In terms of practical applications, sticking with a specific example of car auctions. So, you are a car auction house, and you might want to buy and to get hold of Fredrik and Joshua's research because it essentially helps you to more accurately predict what someone's going to pay. Is it as simple as that?

FREDRIK ØDEGAARD:  Oh, well, that would be one application. So, one application might be if you, as an auction house, are the one who actually takes in the vehicles and then you type up the descriptions and so on.

The one we have access to, a platform where we train our data from, is a public platform similar to eBay. It's called bringyourtrailer.com. And so, if you want to sell your car, you load up some pictures and you create your description. And so, what this research can can show you is suppose I'm thinking about making a description. I can test out that description, feed it into our model, and get the estimation of what you can expect to get for your car. Then, you can say, “okay, what if I write it like this?” And then see how it can actually shift what the demand will look like.

JULIAN BIRKINSHAW: Wow. And how widely can approach be used? I mean, there must be applications in many other industries as well. Perhaps you've done research on this, or perhaps you're just speculating. Give us a sense of where else we can use this.

FREDRIK ØDEGAARD:  Both me and Joshua feel like the real estate market would be another aspect of it.

I think, from my background in revenue management, the airline industry, and the hotel industry, and car rental industry – places where you have different components to it. So, it's not homogeneous, like a loaf of bread, for instance. You need something that builds up, but there's different attributes to it, and that the description can help.

Now, of course, you can say, “oh, this sesame seed, poppy seed bagel, with special yeast…” Maybe. But that's not the main case,

JULIAN BIRKINSHAW: But, frankly, cars, holidays, houses. I mean, these are huge markets in themselves. So, there's plenty of ways we can use it.

So, we're ultimately trying to help businesspeople make better choices. So, how can we think through the consequences of this? Because, yes, we want to understand the pricing better, but if you're running a real estate agency, how does this help me to run my agency better?

FREDRIK ØDEGAARD:  So, there's two components. One is just to understand what it is that actually drives the demand of a value of a product or service that I am providing. So, can I actually terse that out and understand. Then these large language models, the ones that we have provided, can then pick out the specific contribution for these words in a particular sequence. So that's one thing, in that how does the value of a product or service get derived.  

And then on top of that, suppose I would like to introduce a new product or service. Can I get some sense for what it is going to generate? Well, if I explained it like this and I feed this into the market right now then I have an idea.

JULIAN BIRKINSHAW: So, both the people buying and selling cars benefit – and the houses benefit. But, as you say, you go back to BMW itself, in terms of which of their features do users value. And so, perhaps, they don't value the sunroof or the supercharger, but they do value the beautiful styling. I mean I'm just making that up. But the point is that this is giving insight into consumer preferences, therefore that feeds through into essentially designing products to be more effective and more valuable in the eyes of the consumers. Wonderful.

*MUSIC TRANSITION* 

JULIAN BIRKINSHAW: So, let's move from the specific work that you've done to a slightly broader conversation, because we started by acknowledging that the world of generative AI is with ChatGPT and so forth is upon us. But, what you just described to me was what I would always call predictive AI. We're trying to predict prices better. And predictive AI is not exactly the same as generative AI. For a student at an Ivey, how would you describe the difference between these two?

FREDRIK ØDEGAARD:  So, that's exactly right. I think most people, when we associate the word or the term AI, they do think, in the modern era about predictive analytics or forecasting models and so on. But then from a management science perspective, my background, that's just one component of analytics. The bigger one is what we call optimization, sometimes called prescriptive analytics.

For strategic analytics, what do I now do given that I have is forecast? So, there's a whole world beyond just forecasting and predicting. I mean, with generative AI the new component there is that you're using these forecasting tools not just to forecast predefined variables and numbers, but actually to forecast what word fits best here.

So I think that's an amazing sort of creativity to think that I'm going to use these tools where I was able to predict a specific variable that's been predefined to actually predict what should the best response be to this query. And how can I articulate some sort of a sonnet or a description of something that's not numerical.

JULIAN BIRKINSHAW: Indeed. And I think most of us are absolutely astounded by essentially the creativity of generative AI – its ability to come up with things that we would not have thought of.

I personally use it when I'm brainstorming and trying to figure out what are all the different ways of thinking about something. You know, I can think three or four things immediately,  and I ask ChatGPT to give me another five. And of course, it's not being genuinely creative because it is, by definition, trained on a universe of data that's out there. But it's being creative, meaning that it's helping us to think of things that we as individuals wouldn’t.

FREDRIK ØDEGAARD:  Sure.  That's one way. And, I think the creative aspect is the following. So if you pose a query or a question and then it has the response. The combination of words or a combination of responses to this particular one, I can draw from all these different buckets and then it’s not just going to choose the one that's the highest or on average as the best. It’s going to randomize; it’s going to roll this dice. So, I have these ten buckets, I roll this ten-sided dice, and then I happen to pick that response. So, the creativity is that I'm sort of randomly generating responses based on that.

And that's where we then come into this whole thing about hallucination. Because sometimes you get a response, where you think ‘What do you mean? It doesn't make any sense to me.’ Well, it just happened to roll a one and it took from bucket one and bucket one didn't really fit in the context that you were making it, but it fit it in another context.

JULIAN BIRKINSHAW: And that has always intrigued me. You can ask the same question, exactly the same question, to the same ChatGPT, and it will not give you the same answer. It will give you roughly the same answer, but it's going to use different words. Which, by the way, is kind of fascinating because it's one of the reasons why we as faculty at a business school get nervous because we can't prove when our students are using AI.

FREDRIK ØDEGAARD:  Yes. I mean, sometimes there's a smoking gun. Because you do know your students. And so, when I read the students report, I'm like, ‘…this is not Tom, I can tell you that. I don't know who it is, but it's not Tom. He doesn't have that sort of style.’

JULIAN BIRKINSHAW: I think you're spot on. You can just tell that sort of AI style and yet proving definitively that they've done that is impossible.

So, let's just go a little bit deeper into the implications of AI for teaching and learning because, as faculty in a business school, we're training the next generation of students in what's important to them as businesspeople.

We know that they're using ChatGPT and Anthropic and other such models, and I'm personally comfortable with that. I think it's impossible to put the genie back in the bottle once these technologies exist. It's like having a pocket calculator - they will be used. So, the question then becomes: given that the students are going to be using them, what is it that we're actually teaching them?

What is it that an Ivey graduate coming out of here in a couple of years’ time has that we can absolutely, definitively say we've helped them to be more successful as leaders of the future. What are what are we teaching them now?

FREDRIK ØDEGAARD:  I think that's exactly right. I think we really need to reassess the core skills that we want for them. And I think when we think about that, it's stuff like critical thinking, it's articulation, it’s being able to frame a strategy around something. And so, I think we have to go back. I think with AI, it's going to go back more to basics. So, understanding fundamental marketing topics, fundamental finance, fundamental accounting, fundamental operations, fundamental analytics, and so on. And AI is just a tool to do that.

And so, I think we have to then think about what is it business students should know now. Roughly at our school, at the undergraduate space, we have one quarter going into accounting, one quarter going into consulting, one quarter going into finance, and then one quarter going into sort of general things.

And so, in those streams, there are then some specific, tangible skills that the companies that come to recruit our students are looking for. So, they should understand how to read a balance sheet and how to understand a financial statement and so on.

And so with those skills, we then have to figure out what we want to do – and how can AI, as a tool, can help us deliver that.

JULIAN BIRKINSHAW: Yes. I've discussed this with many people, not just yourself, but, we always come back to this question of: what are those sort of uniquely human things that AI can't do? Certainly not today that employers value. And I think you touched on a couple – we talked a little bit about genuine creativity, genuine out of the box thinking, we talked a little bit about critical thinking, and of judgment.

For me, why do people hire Ivey graduates? They hire them because they've got that capacity to make decisions under a great deal of uncertainty.

FREDRIK ØDEGAARD:  That's right. And the environment we put them in. With the case-based method in the classroom, you have to think on your feet and respond quickly. You have to articulate your thoughts, your arguments, and so on. Both skills are very impressive – just making a presentation, like when we have case competitions and so on, our students are really impressive.

JULIAN BIRKINSHAW: Yes. And you touched on case method. Absolutely, the case method in some ways is even more important now in an era of AI because lecturing to students about concepts is something that they can get online for free. But, having a dialogue in the classroom builds critical thinking skills, it builds their relational skills, the ability to make an argument, the ability to challenge. These are what employers want.

*MUSIC TRANSITION* 

JULIAN BIRKINSHAW: So, there's much more we could discuss. We haven't even talked about issues of privacy, issues of computers taking over the world. I mean, that in my view, is the world of science fiction. But it's a conversation in its own right. But let's summarize and wrap up.

So, if listeners wanted to take just one key insight from the conversation, particularly about your own research, what should it be?

FREDRIK ØDEGAARD:  What we’re trying to do is just stimulate that creativity on what are the applications of AI and to think outside the box. I can ask AI to produce a song in the style of Bob Dylan on this topic, or I can ask it to give me a recipe on something. But, come up with new and creative business applications.

I believe we are one of the first to actually do it on how AI can generate the consumer demand, underlying consumer demand, for products and services. And so, think outside the box of what it is that AI, and these gen AI tools, can do with text.  And so that would be the sort of key takeaway to try to think outside the box.

JULIAN BIRKINSHAW: We've got these beautiful new models. Most of us don't understand what's actually happening. And it's up to us to be creative about coming up with new applications.

FREDRIK ØDEGAARD:  That's right. It’s the same way when electricity first came. The first application was generating light. And then, all of a sudden, we had ironing and dishwashers and microwaves and all that stuff. It’s quite amazing.

JULIAN BIRKINSHAW: Yes, and it does fascinate me that at the moment we're using AI to do better things that we've already been doing. What we now need to do is figure out ways of doing things completely differently than we would have imagined even possible.

JULIAN BIRKINSHAW: You've been listening to Dialogue with the Dean from Ivey Business School. A big thank you to my guest, Fredrik Ødegaard, Associate Professor of Management Science, for sharing his time and fresh research with us today.

And of course, thank you for tuning in.

On our next episode, I'll be joined by Hubert Pun, Professor of Management Science, to discuss another digital force shaping the future of global business: Blockchain. Will explore its potential, its role in modern industries, and why Canada must take the lead in this evolving space. Don't miss it. Until next time, goodbye.

KANINA BLANCHARD: This was Dialogue with the Dean, an Ivey Impact Podcast series. For more insights from Ivey, including thought leadership on critical issues and additional podcast episodes, visit IveyImpact.ca or subscribe on your preferred podcast platform.