Marketing Mix Crystal Ball: Inside Facebook & Ocean Spray’s Analytic Methodologies

With today’s channel and media fragmentation, consumer goods brands can’t rely on gut instincts to get their marketing mixes right. And as these channels grow (and budgets shrink), properly leveraging the power of analytics is what’s separating the winners from the losers. A recent webinar dove into how Ocean Spray is integrating marketing mix models into its day-to-day operations — using analytics to drive business and really maximize ROI. Also discussed: How Facebook’s data science team determines which elements of a campaign are working, which aren’t, and how they can be improved quickly and so the insights are actionable. Read on for the webinar transcript and get an education in the future of marketing effectiveness. Alarice Rajagopal: Good morning, everyone, My name is Alarice Rajagopal and as senior editor of CGT, I'll be your moderator for today. The title of the webinar is “The Art of Becoming a Marketing Mixologist.” However, we're not here to talk about cocktails this early. Instead, measuring market spend is the need of the hour, especially as consumer goods organizations continue to identify this as one of their biggest challenges. To help discuss this further, I'm delighted to introduce our subject matter experts on this topic. Our first panelist for today will be Igor Skokan. Igor is part of the marketing science team at Facebook based in London helping agencies and marketers find true business value through a range of solutions designed to measure audience brand and sales outcomes. Nuclear scientist and mathematician by education, prior to joining Facebook, Igor held various senior analytical positions within data, tech, and analytics units in Dubai, London, Warsaw, and Prague. Across his roles before and now, his focus is on the intersection of different facets of verification and effectiveness from contemporary marketing mixed modeling, attribution, and experimentation to media insights. Beyond his passion for numbers, he likes traveling, trail running, and climbing mountains. Joining Igor today will be Yash Sikand, VP global insights, analytics, and planning for Ocean Spray Cranberries. Yash is a 25-year veteran of the CPG industry focused on analytics, insights, and strategy. His passion is to help companies create and execute strategies rooted in consumer insights while driving efficiencies through the use of analytics and processes. He began his career conducting advanced analytics for CPG companies at AC Nielsen, after which he spent more than 20 years working in a variety of leadership roles within Philip Morris, Kraft, Wrigley, MillerCoors, and Ocean Spray. Our third panelist will be Bob Gannon, who currently leads business development for FusionPoint. In a career spanning more than three decades, Bob has wide-ranging domestic and global roles leading marketing and sales strategy, marketing and consumer insights, and data management, data analytics functions on both the client-side and vendor/partner-side. Bob has held executive positions with iconic global marketing research firms including Nielsen and The NPD Group. And he has held senior marketing research and analytics roles with MillerCoors, Darden Restaurants, and Ruby Tuesday Restaurants. And last, but certainly not least, to round out our panel is Garth Viegas, general manager Americas for Analytic Edge. Garth has more than 20 years global analytics, insights, and strategy experience working for Kraft Foods, Mastercard, SABMiller, and TATA Sons. He has helped CEOs and senior executives leverage analytics to contextualize and navigate evolving global business environments to solve challenges and deliver results Garth is passionate in helping drive sales, profit, and brand equity through marketing, mixed modeling, pricing and promotion, media optimization, revenue management, loyalty analytics, and more across the consumer financial and retail verticals in both mature and emerging markets. So, as you can see, we have a very diverse panel of experts to make for a very lively discussion today. So, thank you, Igor, Yash, Bob, and Garth for joining, and I can't wait to dive right in. So, first, let's start from the beginning with the title. Now, Garth, if I could start with you, I'm intrigued by this market mixologist correlation, and although I'm sure we could talk about cocktails, I want to start out by asking why this title? Garth Viegas: Thank you, Alarice, and thank you to my fellow panelists, Yash, Igor, and Bob for being here today. Now, to answer your question, successful brands are built when you mix the right ingredients together to build brand equity, generate revenue, and win market share much like your favorite cocktail. I think that's something we can all agree on. But it's the word mix that intrigued me, and over the last few months, I've either been looking at LinkedIn profiles or job description of marketing directors, and not many marketers talk about getting the mix right. Their skill sets are often siloed by digital promotion, media, so on and so forth, rather than getting the overall mix right. And hence the word mixologist intrigued me. Now, I must be completely honest, I'm not the first to stumble across this concept. Professor James Culliton of the Harvard Business School wrote about this very concept 70 years ago, and he identified one of the key requirements of the marketer is to be a mixer of ingredients. I think in today's world, that concept has even become more important. If you go back 20, 30 years ago, there were very few ingredients to mix to create a successful marketing plan. Today, with channel fragmentation, with media fragmentation, increased 10-, 20-, 30-fold, we can no longer rely on our gut instinct to get the mix right, and therefore we need the power of analytics to get the mix right. Hence the word marketing mixologist. Rajagopal: Thank you, Garth. I especially like the analogy around the number of ingredients increasing because I can't even keep up with just the platforms my kids are using, let alone trying to measure it. So, on that topic, I next want to bring in measurement as a concept. The need to measure is becoming more important especially given the channel and media fragmentation that Garth mentioned. So, Yash, if I could go to you for this next one, what is your measurement agenda? And is this agenda expanding or increasing in importance? Yash Sikand: Thanks. I think the measurement agenda has always been important. I don't know whether it's increasing more. I think there's more of an awareness of it as budgets get tighter and timing is more and essential. With the new media channels available to us that you can change multiple times a day or multiple times an hour, it just becomes much more important to be able to read what you're doing. But I do want to maybe elevate the discussion a little bit is, philosophically, I think what we're doing when we do marketing mix and we read it, is we're reading tactics. But what's very important is strategy, and let's not lose sight of what is the communication strategy because that's what really drives the tactics of the marketing mix. What I would say is at Ocean Spray, we are trying to read more channels faster and cheaper. And none of these should be actually new to anybody, but one of the ways you tend to do that is how do you bring it in-house? How do you read it multiple times? A lot of in the old days, I'm dating myself here 20 years, we would do a mixed model once a year with an external vendor. We'd go into gory details. We'd get lost in the methodology. Candidly, all that is not as important. Now, it is important that you want the best read you can get, but it's really how do you use it for the business? Can you provide the ROIs and the insight at the time the decisions are being made? not six months, a year looking in your rear-view mirror saying, "Ooh, that was a bad decision. How do we get better?" So, to answer your question, the measurement agenda is very important and has to be more real-time. Rajagopal: Right. Absolutely. Thank you, Yash. Igor, can I bring you in here and ask you the same from your perspective at Facebook. What is your measurement agenda like? And is it expanding or increasing in importance? Igor Skokan: So, I'm part of the Facebook marketing science team. This is our mission actually. How can we empower businesses and all marketers to grow using data and science? So, how do we use the data and the science that we have on our side, and how do we achieve this goal? For us, it's really about we are one of the publishers on the plan. We are one of the way where marketers are investing for their growth, but the way how we are delivering on the learning agenda is to empower ecosystem and have a strong ecosystem of trusted third-party partners like Analytic Edge, like many others. But also, increasingly, and to Yash's point, in-house measurement team and in-house data science team. So, it's a little bit shifting from "help me run it" to "help me build it." So, we see a change there on the learning agenda implementation. For the learning agendas themselves, and specifically, the North Star remains for us the measuring the true incremental business value. There are a couple of interesting points in this. First one is the true business value. We believe there are some things that are easy to measure like clicks or likes, etc., but these are sort of proxy metrics that we don't actually think they should. And in digital, a lot of it sometimes get muddled. Oh, it's easy to measure website visits, but do they actually matter? And the incremental piece, right? So, incrementality is an important concept of causality. And this is very difficult. This is very difficult to do. So, from our perspective as well, it's like measurement needs to uncover what is working, which elements of the campaigns are working, what's not, and how can it be improved? And again, to Yash's point, how can we have it more actionable, faster, quicker, near real-time sensitive? And in that sense, we're generally seeing two dimensions in learning agendas. So, one is being in channel, and this could be in Facebook or any other channel. And these are mostly done by testing control experiments. But a lot of it is down to experimentation. How can we maximize the effectiveness of Facebook? How do we make the dollars work harder? What do we need to do? And this is done through rigorous test-and-learn strategies within the channel. What audiences? What frequencies? So, there's a lot of levers that today's marketers have, and how can this be best utilized within channel? But then equally, and maybe even more important is the cross-channel piece. How does it all work together across channels? And this is where attribution and then marketing mix, this is where these methodologies become more important. But we see the magic of learning agendas also happening where we overlay or dovetail these things onto each other, and we see MMM plus experimental attribution, plus experiments calibrated over each other. So, I would say it's increasing. Going back to your question, it is increasing. It's increasing in complexity, it's increasing in speed, but it's also increasing in the ecosystem of partners and the tests and methodologies that we have available as marketers, as analysts, and marketing scientists. Rajagopal: Right. That's interesting perspective. If I can, Bob, can I pose the same question to you as well? And in your experience in working with various clients, what are you seeing with regards to the measurement agenda? Bob Gannon: I concur with my counterparts here. My company FusionPoint is largely a data management company and we've seen a tremendous, but entirely necessary increase in the number of variables that our clients are putting into their marketing, mixed modeling, and other analytics. They've got a strong desire to measure each component. Very important for them to understand their return on investment and all of those fragmented investments. They've got digital investments in social media, search, etc., as well as the traditional. So, they need to know what's paying back and what isn't. We specialize actually in projects that involve large or numerous data sources that require very complicated mapping structures. Complicated hierarchy management. So, what we do is ingest data from our clients, from their partners, and other third parties, and we spend considerable time cleaning that data, harmonizing and integrating it in order to feed our client's own or third-party analytic tools. And visualization tools such as Analytic Edge's Demand Drivers because ultimately, we're the foundation for helping our clients and the clients of companies like Facebook and Analytic Edge help their clients understand the ROI for each of those investments they're making. So, yes, absolutely concur with our counterparts here. Rajagopal: Yeah. It's so important, but sometimes much easier said than done. Gannon: Yes. Rajagopal: Okay. Thank you, Bob. So, Garth, can I bring you in here? Any other thoughts on the measurement agenda and its importance? Viegas: Yes. Thank you, Alarice, and thank you, Bob, Yash, and Igor for that. I have a slightly different point of view. I think for me, a measurement agenda is about developing a shared understanding of the business. Let me give you an example, 10 years ago, I worked for a beverage business. When I visited this small market in Europe, everyone was celebrating because sales had gone through the roof. I walked to the marketing director's office and he said, "We've got this new campaign, and all the sales are due to this new campaign." An hour later, I bumped into the sales director, and he said, "No, no, no. We just got this great new distribution, and we've run this fantastic promotion." Interestingly, over lunch, I was sitting with the finance director and I repeated what I heard from the sales and marketing director. And he laughed, and he said, "No, Garth, it was the hottest weekend of the month, and guess what? Our competition was out of stock." And that meant, I think all of them were right. I think they were all too right. Sales was right, market right, finance was right. They all contributed to sales in their own particular way, but there was no shared understanding the impact of each. Fundamentally, for me, I think the measurement agenda is about a shared learning across the organization that you can actually use to empower the organization and go forward. So, I think we should look at the word shared learning and supplement that with a mission and agenda. Rajagopal: Right. Thank you, guys, for your insights on the management approaches, but Garth, I'd like to stay with you here since you brought it up. How would an organization go about building shared learning? Viegas: Yeah, you're absolutely right. Shared learning can be a challenge for the business today. I alluded to this earlier on, this fragmentation of media and channels. There's a lot of data, and consequently, that's actually led to more confusion upon telling the number of plans we're people are looking at the wrong measures. So, how do you deal with the situation where things are getting more and more complicated going forward? Basically, in my experience, I believe a market mix model can be an effective way to build shared learnings. The reason I bring this up because the market mix model can actually decompose your sales or any other KPI that you might have into its various drivers. But there are challenges. Market mix modeling has traditionally been very expensive and slow to execute. I think Yash articulated that earlier on. He said it's once a month and it's very backward-looking, and so on, and so forth. So, I think, to Yash's point, I think we need to repurpose and reshape this particular science as we go forward. I see us reshaping it in three or four different ways. The first one, and I think Yash concurs with, is it should be always on. Today marketing has evolved. We don't have large campaigns, but it's a continuing drip feed of social media. So, how do we constantly measure ourselves? We got to reshape it to be always on and measure this all the time. Secondly, interestingly I was looking at a stat, and I read somewhere that 70% of all the marketing activity in North America is not measured. At 70% that's a lot. And therefore, what we need to do is democratize tools like market mix modeling and make them scalable across the business. To democratize something, we got to make it cheaper, faster, better. No two ways about it. The last thing, I think successful clients, I think as Igor has mentioned, start with do it for me, then do it with me, and then let me do it myself. I think once companies take this approach, they can still maintain that high level of rigor when it comes to analytics at a far cheaper cost, enabling them to spend. Rather expand this technique across more brands. Rajagopal: Absolutely. Sometimes I think getting started is kind of the biggest hurdle, but at least they can start somewhere, right? Viegas: Absolutely. Rajagopal: Yeah. Thank you, Garth. Yash, if I could ask you the same from your point of view with a consumer goods brand, how do you go about building that shared learning? Sikand: The way I'd probably do it with my career is, like I said, if you're looking at a mixed model and you're looking at it once or twice a year and having a big presentation to the marketers and sales guys and saying, "Here are your ROIs. Here's what's working." To me, that's just scratching the surface. The way you do it is to integrate the marketing mix models into the day-to-day operations of the company or the brand. The way we try to do it is really drive what I would say, take your marketing mix coefficients and use them to help forecast the model. Everybody who runs a business has to deliver a number. Everybody's looking for as much help and guidance as they can to increase their odds to deliver or exceed their number. And so, if on a monthly basis you're looking at your business and you're doing an S&OP process or you're doing forecasting, how do you use those coefficients to help you fine-tune your forecast, refine it, make changes that are based on actual quantification? Not, hey, the sales guy says, "I'm going to run a bigger promotion," or the marketer says, "I've got an extra million bucks and I think it's going to drive X, Y, and Z or whatever it is." So, I do think trying to take the mix models, the forecast models, bring them in-house, having the data integrated in some sort of data lake that the insights or the analytics team can use to provide guidance. And another thing I would just say is what I strive to do that's probably useful is I'm looking for directional accuracy and precision. All I'm trying to do is provide more directional accuracy and say, "Yes, if you do more of this, you're going to get more volume. If you move dollars from X to Y, you're going to move top line or bottom line." And really, that's how you get started in the organization to actually use the analytics to drive business, and not just have it sitting on the side either as some sort of a red, yellow stoplight. So, in my mind, that's helpful, and I think the organization appreciates it because you're actually helping them get better. Rajagopal: Right. Thank you, Yash. Bob, can I bring you in here for the same question? From your perspective, how do you build that shared learning or how do you see your clients, I guess, tackling this task? Gannon: Yeah, I agree with what both Garth and Yash have said one other point that's maybe a bit of a tangent that I'd like to add to this part of the conversation is about the amount of time it takes to prepare data for the modeling process. And it's grown tremendously over the course of the last eight to 10 years along with the number of data inputs. A typical analyst is now spending 80%-plus of their time just preparing data for the modeling exercise, and this takes considerable time away from higher value-added activities for the business. Whether those are related to the modeling process or other needs that the business has. With the technology breakthroughs that are out there today though, such as Analytic Edge's Demand Drivers platform, which include strong data management functionality, much of this effort can be streamlined, particularly after the first wave of models are produced and there's an efficient effective mechanic for getting data from ingestion to input into the model. So, again, reinforcing what they've said, but just also taking into account that there's a lot of efficiency that can be gained from using the new technologies. Rajagopal: Thank you, Bob. So, Igor, as you heard, Yash, Garth, and Bob all addressed and explained the benefits of MMM to build the shared understanding of the business. But from your point of view, is this approach valid for the digital world? Skokan: So, MMM is being very much rediscovered or discovered by the digital world. Obviously, the methodology itself predates the internet, right? It's fascinating, but indeed, MMM was here before the internet even existed, right? So, I sometimes just say that every generation discovers rock and roll, and we're now seeing a lot of the data scientists and digital organization, and digital natives. But when within the bigger CPGs, inside their digital units, sort of rediscovering the methodology. Now, as we all know, the underlying methodology of MMM is extremely robust and can accommodate. And I think Bob or someone mentioned, we're just scratching the surface of what this can do. But it has some challenges for the digital world, and it is obviously powerful and signal resilient, but it's still seen as slow and cumbersome, and it requires a lot of work. Another challenge we hear from the digital side of the world is the analyst bias. So, what I mean by that, given the same data, two analysts could come up with a different model. So, what's the ground truth? How do we compare models and how do we actually look into that? As Yash was mentioning, is it infrequent? It's not actionable. We finish mix model, and we don't know what we would do differently. So, it's expensive, and for time and resources, and resource-intensive. So, our vision as Facebook team for this is how can we have MMM as an AI or machine learning supported, very granular, automated, experiment-calibrated analysis that can deliver insights faster and on continuous basis? So, we’re supporting this vision by obviously being data provider into these systems. Making sure that data underpins everything so any data that comes out of Facebook is aggregated, obviously, but it's accurate. Its granularity enables precision and sensitivity. We're working on methodologies, and thought leadership with regards to higher cadence models or more actionable models or how can we achieve more validation, automation, and scale deployment? So, we're trying to do our bit from the digital and try to lead the digital platforms when it comes to data, methodology and thought leadership for companies to succeed in doing it more often, faster, and sensitive. Rajagopal: Right. Thank you, Igor. I also like the rock and roll. It's a good visual, rediscovering rock and roll. Okay. So, Bob, if I could bring you back in here, what are you seeing with your clients with regards to digital? Gannon: Yeah, our clients are absolutely looking at digital as an important component of what they're trying to measure. They're benefiting from adding digital sources into their marketing mix modeling efforts. They're learning what works, what doesn't, what combinations of things work, what don't. How to tweak this and that. It's helping them, again, better understand their investments, and critically important, helping them grow, achieve, and maintain leadership in their category. So, a critical piece of what they're trying to do, and I think, to some of the points that Igor and Yash and Garth have made, it's not always easy to get those learnings. Marketing mix modeling has evolved quite a bit, and now we're pushing it into the future is what we're all trying to do today. Sikand: When I look at Ocean Spray's spending and advertising, seven, eight years ago, we were 80-90% traditional. Now, we're probably 60% digital, right? Really made a rapid shift. We still use traditional. We need to get reach and frequency, and it's still very efficient, but that doesn't preclude us from trying to get an ROI in digital, and of course, in digital, it's a whole family of different channels and you can go after it, and Igor is specialized in one. But I think we have to figure it out, and I do think we as kind of an analytics industry need to figure out where we spend the time and effort because we can go really deep on something that really doesn't provide a lot of value to the organization. That's what I'm fighting most of the time because a marketer will come and say, "I got to read this. I got to read that." And I'm like, "You know what? In the grand scheme of things, it's not that important. Let's focus on the top six, seven areas where we're spending a lot of money on digital, and the rest of it, you know what? We can figure out later." So, that's one way of focusing the mix models, focusing the money spent because you do need data streams and getting data streams and trying to link those to volumes. In the end, what I keep falling back to is what we're trying to do is measure consumer behavior because consumer behavior translates in my mind into a purchase. And yes, you can track ad awareness, you can track awareness, you can track purchase intent. You can do all those what I call soft measures. Rajagopal: And I think some of the things you were talking about, Yash, are kind of across the board. We've been seeing that quite a bit in our coverage of the industry too, especially with that digital switch now. It gets so much harder... Way more complicated to measure and everyone's just jumping into digital because you have to, right? So, yeah, I would agree with what you are saying. Before we cut over to the Q&A, I do want to bring you all back in for one more question. So, Igor, if I could start with you, if you can look into your crystal ball, what excites you about the future of marketing effectiveness measurement and the part that MMM will play? And how do you see it having to change to keep up or do you see it having to change to keep up? Skokan: What really excites me is actually that a lot of this can be automated, but the interplay between the human and the machine, and the analyst and the machine. I think what I find really interesting is what computers and machine learning find easy, patterns and so on, humans find very difficult, and vice versa. So, how people can make the connections and predictions that no machine could possibly uncover. So, I think this is one where I am super excited about. Analysts who know their client business challenges, what questions to ask, what problems need to be solved, how to prioritize. I think, empathizing and relating with the audiences and the stakeholders inside that organization. Building that shared learning, building the actionability because the modeling only gets us somewhere. There needs to be action, it needs to be implemented, it needs to be brought to life. What also excites me is the constant changes in the ecosystem, and these are driven by meeting people's growing expectations around privacy such as GDPR in Europe, CCPA in California, LGPD in Brazil. Companies are modifying their policies to put more control in the hands of people, and these have impact on longstanding advertising mechanisms, but also requiring businesses to think differently about how they reach people and in general. But what excites me about this is that good measurement will remain to be about understanding true business value of your marketing. It will remain about being privacy-conscious and marrying these two principles and coming up with ways to do this. And we, just as the rest of advertising industry, are still exactly learning how measurement and marketing itself will ultimately evolve to meet people's expectations around privacy. So, I think those sort of two things are really exciting from my side. Rajagopal: Bob, what excites you about the future of marketing effectiveness measurement? Gannon: It's similar to what Igor was saying. What excites me is thinking about all the different ways we as consumers are interacting with media on a daily basis or media's interacting with us. From our phones to our computers or televisions, to billboards. The way companies are trying to influence us to engage using a large number of touchpoints across online and offline channels. What's really important and the exciting part is evolving the ability to measure all of this. We're learning and we'll come to a better understanding of the interplay between all of those touchpoints. This increases in media consumption have made it crucial for all of us as marketers to personalize our outreach and efforts even more than we're doing today. So, it's really about as modern consumers with access to such a wide media mix, we can ignore and gauge based on whether brand messages are relevant to us or not. Knowing more about that and helping companies make decisions about the content, and even content down to individual consumers is a real exciting area. Rajagopal: Absolutely. Thank you, Bob. Garth, if I could bring you in here, and then before we go on to Q&A, we'll see if Yash is back so we can get his perspective. But in the meantime, Garth, what excites you about the future of marketing effectiveness measurement? Viegas: So, Alarice, back to my original comment, market mix modeling helps you become a better marketing mixologist. It's a science that helps you understand what's actually driving your business. The wider marketing community, I feel has neglected this area, and we've become too focused on measuring individual silos, digital, so on and so forth. I think Yash alluded to that earlier on. Therefore, what I like about marketing mix is it's this broad tool that's able to focus and measure what we call hard measures such as sales. And marketing mix model allows you to do that. I personally in the recent past have seen a resurgence of market mix modeling. And Igor, I need to copy that line of yours, "Every generation discovers rock and roll." I think that's absolutely true, and I think this resurgence of market mix modeling is driven by three or four factors. The first one is privacy issues and signal loss. The second one is the technology coming into play, so always on. You don't wait for once a year, you don't wait twice a year, but you're almost getting your data regularly and make much snappier and faster decisions. And then the third thing is the democratization of these two with the advance in machine learning, AI, and things like that, we can definitely make this tool cheaper, faster, and better. And as I alluded to earlier on, 70% of all the marketing activity in the US is not evaluated probably because the tools are expensive. If we had something cheaper, faster, we could really expand this thing across our whole entire industry. Alarice, in conclusion. I don't think we might call it MMM. People seem to reshape it and we'll call it something else. The other day I heard somebody call it top-down incrementality. I'm not fussed about that because fundamentally, the fundamentals and the science does remain the same. It's all about getting shared learnings in place. It's all about getting the right mix in place. And it's all about optimizing your business revenue, be it market share, be it revenue, profits, so on, and so forth. So, that's really what excites me about market mix modeling. Rajagopal: Thank you, Garth. Okay. So, while we're pulling everybody up for Q&A, I just want to thank you all for your insights that you've shared so far. We do have some time for questions, so I want to dive right in, and if you haven't submitted a question yet, now would be the right time to do so. So, again, you can find the Q&A function on the left side of your screen down below the presenter headshots. Okay. So, before we dive into some of these questions, I do want to try Yash. Yash, what excites you about the future of marketing effectiveness measurement? Sikand: Yeah, I think there's two things. One is from a technical analytical point of view. I think as media has fragmented and as technology has developed, we can do so much more from a modeling perspective and try to drive real-time analytics to help people at the time decisions are being made. I think that's exciting just from an educational point of view is how do we drive better models, models that account for more variables and try to quantify the incrementality of all the great marketing thinking that's going on I think the second thing to me, which is just as important if not more important is if you have the right people that are running the models that know the business, they have the ability to influence the business. In my line of work and in my team, I keep telling my folks is, "We got to drive decisions and influence the business because we're the objective voice at the table. We're coming with data. We're coming with facts. Every sales guy is going to say, "Hey, my sales program is the best." Every marketer, as Garth alluded to, is going to say, "My marketing program is the best." But we are bringing kind of an objective unbiased point of view that can guide the organization to either grow, spend money more effectively, whatever we are trying to do. But that's very important. So, having the right people at the table influencing the decisions is also really exciting for me because that's what we're there for. Rajagopal: Right. And speaking of people, the first question I have here for you, Yash, is around data scientists. So, it says, "While data scientists are scarce resources and businesses are often reluctant to try new approaches when they're already in demand for existing needs, how can organizations make the most of their data science teams when looking at a DIY or DIY MMM options?" Sikand: Okay. So here is my thought about data scientists. I think it's going back to what I said a few minutes ago is you can be the best modeler on earth. You can quantify, have the most elaborate models that quantify everything. If you cannot communicate and you cannot influence the organization, the value is greatly diminished. And I think that's where organizations get into trouble. They can invest a lot of time and effort in building the technology, the data lakes, and do all this fancy stuff. But if they're not geared to what does marketing want? What are the answers and what impact is it going to have on the business? And can you influence them? So, to me, there's the whole science and art part. The science is really important, but then the art of negotiating, the art of influencing is just as important. And so, I think when you say, "How do you become more influential?" I think use cases are very important, but then also, being able to influence them with facts and understand where they are coming from because most marketers when they produce a marketing program and you're going in and telling them the baby is ugly, they don't like it. How you do it and how you help them get better and not just say, "Hey, this is not working." But it's how do you take that and say, "This is not working, but if you try this, there's a better chance of it working"? So, how do you build with them and help them, than just be what I would say this yellow, green, and red light kind of stop sign there? I do think that's very important. Rajagopal: Right, and I would echo that's probably across industries too because I do hear that very commonly where they call it a unicorn, I guess. The person who can communicate that effectively and do it all, right? Sikand: Yeah. Rajagopal: Absolutely. Okay, thank you, Yash. So, Igor, the next question is for you. It says, "Is MMM just for multichannel brands who need to measure digital and non-digital spending, or can it provide the same strategic overview of the business for digital brands?" Skokan: Yes. I think MMM offers remarkable added value. I mean, it's the combination of the business and the media perspective like one unique model, and the possibility of the total holistic understanding of the structural and dynamic drivers of growth of any company. It is our strong belief that MMM is equally important for digital and non-digital brands and media. Obviously, there is an interplay with the attribution methods or multi-touch attribution methods, so we shall see how that will evolve over time. But the aggregated data that is used in marketing mix models will always be available even if the journey-level data or clickstream data may not exist in future in the same way how it used to. So, we absolutely believe that MMM can be customized and adapted to the specific context of each advertiser be it digital or offline, and there's enough flexibility and customization in the data, the methodology, and the application of MMM to accommodate. Rajagopal: Thank you, Igor. Okay, if I could bring back Bob, I have one for you. It says, "Who should own MMM? Is it IT, or is it analytics, or dare I even add, somewhere else?" Gannon: Sure. Well, every function within an organization be it finance, operations, IT marketing, they have analytic functions within them. I've observed over the course of my career great collaboration between IT and marketing when it comes to analytics. IT facilitating marketing doing analytics by providing all of the necessary data. I've observed places where there was actually conflict between IT and the CMO organization. It works best when there's collaboration. When both parties sit down at a table, they discuss what the needs are, and both parties are open to that collaboration, working together to make sure that the data that's required for the analytics, the marketing analytics is in place, and fed in an efficient way. So, it can be owned by both. Obviously, the marketing people know a lot more about the tactics that they're trying to implement. But it should be a collaborative effort. That's where I've seen in my career on both the vendor side and the client side, it working best. Rajagopal: We see the same in some of our research with consumers in retail. Someone might own it or a couple of places might own it, but they all really want that center of excellence. I'll use that term loosely, but yeah, it's a collaborative effort. I would agree with that. Gannon: Yes, and the place I saw it work best, that function was called the center of excellence, and it was an intermediary function between finance, IT operations, and marketing. Yeah. Rajagopal: Yep. Great. Thank you, Bob. Okay. So, the next question I have for Garth, and it says, "What about the post-COVID world and the implications for MMM? Are companies going to work differently or see an impact on executing on the analytics for MMM?" Viegas: Thank you, Alarice. I think that's a good question. I think it's the 800-pound gorilla in the room. So, glad somebody raised that question. You're all aware that market mix modeling, or for that matter, any analytics heavily depends on historical data. COVID has actually distorted historical data. Depending on the industry, you're either 40% or you're 40% down. There's a lot of confusion around it. Now, dealing with it. Initially, most companies assumed that COVID would last maybe two, three months. I think we all did, and we said we can just strip out that data from the analytics, and life can go on as normal. But close to 15 months later, that hasn't changed. So, very early on, we at Analytic Edge adopted two strategies to actually deal with the impact of COVID. One was a data strategy. We started using Google mobility data, and that definitely helped us partially understand the impact of COVID. But that by itself was not enough. Definitely, some of our data scientists developed a very unique technique that was a combination of three ensemble models: traditional parametric model, machine learning, and AI. I don't have time to go into it, but happy to do any follow-up questions. But this unique technique has enabled us to actually improve our forecasting dramatically despite COVID conditions. Despite the lockups and lockdowns things like that. So, we've reinvented ourselves to be able to forecast accurately in this COVID world. Rajagopal: Thank you, Garth. I have one more for Igor. I have a question here that says, "You mentioned the validation of MMM models with experiments. Can you explain a little bit more about that?" Skokan: Yes. I mean, in one quick sentence, it's how do we bring MMM, which are typically validated using statistical techniques? I don't know, error or goodness-of-fit, R-squared whatnot. And then maybe forecasting power and things like that. But on the other hand, when we have ground-truth data such as experiments, and these, for example, experiments are relatively easy to set up on Facebook and a lot of the digital channels. You can just put aside 5%, 10% of the campaign holdout. Campaign that is normally people don't get impressions delivered. If they are in the control group and test group, receive the campaign normally, and then it's relatively easy to compare the groups and get a sense of incrementality of the campaign. And then bringing this information back into the model, there isn't necessarily an established way to do it, but if the models are Bayesian, this can be used as a prior, or if they are frequentist, there is a way how you can overlay the results of it with the model in addition to choosing the models that are closer to experiments during the model building. Skokan: So, we think that this is going to be and there are already case studies and approaches out there. But we think this is going to be more and more popular way to validate and calibrate marketing mix models with experiments. Rajagopal: Great. Thank you, Igor. Okay, I'm going to spring one more on you for your last thoughts. So, I like to always leave the audience with a best practice, a tip how to get started. Maybe it's a what not to do. So, I just want to bring you guys up one more time just for your last thoughts and while you're preparing, if you don't have something to end on, maybe it's your favorite cocktail. So, I think Igor, since we heard from you last, if I could bring you back up, do you have any last thoughts for the audience? Skokan: Yeah. I think one thing that I would want to say is folks on the call should rethink the measurement stock and start maybe by mapping the current, the future and maybe embrace the risks, accept certain trade-offs, but map what's important to your business, and what's future proof? And start what you need to know and what cadence, what actions and decisions are you taking? And the key, in my opinion, is going to be the triangulation of techniques. MMM, MTA attribution, pricing analytics, royalty analytics, everything that we heard that Garth does in his bio. All of that triangulated together. And my favorite cocktail is Negroni Rajagopal: Bob, any last thoughts? Gannon: Yeah, I'm just excited about the future of analytics and we're going to be learning a lot. This is kind of an inflection point for marketing mix modeling, and I think the type of work that Facebook is doing, the type of work that Analytic Edge is doing to make things faster, more affordable, and thus able to be spread more across companies and brands is a very exciting point for our industry. Rajagopal: Thank you, Bob. And Garth, if you wouldn't mind, last thoughts? Viegas: Thank you, Alarice. I mean, for me, it's all about getting the mix right. A lot of marketers think they can do it by gut, but unfortunately, with media fragmentation, channel fragmentation, that is becoming more and more challenging, and you need the power of analytics to actually help you guide you through that vast number of data and decisions. I think at the end of the day, a market mix model provides that structure for you to have those shared learnings in the organization, for you to optimize your revenue, and for you to fundamentally understand how your business is growing. My favorite cocktail is a dry martini. Rajagopal: Fabulous, thank you. Great. So, since we're running out of time, I'd like to, again, thank our speakers Igor, Yash, Bob, and Garth for giving us their subject matter expertise for today. I'd also like to thank Analytic Edge for sponsoring today's webinar. And finally, thank you to our attendees for devoting some of your very valuable time to be with us today as well. We hope you found it worthwhile.

Related Topics