Navigating AI: New Ways to Measure Marketing Success

Article Summary

Many companies are trying to understand how to measure the effectiveness of Artificial Intelligence (AI) in marketing. It is difficult to know how AI chatbots influence how people discover brands. Michael McCash from Lendi Group, a home loan company, states that "you can't improve what you can't measure." Two years ago, Michael McCash joined Lendi. His goal was to improve their annual paid media budget, which was over $10 million. To do this, he helped create an internal system for multi-touch attribution and hired Mutinex for marketing mix modeling (MMM). Lendi is also a pilot brand customer for Mutinex's new AI measurement solution called 'AI Visibility'.

Using MMM helped Lendi discover surprising insights. For instance, Lendi used to spend a lot on streaming TV. However, MMM reports showed that older, free-to-air television was actually more efficient and had been underweighted in their budget. Lendi then moved money from less effective channels to free-to-air TV, which significantly increased their ROI (Return on Investment). Marketers often prefer channels they use themselves, like streaming, and trust data that is easy to see, such as clicks or impressions. But AI has made the initial brand discovery phase less clear, similar to how online search was when it was new. It's hard to see how AI platforms like ChatGPT surface brands or respond to relevant searches.

To solve this, tools like Mutinex's AI Visibility mass-prompt LLMs (large language models) with questions like "find home loans." They then analyze the answers to get visibility into AI's behavior. Lendi also works with other vendors to track brand metrics and how AI treats their brand compared to competitors. While Lendi is testing ads in AI chatbots, the customer acquisition rates and overall ROI have shown middling performance so far. However, they see a new type of customer: one who has already done extensive research using AI chatbots. These customers are more informed and closer to making a purchase. Therefore, Lendi's current AI investment focuses on measurement and understanding AI's impact on discovery, rather than just buying ads. They believe gaining access to this customer research data is the next frontier for marketers, as AI chatbots are becoming the first port of call for millions of people.

Key Vocabulary

Comprehension Questions

1. What was Lendi Group's main goal when Michael McCash joined two years ago?

• To increase their streaming TV budget
• To develop new AI chatbot features
• To improve their annual paid media spending
• To become a marketing mix modeling vendor

2. According to the article, which advertising channel did Lendi discover was more efficient than previously thought?

• Streaming TV
• Social media ads
• Free-to-air television
• AI chatbot advertising

3. Why did Lendi decide to shift advertising spending from streaming TV to free-to-air television?

• Because free-to-air TV was a new and untested channel.
• Because MMM reports showed free-to-air TV offered better ROI.
• Because their target customers stopped watching streaming TV.
• Because streaming TV costs became too high unexpectedly.

4. Based on the article, what is the main challenge marketers face when trying to understand AI's impact on brand discovery?

• The high cost of AI chatbot advertising.
• The lack of clear data on how AI platforms surface brands.
• Customers prefer traditional search engines over AI chatbots.
• AI chatbots are not yet popular enough for brand discovery.

5. Do you think Lendi's strategy of focusing on measurement and understanding AI's impact on discovery, rather than direct advertising in chatbots, is a wise approach? Why or why not?

• Yes, because understanding customer behavior is more valuable long-term.
• No, because direct advertising always guarantees better immediate sales.
• Yes, but only if AI chatbots become the sole source of information.
• No, they should invest equally in both to maximize reach.

Discussion Prompts

1. How do you currently measure the success of your company's marketing efforts? Are there any challenges in getting accurate data?

2. Has your company explored using AI tools, either for internal processes or for engaging with customers? What were the results or expectations?

3. The article mentions that marketers sometimes prefer channels they use themselves. How might personal preferences influence business decisions in your industry, and how can this be managed?

Live Session Prep & Cheat Sheet

🎯 Speaking Targets (Vocabulary)

Try to use these target terms in your speaking turns:

• metrics
• ROI
• visibility
• customer acquisition rates
• first port of call

⚙️ Grammar Target Formula

Expressing Cause and Effect with Conjunctions: Cause: (Because/Since/As + clause) --> Effect: (clause). OR Effect: (clause) --> Result: (So/As a result + clause).

💬 Discussion Openers

Use these phrases to open or structure your arguments:

• In my experience...
• I agree with that point, and I'd add...
• Could you explain what you mean by...?
• From my perspective...
• That's an interesting point, but I wonder if...

Teacher Notes

This lesson explores how a company, Lendi, is adapting its marketing strategy in the age of AI. Students will discuss the challenges of measuring AI's impact and evaluate different approaches to digital marketing. Encourage students to share their own experiences with marketing measurement and AI in their professional lives. The grammar focus on cause and effect will help them articulate complex business situations clearly.

Speaking Class Facilitation Guide (Tutors/Moderators Only)

🎭 Role-Play Scenario

Situation: Your company needs to decide where to invest its marketing budget for the next quarter. Recent data shows traditional channels are still effective, but there's increasing pressure to invest in new AI-driven platforms, even though their ROI is still unclear.

Goal: Reach a mutual agreement on how to distribute the marketing budget, considering both innovation and profitability.

⚖️ Debate Prompt

{"side_a":["Early adoption provides a competitive edge.","AI will become the primary customer interface.","New channels attract new customer segments."],"side_b":["Focusing on proven channels guarantees higher ROI.","Unproven channels carry high financial risk.","Traditional channels still reach a large audience effectively."],"question":"Should companies prioritize investing in new, unproven AI marketing channels over established, profitable traditional channels?"}

💡 Discussion Facilitation Tips

Encourage students to use the cause/effect conjunctions discussed in the grammar focus. Prompt students to back up their opinions with examples from their own professional experience or the article. Remind students to actively listen and build on each other's points using phrases from the discussion openers.

Session Blueprint

How do you currently measure the success of your company's marketing efforts? Are there any challenges in getting accurate data?