Rebecca Warren and Michael Watson discussed the challenges of transitioning from AI pilots to performance, emphasizing the importance of the CIO-CHRO alliance. They highlighted a survey of 700 executives, revealing that while 92% of companies experiment with AI, only 21% scale it effectively. The conversation covered the “pilot purgatory paradox,” where prolonged pilot phases indicate organizational avoidance. They stressed the need for robust data sets to avoid bias and the importance of leadership and HR alignment to manage change and foster adoption. The key is to shift from a knowledge-expert mindset to a curious-learner approach, focusing on augmentation rather than replacement.

Introduction and backgrounds of speakers

• Rebecca Warren introduces herself and Michael Watson, explaining their roles and backgrounds at Eightfold.
• Rebecca Warren has been at Eightfold for over five years, moving from talent acquisition to leadership and now focuses on talent center transformation.
• Michael Watson has been at Eightfold for six years, initially as a customer and later as part of the customer success team.
• Both speakers express excitement about discussing the pilot purgatory paradox and its implications for AI success.

Understanding the pilot purgatory paradox

• Rebecca Warren explains the concept of pilot purgatory, where companies repeatedly pilot AI projects without achieving significant results.
• The speakers discuss the misconception that AI implementation is as simple as a software update.
• Rebecca Warren emphasizes that AI projects require significant organizational changes, not just technical updates.
• Michael Watson highlights the challenges of integrating AI with legal and compliance requirements, which can complicate pilot projects.

Challenges of AI implementation

• Rebecca Warren argues that if an AI pilot lasts more than six months, it is likely organizational avoidance rather than a genuine pilot.
• Michael Watson agrees, noting that fear of failure and the desire to avoid responsibility can lead to prolonged pilot phases.
• The speakers discuss the importance of having a clear plan and commitment to AI projects to avoid getting stuck in pilot purgatory.
• Rebecca Warren and Michael Watson stress the need for collaboration between IT and HR to ensure successful AI implementation.

Data and gaps in AI scaling

• Rebecca Warren presents data showing that while many companies experiment with AI, only a small percentage successfully scale their AI initiatives.
• The speakers discuss the challenges of the “messy middle” where AI strategies often stall and fail to gain traction.
• Michael Watson explains the importance of understanding the different types of AI and avoiding the pitfall of trying to implement AI in every aspect of the organization.
• Rebecca Warren highlights the need for proper data management and organizational alignment to achieve successful AI scaling.

Myths of an IT upgrade

• Rebecca Warren argues that treating AI as an IT upgrade rather than a cultural and organizational change can lead to shallow adoption and resistance.
• Michael Watson agrees, noting that AI implementation requires significant change management and collaboration with HR.
• The speakers discuss the importance of addressing human friction and cultural resistance to achieve successful AI adoption.
• Rebecca Warren emphasizes that AI does not create new issues but amplifies existing ones, making it crucial to address underlying organizational challenges.

The role of leadership and HR in AI success

• Rebecca Warren and Michael Watson discuss the importance of leadership in guiding the organization through AI changes.
• The speakers highlight the need for leaders to be open to hiring people smarter than themselves and to focus on areas where they excel.
• Michael Watson shares his experience of leading AI initiatives and the importance of collaboration between IT and HR.
• Rebecca Warren emphasizes the need for a strong partnership between CIOs and CHROs to ensure successful AI implementation and organizational growth.

Shifting the narrative on AI

• Rebecca Warren argues that the fear of being replaced by AI is a significant barrier to successful implementation.
• The speakers discuss the importance of shifting the narrative from AI as a threat to AI as an opportunity for augmentation and growth.
• Michael Watson shares his perspective on the generational shift, noting that younger employees are more comfortable with AI and expect forward-thinking organizations.
• Rebecca Warren emphasizes the need for organizations to build skills and roles that evolve with AI rather than focusing on static roles.

Final thoughts and recommendations

• Rebecca Warren and Michael Watson wrap up the discussion by emphasizing the importance of collaboration and alignment between IT and HR.
• Michael Watson advises CIOs to evaluate AI vendors based on the size and robustness of their data sets to avoid bias.
• Rebecca Warren encourages leaders to transition from knowledge experts to curious learners and to build trust and partnership across the organization.
• The speakers highlight the need for organizations to embrace AI as an opportunity for growth and to prepare for the future workforce.

Rebecca Warren 0:08 Hey folks, super excited to be here today. We are talking with you, the Argyle audience, about moving from pilots to performance and why AI success depends on the CIO-CHRO alliance. I am Rebecca Warren, and I have Mr. Michael Watson with me. We will introduce ourselves and tell you a little bit more about who we are, and then we’ll jump into our topic for the day. So, Rebecca Warren—I have been at Eightfold for over five years now. My background is in talent acquisition. I moved from practitioner to leadership and then over to Customer Success at Eightfold, which felt like a strange jump at the beginning, but I made it work. I spent a couple of years in Customer Success helping to build out the team and moved into leadership there. Now I sit under Marketing in the Talent Center Transformation area, where we spend time thinking about thought leadership. How do we move from jobs to talent, and from org charts to potential? All right, I’m going to flip it on over to Mike. Michael, I’m not sure how you want to be addressed, but you’re up.

Michael Watson 1:21 Either; I respond to both. So Rebecca, thank you all so much. Argyle team, nice to see everyone here. Michael Watson, with Eightfold. I just celebrated my sixth anniversary with Eightfold. It’s amazing how time flies when you’re having a great time. I was originally a customer of Eightfold, so I remember having these conversations with my CIO (Chief Information Officer) about, “Hey, I want to implement some AI technology.” This was back in 2018 and 2019—people looked at me like I was crazy. Nobody was thinking about that then, right? So, I’m happy to be here. For the first five years with Eightfold, I helped build out the Customer Success team. Rebecca was my very first hire, and then she broke my heart and left me to move on to other parts of the organization, and here we are reunited again. For the last year, I’ve been running our Customer Enablement and, specifically, our education and training delivery. We’re excited to talk about this “Pilot Purgatory Paradox.”

Rebecca Warren 2:24 What does this mean? Okay, so let’s jump in. We did a survey recently with ThoughtLab where we had over 700 executives, and we got their thoughts on AI, HR, and tech. We’re going to talk about some of the things that we learned, and this “Pilot Purgatory” concept came out of that. A lot of folks talk about AI like it’s just a software update. Michael, as you introduced yourself, you mentioned that when you brought it up to folks, they looked at you like you were a little bit crazy. It feels like just a project—we just need to click “Install,” get our coffee, and come back to productivity gains all because of a software update. But that’s not really what’s happening. What we’re actually seeing is folks who say, “We’re going to do AI,” so they start a pilot or a test. They find a complication, they pause, and they rename it. It’s still the same project. They swap out the team and try again. Sometimes they quit. That cycle is what we’re calling Pilot Purgatory. These companies aren’t necessarily picking the wrong technology, but they underestimated what it takes to move from experimenting with AI as a project to actually changing how work gets done. Michael, what do you think?

Michael Watson 4:21 I definitely see it. I think because it’s so easy for us on the consumer side to download the ChatGPT app and use it, we think we’re “doing AI.” Then we think we can automatically transfer that to the corporate world, not realizing we have legal teams that have to review data retention policies, GDPR (General Data Protection Regulation), and redline contracts. A big part of this Pilot Purgatory is running into Legal. You have a great use case, you set up a pilot, and then Legal says, “Hey, we’re not ready for this.” If you are going to move beyond the pilot, picking the right AI solution and knowing what to look for is vital to keep you out of hot water.

Rebecca Warren 5:44 I think there are a lot of reasons why people do a pilot. It feels easier; it’s dipping your toe in the water. But I’m going to throw a spicy statement at you: If your AI pilot has lasted more than six months, it’s not a pilot—it’s organizational avoidance with a budget. What do you think about that?

Michael Watson 6:35 I think if that’s the situation you’re in, someone is afraid to put their name behind something because they’re afraid that if it fails, they might get fired. Once the price point creeps up to where your job is on the line, people say, “Boss, it’s just a pilot. We’re just kicking the tires.” There’s also the “Fear Of Missing Out.” People are afraid to make a choice now because of what might come out next week. You find yourself jumping from pilot to pilot and never really having any measurable ROI (Return on Investment).

Rebecca Warren 9:20 Moving on to data and gaps. We’ve got 92% of companies “doing AI,” but only 21% are actually scaling AI—the ones winning the game. This is about the “Messy Middle,” where things tend to stall. It’s not usually that the model didn’t work; it’s about not having a plan. We don’t know where AI fits, we don’t have the skills, or we haven’t aligned the organization. Strategies often don’t end with a big blow-up; they just fade away. Michael, I also think people confuse simple automation with AI.

Michael Watson 11:36 Exactly. There’s a chasm because of what I call “boiling the ocean.” Companies expect AI to be a panacea. Marketing is making AI commercials, Sales has AI bots, and HR is running their own thing with no consistency. As a CIO, everything has to come through you—SSO (Single Sign-On), security clearances, ISO (International Organization for Standardization) standards. Instead of boiling the ocean, try a smaller, piecemeal approach. Start with a small team and a specific goal, then expand. Doing too much too early leads to analysis paralysis.

Rebecca Warren 13:30 You’re “peanut butter spreading” AI on everything. You’re just putting lipstick on a pig if you don’t fix the underlying issues. The Messy Middle happens due to bad data, lack of frameworks, or systems that don’t talk to each other.

Rebecca Warren 15:25 Let’s talk about the myths of an IT (Information Technology) upgrade. When we look at the surface, it feels like a software deployment. But underneath the surface, there is human friction, culture, and structure.

Michael Watson 16:46 I never fear losing my job because IT is upgrading to the next Windows version or moving from Webex to Microsoft Teams. But when you bring in AI, humans think, “Is my job at risk?” This is why the partnership between the CIO and CHRO (Chief Human Resources Officer) is so important. The human aspect—the fear, the trepidation, and the reskilling—falls on HR.

Rebecca Warren 17:48 AI doesn’t create issues; it amplifies the ones that already exist. Here’s another spicy statement: AI doesn’t just expose skills gaps; AI exposes leadership gaps. That’s why it makes people uncomfortable.

Michael Watson 19:05 I think it’s both. As a leader, you have to be willing to hire people smarter than yourself. If you don’t want to do that because of ego, you’re going to have a hard time.

Rebecca Warren 21:28 I think what makes leaders uncomfortable is the transition from being a knowledge expert to being a curious learner. Leaders are usually elevated because they know the most. When you have to say, “I don’t have all the answers; let’s figure this out together,” it’s a major shift.

Rebecca Warren 24:13 If these two groups aren’t aligned, things break. We use a car analogy: IT provides the engine and the horsepower, but the CHRO and the people provide the driver and the navigation. If the HR team isn’t in the room from day one, the organization has already decided that adoption is optional.

Michael Watson 25:45 I can point to Blockbuster, Kodak, or BlackBerry—cases where they didn’t see tech trends coming. To the CIOs out there: you are on the front lines to ensure your company doesn’t go extinct. But remember, 70% of software implementations fail because of change management. If you didn’t get HR involved early to ask, “What’s in it for the employees?”, you’re going to struggle.

Rebecca Warren 28:07 We have to flip the narrative. Leaders succeeding with AI are focusing on augmentation, not replacement. We are moving from hiring for static roles to looking at learning potential and evolving skills.

Michael Watson 30:25 I have a 20-year-old in college. For Generation Z and Generation Alpha, AI is all they know. My 88-year-old dad says he would never interview with an AI bot; my son says he’d rather interview with an AI bot than a human. Organizations need to build systems for the generation that has embraced AI from day one.

Rebecca Warren 33:22 Failing at AI isn’t a tech problem; it’s an alignment problem. Mike, what’s your last two-second wrap-up?

Michael Watson 34:04 For the CIOs: when evaluating vendors, check the size of their data sets. Larger, more robust data sets eliminate bias. Think of it like a glass of salt water—if you dump it into a fresh-water pool, you won’t taste the salt. Evaluate the “salt” in your data.

Rebecca Warren 34:56 And my final thought: move from knowledge expert to curious learner. Partner across the entire organization. That builds the trust and resiliency that allows you to scale. That’s all we have for today. Thanks for joining us!