Technical Experts Must Engage Closely with Business Peers to Avoid AI Failures
A recent RAND Corporation study has shed light on the difficulties of applying agile software development practices to artificial intelligence (AI) projects.
Conducted for the U.S. Department of Defense and finalized in April 2024, the research highlights significant challenges faced when integrating agile methodologies with AI design and implementation.
The study, led by James Ryseff, senior technical policy analyst at RAND, draws on interviews with 65 experienced data scientists and engineers from both industry and academia. According to the report, while agile practices have been successful in many software development contexts, they may only be suitable for some AI projects.
“All too often, AI projects flounder or never get off the ground,” Ryseff said. The report notes that formal agile practices may impede AI success rather than foster it. “Several interviewees (10 of 50) believed that rigid interpretations of agile processes are a poor fit for AI projects.”
The study indicates that rigid applications of agile processes can hinder rather than help AI development. Some interviewees expressed concerns that traditional agile methods could be a better fit for AI, where data exploration and experimentation often require more flexible and extended timelines.
Key Concerns of AI Project Failures
The RAND report identifies several key factors contributing to AI project failures:
- Misunderstood Problems: “Industry stakeholders often misunderstand — or miscommunicate — what problem needs to be solved using AI.” Consequently, models may optimize incorrect metrics or fail to integrate effectively into existing workflows.
- Inadequate Data: “Many AI projects fail because the organization lacks the necessary data to train an effective AI model adequately.”
- Focus on Technology Over Solutions: “The organization focuses more on using the latest and greatest technology than solving real problems for their intended users.”
- Insufficient Infrastructure: “Organizations might not have adequate infrastructure to manage their data and deploy completed AI models, which increases the likelihood of project failure.”
- Overly Ambitious Applications: “AI projects fail because the technology is applied to problems that are too difficult for AI to solve.”
Recommendations For Success – Study Says
To overcome these challenges, the report suggests several strategies:
- Open Communication: “Misunderstandings and miscommunications about the intent and purpose of the project are the most common reasons for AI project failure.” Technical teams should engage in frequent and open communication with business stakeholders to ensure alignment and build trust.
- Commit to Long-Term Goals: “AI projects require time and patience to complete.” Leaders should commit to solving significant problems over extended periods. AI projects require long-term dedication, often spanning at least a year.
- Focus on Problem-Solving, Not Just Technology: “Successful projects are laser-focused on the problem to be solved, not the technology used to solve it.” Prioritize addressing real problems users face rather than merely adopting the latest technologies.
- Invest in Infrastructure: Upfront investments in infrastructure to support data governance and model deployment can substantially reduce the time required to complete AI projects and increase the volume of high-quality data available to train effective AI models.
- Understand AI’s limitations: “When considering a potential AI project, leaders need to include technical experts to assess the project’s feasibility.”
Summary!
While agile methodologies have proven effective in many areas of software development, AI projects present unique challenges that require flexible approaches and robust communication. Understanding these challenges and investing in appropriate infrastructure are crucial steps for organizations looking to leverage AI successfully.
Co-founder and Chief growth officer