Agile methodologies have gained widespread adoption across all market sectors since their conception, yielding positive results. The industry's acceptance of Agile continues to grow even today.
As companies increasingly integrate Artificial Intelligence-powered technologies into their products, questions arise regarding project and initiative management around these products. Is Agile the solution for AI development, or should organizations seek alternative methods to manage their AI-related software projects?
To begin, AI development differs in many ways from traditional software development. The programming paradigm varies between the two, making it challenging to apply the same agile or predictive framework to support the necessary activities for anticipated increments.
Two of the main challenges in AI development are the uncertainty when planning expected increments within a specific timeframe and the management of tasks by stakeholders due to the high technical effort required throughout the iterative experimentation process. This process may involve multiple iterations before achieving a tangible user product.
Although AI environments have their own complexities that differ from traditional Agile frameworks, it is important to note that Agile methodologies were specifically designed to address complex problems where traditional or predictive methodologies fell short.
Therefore, Agile possesses the necessary elements to ensure a suitable set of practices for successful AI product development. However, it is crucial to clearly understand the context to determine the most effective way to leverage Agile in AI product development.
First, it is important to address the concept of AI and then identify how it matches the Agile definition.
Artificial Intelligence and Traditional Agile Methodologies
Artificial intelligence (AI) refers to the capability of a digital computer or computer-controlled robot to perform tasks typically associated with intelligent beings. Examples of AI applications include expert systems, natural language processing, speech recognition, and machine vision. AI is employed in various areas, such as advanced web search engines, customer support systems, speech comprehension, self-driving cars, generative tools, automated decision-making, and strategic game systems. AI systems analyze large volumes of training data to identify correlations and patterns that are then utilized to make predictions.
On the other hand, Agile is a broad concept encompassing different methodologies and frameworks based on the principles outlined in the Agile Manifesto published in 2001. Among these, Scrum is widely used by companies adopting Agile as their primary approach to work. It is important to understand that while Scrum is part of the Agile framework, Agile extends beyond the individual frameworks and encompasses a wider range of practices than what is typically associated with Scrum or Kanban.
In traditional Agile implementation, often based on Scrum, the organization, in collaboration with the team, defines a vision for the desired product. This vision is then broken down into deliverables, grouped into releases, and divided into incremental sprint cycles. Each release and sprint is assigned a specific timeframe, serving as the iterative cadence for developing the solution. During this timeframe, various ceremonies and activities are carried out to achieve the desired product increments.
When it comes to AI development, one of the first issues that arise is that the product increment is only sometimes visually or tangibly defined straightforwardly by the product owner since much of the development effort happens at the technical layers.
Another factor to consider is that user stories or product backlog items in traditional Agile development contain desired functionalities and criteria for validating the product against end-user expectations. However, in AI development, it can be challenging to define these cards with sufficient detail to allow for accurate estimation by the team.
Moreover, developing artificial intelligence requires a detailed study of available information, careful analysis of solution alternatives, and repeated hypothesis testing to determine the best approach for obtaining the expected product/solution. Consequently, much of the effort focuses on research, learning, and adjustment before tangible increments can be achieved.
All of these factors create a "collision" since the results of AI development are not always predictable or linear, especially in terms of effort and time. The process requires constant experimentation and exploration, increasing uncertainty in planning and projecting increments. This, in turn, biases the expected result of each process, at least from a traditional viewpoint.
Agile in Complex Environments
When dealing with complex systems, the Cynefin model identifies complexity as a category where solutions are unpredictable and often require extensive experimentation or adaptation. In this context, the model proposes the "Probe – Sense – Respond" approach to navigate the complexity effectively.
Given the inherent variability in finding solutions, traditional predictive methodologies like Waterfall are not well-suited. Instead, Agile and its mindset provide a more suitable approach to address the challenges that may emerge during the process.
However, not all Agile frameworks may be suitable for complex scenarios. That's why it is crucial to focus on the Agile mindset, principles, and pillars outlined in the Agile Manifesto. By embracing these foundational elements, we can find ways to address any scenario by incorporating work cycles that enable continuous inspection and adaptation, along with early and frequent delivery of value.
Therefore, before establishing a specific framework for executing AI developments, ensuring a shared understanding of the spirit of agility among technical and business team members is important. With this common understanding, the process that best aligns with the pursued objectives can be defined.
Defining Keys to Identify Practices
To address a complex scenario effectively, it is important to define key points that provide transparency regarding the framework's expectations to all parties. This will allow for the selection of practices that best fit the needs.
Every organization has different reasons for adopting an Agile perspective and investing the effort required to make the most of its implementation. Based on these objectives, you can achieve a methodological definition that makes sense for the best use of Agile practices in developing AI projects.
Understanding Agile as a whole enables companies to identify the best way to leverage all the tools it provides, whether they want to implement a new way of working or improve an existing framework.
By expanding the range of practices and tools available under the Agile umbrella, teams can enhance their capabilities vertically, through scaling the practice, and horizontally, by improving their current Agile application to foster greater team maturity and productivity.
Example of an Agile Approach to AI
One of the Agile frameworks well-suited for the characteristics of AI is Kanban, primarily due to its visibility. It empowers teams to establish a customized workflow that fosters synchronicity among all team members and initiatives at various stages on the board. By incorporating elements of Scrum, such as daily ceremonies, teams can make optimal decisions and take appropriate actions in response to any changes that arise during the process.
Another valuable aspect of Kanban is its adaptive use of diverse activities, artifacts, tools, and Agile practices. It remains flexible to adapt according to the needs while ensuring the ultimate goal of its application is not compromised.
This allows for the incorporation of periodic product review sessions and daily ceremonies. These sessions enable real-time validation of completed products without disrupting the workflow. Moreover, they allow relevant stakeholders to participate at the most opportune moments when they can contribute the most value.
Finally, a continuous improvement cycle is essential for successfully applying agility. While a Scrum retrospective occurs at the end of each iteration, sprint cycles are not inherent to Kanban. Therefore, a "soft" cycle is defined to incorporate an inspection and adaptation cadence.
These inspection and adaptation sessions facilitate the continuous improvement of the model, work team, and the different Agile practices being implemented. They aim to enhance existing strengths and create action plans for identified improvement opportunities.
Conclusion
The essence of the Agile framework umbrella lies in its ability to address the most complex market scenarios. By utilizing different practices, techniques, and tools, it provides the necessary flexibility in defining a schema to consistently generate expected value for the organization. This is achieved within an orderly, formal, and scalable environment.
To achieve success, it is essential to align with the core principles and pillars of agility. These should serve as the axis of action and be connected with the organization’s expected value to identify appropriate strategies for success in each scenario.
About Encora
Fast-growing tech companies partner with Encora to outsource product development and drive growth. Contact us to learn more about our software engineering capabilities.
