Challenges in Agile Product Development

Agile methods are used in product development for their benefits – efficiency and effectiveness. However, practitioners face a variety of challenges in successfully transitioning into agile mode of thinking and working. This article attempts to summarize the various challenges that agile teams face leaning on the author’s experience and referring to few published articles in this regard. The challenges are grouped into three areas – Alignment, Enablement and Operationalization.

1. Lack of Alignment within the organization and with external stakeholders on the agile transformation initiative

Transition to agile requires alignment across the organization (across levels & functions) and with stakeholders outside the organization as well. Top management commitment and support is to be obtained right at the start. While it is an advantage if the trigger for agile transformation comes from top management, the risk with that is that the organization may be forced into Agile transformation even if it is not suitable for the organization (or) without adequate enablement (or) with agile approaches chosen without adequate consideration of the organization’s business context. Adopting Agile methods should not be seen as the end in itself but as a means to achieve better business results. Leaders need to cascade the vision down to the teams clearly stating the benefits to the organization and employees, the strategic objectives, goals and the enablement plan. This will avoid the lack of ownership within the team members. Alignment with customers is crucial as agile methods heavily depend on the customer participation. Even in the case of internal IT projects, alignment with internal customers is required. Lack of alignment across other functions in the organization will lead to operational issues. This needs to be thought through and discussed well in advance.

2. Lack of Enablement through training & coaching for translating the agile philosophy into practice with appropriate changes in systems, processes and tools

Agile approach is basically a bunch of guiding principles and rest is to be derived and applied considering the context. Without training and coaching by professionals who have good experience in applying agile approach, teams will not be able to successfully apply agile approach. People playing critical roles like Agile Evangelist, Agile Coach and specifically in the case of SCRUM method the roles of Product Owner and SCRUM Master should be carefully chosen with required mindset, expertise and experience. Team members tend to overcommit without experienced people guiding them and this potentially leads to burnout. The project and program governance structure dashboards that are required for different levels are to be worked out carefully maintaining the spirit of the Agile philosophy. This needs the involvement of a good Agile coach. The agile transformation needs a cross functional team to guide in updating the systems and processes to enable the transformation and the teams. Specific tools are also to be introduced with the guidance of the Agile Coach. Even business models need to be worked out for the Agile context as discussed in detail in the post Agile product development: Sharing risks and rewards. In Agile methods, people important and hence the HR policies & processes need to be fine tuned.

3. Operationalization

Besides Alignment and Enablement, as they say ‘when the rubber meets the road’ there are several critical challenges during Operationalization. The challenges are specifically in:

Bringing about the changes especially in the Mindset

Many transformations fail to achieve desired results due to the gap in good change management. While Alignment and Enablement set the foundation for the changes to happen, special attention should be given to this aspect during the operational phase. The critical part in this is in bringing about the changes in mindset across all levels especially due to the heavy dependency of the Agile methods on a different mindset. Changes will definitely be possible if the concerned individuals understand what is in it for them and if there is a win-win situation. It is the responsibility of the concerned leaders and task force to do this through effective communication.

  • Ensuring good communication within team and with stakeholders

Communication starts from the beginning as early as the Alignment phase. During the operations, relevant and sufficient communication is to be ensured with all the parties involved. Of course, Agile methods rely heavily on the close interactions with customers and between team members and hence communication and collaboration is part of the approach. It needs to be ensured it happens and is effectively done.

  • Managing Scope, Cost, Schedule, Quality and other issues

Typical challenges associated with any project management activity involves Scope, Cost, Schedule and Quality.  In agile projects, since change is welcome as a philosophy, as mentioned earlier, the business model needs to be worked out to handle changes in scope well to avoid effort overruns. As evident these parameters are interlinked and they need to be managed in totality. From the cost perspective, budgeting for the total project is a challenge and some approaches are discussed in Agile product development: Sharing risks and rewards. Planning the budget, timeline and scope and adjusting is enabled well by agile methods but only if they are managed with right approach and mindset with good cooperation between the stakeholders and collaboration between team members.

Specifically, agile is not easy to be adopted in large projects as they are invariably distributed across geographies with many teams. Agile methods have evolved to address these challenges for such large and distributed projects. With support of the experienced Agile coach such projects can be accomplished successfully. Quality in agile methods stress on product quality with the emphasis on the process quality not so visible – to the extent that sometimes processes tend to be ignored in projects leading to different set of challenges. Team needs to ensure that value adding process steps – focused on product quality and other project objectives – are identified and followed.  

A partial list of articles referred is provided below for additional reading – this list is by no means exhaustive!

Software Productivity – Is it still relevant?

Much has been talked about how productivity is different in software industry, the challenges in not only measuring and interpreting but in defining productivity itself. Productivity is just about efficiency – the ratio of the output to input. Input is typically the human effort involved. Though it appears easy to get this through logs done by team members, there are challenges in getting the actual efforts spent due to variety of reasons. Besides, the team members do a variety of tasks which may not be directly related to the work on hand and the challenge is in dealing with efforts on such work. On measuring output the issue is in determining what is to be considered as output – Lines of Code, Function Points, Story Points, variants of these, Business Value delivered etc. Martin Fowler has discussed in detail and concludes that productivity cannot be measured.

Is productivity still relevant?

Productivity is defined and measured at different levels – individual, team and organization. It is measured to compare the efficiency in the production of software with respect to other organizations in the industry, across teams and between individuals. This efficiency determines the cost and the profitability at project and organization level. Hence, productivity is a critical input to budgeting of internal product development projects and estimation of costs for bids in fixed pricing model projects for customers (of course, the underlying big assumption is that the software requirements have been sized correctly!). At an organization level it is about profitability. However, in order to analyze organization’s profitability and improve the same, there is a need to define and measure productivity at a lifecycle level. At the same time going to individual level productivity measurement can impact team work and collaboration amongst team members leading to in the worst case pockets of high efficiency and overall degradation in productivity. Agile methodology appears to be not/less focused on productivity. The focus there is on delivering value incrementally in every release with product quality. However, the velocity with which value is delivered assuming a stable team exists is the focus – this comes with its own similar challenges. Treating software work as a flow is another possibility so that methodologies like Kanban and Theory of Constraints can be applied effectively. For some lifecycle steps, it is very straight forward to apply Kanban approach and measure productivity as well as other throughput measures like inventory and cycle time. Steve McConnell author of ‘Code Complete’ – the popular book on best practices in software production – has discussed various aspects of productivity in his talk. He has proposed a scorecard approach for productivity in which the involvement of stakeholders and their perspective on what is business value is considered. Other alternatives are to break down the work in the software industry into its lifecycle steps and deal with productivity in each step. Organizations and teams do have their own variants in this regards as per context.

Thus, organizations and teams need to identify why they want to measure productivity given their context (nature of work done, project lifecycles, execution methodologies, management policies and organization culture), define it appropriately along with other alternative metrics to productivity, measure them to the extent required, analyze the measured performance carefully to draw insights and use the same!

Do we have a strategy?

While ‘strategy’ is a commonly used term in corporate world, it is quite difficult to define what it is, to determine if an organization has one and to communicate the organization’s strategy. Many times this results in the question ‘Do we have a strategy?’. Hambrick, D.C., & Fredrickson, J.W., (2005) have introduced the ‘Strategy Diamond’ framework which is useful to identify 5 key elements of an organization’s strategy, to address the challenges in comprehensively specifying an organization’s strategy and communicating the same. In their paper, they have positioned strategy as a central integrated and externally oriented approach of an organization to achieve its goals. A key input to this strategy formulation is the outcome of the strategic analysis of the organization in the context of the industry.

The elements of the ‘Strategy Diamond’ are:

  • Arenas: Where will we be active and with how much emphasis?
  • Vehicles: How will we get there?
  • Differentiators: How will we win in the marketplace?
  • Staging: What will be our speed and sequence of moves?
  • Economic Logic: How will we obtain our returns?

By addressing the specific details spelt out in the paper for each of these 5 elements, organizations can come up with their strategy, specify it objectively and communicate the same to stakeholders. The authors have illustrated the application of the ‘Strategy Diamond’ in the context of two organizations.

Reference:

  • Hambrick, D. C., & Fredrickson, J. W. (2005). Are you sure you have a strategy?. Academy of Management Perspectives19(4), 51-62.

Execution Excellence through 4DX

“Strategy is a commodity, execution is an art. “

Peter Drucker

Strategy needs to be executed for achieving the desired outcomes. In the book titled ‘The 4 Disciplines of Execution: Achieving Your Wildly Important Goals”, the authors McChesney, C., Huling, J., Covey, S and Kaiker, R share their insights in the form of 4 disciplines which leaders and employees have adopted to produce results. I am sharing a brief description of these disciplines here for your quick reference.

The four disciplines of execution are:

  1. Focus on the wildly important
  2. Act on the lead measures
  3. Keep a compelling scoreboard
  4. Create a cadence of accountability

Focus on the wildly important: ‘Focus on less and accomplish more’ is the message here. We can easily relate to this as we tend to accomplish those few goals that we treat as ‘must-do’. Time and effort that we can invest in anything is bounded by constraints and hence prioritizing what is important and narrowing down the focus to the wildly important goals is the first step in execution excellence.

Act on the lead measures: They say “What gets measured gets done”. What we measure, track and act too needs a shift. Typically the focus is on the lag measures which track the outcome of the wildly important goal. Instead this discipline expects a shift to track and act upon lead measures which lead to the lag measures (or) the final outcome. This way we work on what we can influence.

Keep a compelling scoreboard: Individuals and teams need to know where they are with respect to the goal and through the lead and lag measures. A compelling scoreboard created by the team and accessible to them will create an engagement within the team improving the chances of successful outcomes.

Create a cadence of accountability: Having identified the wildly important goal, the lead measures to measure and track and a compelling scoreboard the team is ready to execute. During execution there should be a regular tracking and review mechanism which is brief and effective focused on the accountabilities of each individual for achieving the wildly important goal. SCRUM methodology uses daily standup meetings for this. In this meeting the individuals make commitments and update the progress on their commitments.

These 4 disciplines of execution excellence are practical and impactful. The book provides simple tools to aid in the adoption of these disciplines by teams and organizations. For more details, you can watch the introductory video and refer the book: McChesney, C., Huling, J., Covey, S. (2016). The 4 Disciplines of Execution: Achieving Your Wildly Important Goals. United Kingdom: Free Press.

Preparing industry-ready managers!

Management institutes prepare students into managers for the industry through well established methods. I had the opportunity to share my perspectives on this topic to a group of management faculty based on my experience as a practitioner, teacher and student. I am sharing a gist of my talk here.

What does it take to be an industry-ready manager?: I have summarized what I found published and what I could relate to based on my experience in the picture above. At the core is the value system. Values serve as the canvas for any profession and management is no exception. For example, ‘Integrity’ is a value which is expected of every manager. I have grouped the skills into ‘Employability Skills’ and ‘Professional Skills’. The employability skills are common across professions and I found a good grouping of these in a published report (Gibb, J. (2004). Generic Skills in Vocational Education and Training: Research Readings. National Centre for Vocational Education Research (NCVER)). They are Communication, Teamwork, Problem Solving, Initiative & Enterprise, Planning & Organizing and Learning skills. I have grouped the professional skills required for managers into ‘Management Practice’, Tools and Best Practices. These can be viewed in two parts – Generic and Industry-specific as different industries have their specific needs and the management concepts, techniques and practices get evolved for their specific context. Besides these three professional skills, an awareness of the industry in which the management professional is working is key to be successful.

With this backdrop on what it takes, let us dwell into how to enable management students become industry-ready. It is well accepted practice in good management institutes to weave experiential learning with conceptual learning. I had the chance to see this really work in my experience of grooming project managers on the job and teaching them the concepts in an internal project management academy setting within my organization. The typical coursework in management institutes covers the concepts very well coupled with assignments and case discussions. I would like to emphasize on the industry immersion through internships and live industry projects. The success of this will depend on the involvement of the industry sponsor/advisor, the faculty identified and the student. Sufficient time should be provided for the industry immersion with appropriate credits. Besides these, students need to be connected with industry leaders through forum like expert talks and mentorship programs. Internal societies/clubs are also a great way to create industry-like experience within campus. Students will have to carefully choose the industry they would like to focus on and pick the projects/internships accordingly. Professional certifications by industry forums do help in many cases. Students should consult industry leaders/mentors in choosing these considering the returns for their efforts and money invested.

I hope this information is useful to the management faculty and management students. Look forward to your views, comments and any additional inputs.