BI and Agile : Emergent Products
This article is the first in a series exploring the specific challenges of applying Agile values, principles and practices in a Business Intelligence (BI) and data platform domain. We will cover the insights generated in several interactive workshops with both people from the BI domain as well as Agile practitioners.
While there are plenty of articles available on Agile practices in software and app development, there is relatively little shared on experiences applying these principles to the specific domain of BI; the traditional approach to developing in this field creates its own unique challenges when switching to an Agile way of working. These sessions and articles aim to fill this void and assist those who are currently struggling with these questions. The first of the series is about product ownership surrounding BI products.
The BI Product
The first question to answer, possibly one that gets overlooked, is: what is a BI product? While it might seem obvious at first, it quickly becomes an intriguing question. Some would consider the environment storing data and generating reports to be the product. Others might even describe the data architecture as a product on its own right. However, consider this: as a bank, would you consider the building housing an ATM machine (or its architecture) to be a product? These things are what a BI product needs, but not what it is.
So, what then is the BI product? The dreadful answer is: it depends.
One approach to identifying the BI product is to follow its value proposition: what is made possible by the solution you are building. This is largely determined by the customer (or customers) that will be receiving its benefits. Understanding the needs of the customer helps determine the purpose of the product.
Once there is a clear picture of the value proposition it is essential to understand how the value will be generated. The beneficiary of the product might differ from those who use it to generate the value. Understanding how users will interact with the product helps determine the shape it takes.
Bringing this all together, the BI product can take various (coexisting) forms:
- Weekly reports
- Real-time dashboards
- Media campaigns targeting the customer base
- A data platform for data scientists
- A self-service report building environment, or
- Something else entirely
Its shape depends on the needs of the customer and how these needs are being met. In short: follow the value and define your product.
BI Product Backlog Management
Next to the idea of daily scrum and the existence of sprints the Product Backlog is probably the most known artifact of the Agile approach. Looking at the backlog, we see that we must respect the usual principles that govern the backlog. But the main question is how we divide our defined value into smaller pieces of work.
It is tempting to create the Product Backlog based on the end user product (reports, dashboards). But in many cases a data platform provides value as a central service to many reports and dashboards. The BI-product is far more than the user sees. The central architecture supports multiple BI-products. When focusing on just the BI-Products that the users see, you run the risk of creating many overlapping product backlog items.
To work around that risk, a duplex approach with a split in the backlog between the central, architectural components and the specific end user products can provide better focus on impacted components of the total solution.
But this will have a direct link with the total product and in what way platform thinking will define the definition of the product. This will also depend on the approach and knowledge of the stakeholders.
Stakeholder Management
Given the duplex nature of the BI Product, it stands to reason that there will be a very diverse group of stakeholders. There are of course the business users, data managers, GDPR officers, external partners, etcetera. The main concern is balancing the need for information with the availability of the data supporting it. So, the focus is on both the business users (consumers) and the suppliers (data owners, external parties, etcetera).
Let’s start with the challenges the consumers bring. They want information, but often don’t know how to formulate the question or know what is possible. In this case an Agile approach can actually be very helpful. We can help by really getting them to focus on what they want to achieve. What do they want to do with the information? How is this going to help them make better decisions and how does this benefit the business? Answering these questions helps determine the priority of the work and thus optimises the value for the organisation as a whole.
The second group are our data suppliers. If consumers don’t know, suppliers may be even more challenged. The challenge here is that they often only consider their own little corner of the globe, but don’t see the bigger picture. When implementing BI-products, their data is going to be used in far more ways than they can anticipate. To improve the availability of data, changes might even be required to their application. How do we make sure that changes for the one won’t conflict with the interests of the other? This requires a complex set of backlogs that need to remain synchronised as much as possible.
In this situation agile can also help us to take control of the complexity. Consumers can embrace (scaled) agile techniques to create transparency on the potential value creation for the business through clear product backlogs. Suppliers can use these scaled agile techniques to translate these product backlogs into viable solutions.
Take-aways
After our first session, we can definitely conclude that a BI solution requires special attention to the way we set up our agile environment. This requires finding the BI product in the value it creates, splitting the product backlog and adopting a scaled approach. If we don’t adapt our agile way of working to the nature of the beast, then we will struggle to keep things on track. But we need to make sure that not only we understand why we’ve adapted our agile way of working, but the rest of the organisation also needs to understand this as well.
Next Topic
In this article we have zoomed in on various aspects of the BI product management, which helps to determine what to build. The logical next step is to look at how to build such a product, which has its own set of challenges. Traditional ways of developing tend to conflict with the fast Agile iterations and short learning cycles. In our next article (Agile data and BI: Architecture for change) we will be exploring how to build for change without losing robustness or integrity.
Rob Kool Ivo van der Heijden Rogier den Dulk Are you interested in more information about BI, data, agile or other subjects, please check out our website. Are you interested in other blogs about interesting (technical) subjects? Please click here.
