Data-Driven Product Management: How Your Business Profits In navigating new, unprecedented change, disruption, and fluidity, teams simply can’t rely on guesswork and second-hand interpretations. To succeed, teams need to embrace data-driven product management. Data Driven Product Management. There are many ways in which data-driven product management is described but, put simply, data-driven product management means making decisions based on real-world information. Understanding data-driven product management can help you to use the right data, uncover the right insights, and ultimately build the right product.
- Data Driven Product Management Tools
- Data Driven Product Management System
- Data Driven Decision Making
Source:-infoq.com
Key Takeaways
Stating Product Conjectures or Hypotheses in plain English is a common practice in Product Management. In order to make data-driven decisions using Hypotheses, they need to be formulated in a way that lends itself to automated evaluation.
Hypotheses can be formulated semi-formally using Capability / Outcome / Measurable Signal triples, which allows for a good mix of qualitative and quantitative specifications.
Automated evaluation of Hypotheses requires development capacity to be planned for software developers as part of feature implementation.
Automated evaluation of Hypotheses also requires software developers to possess Operations skills and apply them during feature implementation.
Documentation of Hypotheses needs to be done directly in the tool where user stories are specified.
Data-Driven Decision Making Series
Stating Product Conjectures or Hypotheses in plain English is a common practice in Product Management. In order to make data-driven decisions using Hypotheses, they need to be formulated in a way that lends itself to automated evaluation.
Hypotheses can be formulated semi-formally using Capability / Outcome / Measurable Signal triples, which allows for a good mix of qualitative and quantitative specifications.
Automated evaluation of Hypotheses requires development capacity to be planned for software developers as part of feature implementation.
Automated evaluation of Hypotheses also requires software developers to possess Operations skills and apply them during feature implementation.
Documentation of Hypotheses needs to be done directly in the tool where user stories are specified.
Data-Driven Decision Making Series
X plane 737. The Data-Driven Decision Making Series provides an overview of how the three main activities in the software delivery – Product Management, Development and Operations – can be supported by data-driven decision making.
It consists of several articles, each highlighting an area where data-driven decision making can be applied:
In Product Management, Hypotheses can be used to steer the effectiveness of product decisions.
In Development, Continuous Delivery Indicators can be used to steer the efficiency of the development process.
In Operations, SRE’s SLIs and SLOs can be used to steer the reliability of services in production.
In the Series we also show how applying Hypotheses, CD Indicators and SRE’s SLIs / SLOs at the same time enables the software delivery organization to optimize for effectiveness, efficiency and service reliability in parallel.
In Development, Continuous Delivery Indicators can be used to steer the efficiency of the development process.
In Operations, SRE’s SLIs and SLOs can be used to steer the reliability of services in production.
In the Series we also show how applying Hypotheses, CD Indicators and SRE’s SLIs / SLOs at the same time enables the software delivery organization to optimize for effectiveness, efficiency and service reliability in parallel.
All the articles in the Series are listed on Vladyslav Ukis’s InfoQ profile.
Introduction
Software product delivery organizations deliver complex software systems on an evermore frequent basis. The main activities involved in the software delivery are Product Management, Development and Operations (by this we really mean activities as opposed to separate siloed departments that we do not recommend). In each of the activities many decisions have to be made fast to advance the delivery. In Product Management, the decisions are about feature prioritization. In Development, it is about the efficiency of the development process. And in Operations, it is about reliability.
Software product delivery organizations deliver complex software systems on an evermore frequent basis. The main activities involved in the software delivery are Product Management, Development and Operations (by this we really mean activities as opposed to separate siloed departments that we do not recommend). In each of the activities many decisions have to be made fast to advance the delivery. In Product Management, the decisions are about feature prioritization. In Development, it is about the efficiency of the development process. And in Operations, it is about reliability.
RELATED VENDOR CONTENT
Mastering Remote Meetings – Download the InfoQ eMag
Free Product Owner Learning Path
Free Scrum Master Learning Path
Conversation-Driven Development: Introduction to ChatOps
Digital Transformation Game Plan – Download Now (By O’Reilly)
RELATED SPONSOR
Mastering Remote Meetings – Download the InfoQ eMag
Free Product Owner Learning Path
Free Scrum Master Learning Path
Conversation-Driven Development: Introduction to ChatOps
Digital Transformation Game Plan – Download Now (By O’Reilly)
RELATED SPONSOR
LaunchDarkly Feature Management Platform. Dynamically control the availability of application features to your users. Start Free Trial.
The decisions can be made based on the experience of the team members. Additionally, the decisions can be made based on data. This should lead to a more objective and transparent decision making process. Especially with the increasing speed of the delivery and the growing number of delivery teams, an organization’s ability to be transparent is an important means for everyone’s continuous alignment without time-consuming synchronization meetings.
In this article, we explore how the activities of Product Management can be supported by data from Hypotheses and how the data can be used for rapid data-driven decision making. This, in turn, leads to increased transparency and decreased politicization of the product delivery organization, ultimately supporting better business results such as user engagement with the software and accrued revenue.
We report on the application of Hypotheses in Product Management at Siemens Healthineers in a large-scale distributed software delivery organization consisting of 16 software delivery teams located in thee countries.
Process Indicators, Not People KPIs
In order to steer Product Management in a data-driven way, we need to have a way of expressing the main activities in Product Management using data. That data needs to be treated as Process Indicators of what is going on, rather than as People Key Performance Indicators (KPIs) used for people evaluation. This is important because if used for people evaluation, the people may be inclined to tweak the data to be evaluated in favorable terms.
In order to steer Product Management in a data-driven way, we need to have a way of expressing the main activities in Product Management using data. That data needs to be treated as Process Indicators of what is going on, rather than as People Key Performance Indicators (KPIs) used for people evaluation. This is important because if used for people evaluation, the people may be inclined to tweak the data to be evaluated in favorable terms.
It is important that this approach to the data being treated as Process Indicators instead of people evaluation KPIs be set by the leadership of the product delivery organization in order to achieve unskewed data quality and data evaluation.
Hypotheses
One of the central questions in Product Management is “what to build?” This is very important to get right as building software features that, once ready, do not get used by the customers is a total waste.
One of the central questions in Product Management is “what to build?” This is very important to get right as building software features that, once ready, do not get used by the customers is a total waste.
In order to approach the question of “what to build?” product delivery teams run small experiments to explore the customer needs, ideally in Production. Each experiment needs associated measurements that are used to either confirm or disprove initial assumptions.
This process is the subject of the Hypothesis Driven Development (HDD). It is well-described in How to Implement Hypothesis-Driven Development. In essence, an experiment is called Hypothesis in HDD and is described using a <Capability> / <Outcome> / <Measurable Signal> notation:
Hypothesis:
We believe that this <Capability>
Will result in this customer <Outcome>
We will know we have succeeded when we see this <Measurable Signal> in production
We believe that this <Capability>
Will result in this customer <Outcome>
We will know we have succeeded when we see this <Measurable Signal> in production
The definition of Hypothesis for a feature is done before the feature implementation begins. A product delivery team declares which <Capability> they want to put into the product to achieve a specific customer <Outcome>. The customer <Outcome> becomes evident when a defined <Measurable Signal> becomes visible in production.
With that the product delivery teams define upfront how they envision the features being used by customers in production. During the feature development, the teams additionally implement <Measurable Signals> to create instrumentation necessary to see how the features are being used in production. Finally, once the features have been deployed to production, the teams evaluate the actual <Measurable Signals> to understand whether the Hypothesis turned out to be true of false (Build → Measure → Learn).
After that, the process is repeated; based on the results of the first Hypothesis, the second one is formulated, implemented and measured, and so forth.
In our experience, the entire process also moves a product delivery team from being “just a feature factory” towards becoming responsible for delivering software that is actually being used by customers in production.
That is, the focus of the product delivery teams is set on the value they provide to the customers, as opposed to just counting features delivered to production.
A team that consistently works with Hypotheses navigates the customer problem domain effectively by optimizing for customer usage of the software based on data from production (automated feedback loop; every release is a scientific experiment). A team that does not work with Hypotheses just produces features and does not measure the feature usage by customers in production.
From Projects to Products
Today, many organizations transition from projects to products. Introducing Hypotheses can support the transition. This is because in projects, the requirement engineering is scoped for Development, where the project often ends, thus excluding Release and Operations aspects.
Today, many organizations transition from projects to products. Introducing Hypotheses can support the transition. This is because in projects, the requirement engineering is scoped for Development, where the project often ends, thus excluding Release and Operations aspects.
Stating a Hypothesis with a <Measurable Signal> as part of the requirement engineering process necessarily includes Release and Operations aspects (Dev + Ops = DevOps, so to speak). This way, the requirement engineering includes the evidence of user behavior, which is a step towards products and away from projects.
Additional Hypotheses-related aspects that help transit from projects to products are:
As the Hypotheses cover business success criteria to be achieved, the development team is becoming more business-driven without the need of a project.
The development team can run with less management involvement under the guidance of <Outcome> being measured using <Measurable Signals>. The team can do many releases on their own, which used to require separate projects.
The current state of the team is transparent with the values of <Measurable Signals>, which also guide the team decision making on the way to the <Outcome>.
Enablement
With the Indicators Framework defined, it was clear to us that its introduction to the organization of 16 dev teams could only be effective if sufficient support could be provided to the teams.
The development team can run with less management involvement under the guidance of <Outcome> being measured using <Measurable Signals>. The team can do many releases on their own, which used to require separate projects.
The current state of the team is transparent with the values of <Measurable Signals>, which also guide the team decision making on the way to the <Outcome>.
Enablement
With the Indicators Framework defined, it was clear to us that its introduction to the organization of 16 dev teams could only be effective if sufficient support could be provided to the teams.
For the definition of Hypotheses we expanded our Business Feature template to include the <Capability>, <Outcome> and <Measurable Signals> fields.
We ran small workshops of 30 – 60 minutes with the teams where we took one requirement the team was about to take up and turned it into a Hypothesis. Later, once the requirement was deployed to production, we met the team and evaluated the <Measurable Signals>.
Adoption
We introduced the suggested Indicators Framework to an organization of 16 dev teams working on “teamplay” – a global digital service from the healthcare domain (more about “teamplay” can be learned at Adopting Continuous Delivery at teamplay, Siemens Healthineers). The teams got quite interested in Hypotheses right from the start.
We introduced the suggested Indicators Framework to an organization of 16 dev teams working on “teamplay” – a global digital service from the healthcare domain (more about “teamplay” can be learned at Adopting Continuous Delivery at teamplay, Siemens Healthineers). The teams got quite interested in Hypotheses right from the start.
It was easy for the teams to grasp the Hypotheses definition process. The discussions during the feature Hypotheses definition proved to be very valuable for further requirement engineering (using BDD). It was possible to hammer out the scope of the features very early on. A Hypothesis definition at this point created a nicely defined boundary, in which detailed user stories can be created later on.
An example Hypothesis definition from one of teamplay User Administration is:
Capability Outcome Measurable Signal
Enable user invitations by hospital admins
Admin User: ability to onboard non-admin users with appropriate user rights
Enable user invitations by hospital admins
Admin User: ability to onboard non-admin users with appropriate user rights
Non-Admin User: ability to skip the registration process and access applications by just accepting an invitation from the hospital admin
Average users per hospital in 2020 > Average users per hospital in 2019 + 30%
> 10% of all hospitals registered before 31.12.2019 used the user invitation feature at least once in 2020
> 50% of all hospitals registered after 01.01.2020 used the user invitation feature at least once in 2020
Time between the hospital admin sent the invitation and the user accepted the invitation is < 1 week
When implemented, the first Measurable Signal values showed that it will take time until the usage of the feature increases to the point that was specified in the Hypothesis. However, there was qualitative feedback from Sales that customer onboarding using user invitations became significantly easier (unexpected qualitative Measurable Signal). Based on that feedback, the team can see over time whether the Hypothesis definition should be changed to reflect the usefulness of the feature to the customers as reported by the customers.
> 10% of all hospitals registered before 31.12.2019 used the user invitation feature at least once in 2020
> 50% of all hospitals registered after 01.01.2020 used the user invitation feature at least once in 2020
Time between the hospital admin sent the invitation and the user accepted the invitation is < 1 week
When implemented, the first Measurable Signal values showed that it will take time until the usage of the feature increases to the point that was specified in the Hypothesis. However, there was qualitative feedback from Sales that customer onboarding using user invitations became significantly easier (unexpected qualitative Measurable Signal). Based on that feedback, the team can see over time whether the Hypothesis definition should be changed to reflect the usefulness of the feature to the customers as reported by the customers.
Another example Hypothesis from teamplay Data Access Management is:
Capability Outcome Measurable Signal
Data access by hospital location and department
Admin User: ability to enable users to see data appropriate to their scope of work
Data access by hospital location and department
Admin User: ability to enable users to see data appropriate to their scope of work
Non-Admin User: default focus on data from my hospital department to streamline my work
C-Level User: compare KPIs of different hospital departments
> 50% of hospital networks enabled Data Access Management
# of non-admin users in hospital networks having access to all data > 0
The implementation of measurable signals varied by team. A few teams implemented the measurable signals in production. With the planned increase of the frequency of team releases, we think that the teams will increasingly adopt the implementation of the measurable signals.
# of non-admin users in hospital networks having access to all data > 0
The implementation of measurable signals varied by team. A few teams implemented the measurable signals in production. With the planned increase of the frequency of team releases, we think that the teams will increasingly adopt the implementation of the measurable signals.
One of the teams had an interesting experience with Hypotheses. After a Hypothesis definition for a feature, the team started implementation. During the implementation, they encountered significant limitations in an external framework used. The limitations were so severe that it became clear that the <Capability> could not be implemented as intended and, therefore, the <Outcome> could not be achieved. The team started looking for another external framework.
Soon, we will be introducing a “teamplay Service Standard” – a short list of topics important to us in a service. Two topics there, will concern Hypotheses: “Establish data-driven prioritization for a service” and “Define what success looks like and iterate towards it”. With that, we are going to give the Hypotheses an additional push in the organization.
Prioritization
Our teams need more experience with Hypotheses in order to consistently use the data at hand as an input for prioritization. The data comes in different forms:
Our teams need more experience with Hypotheses in order to consistently use the data at hand as an input for prioritization. The data comes in different forms:
Positively / negatively tested Hypotheses
Unexpected insights from Measurable Signals
Now that the data is available, it needs to be taken into account by the dev teams, and especially product owners, to make the best prioritization decisions. The prioritization trade-offs are:
Unexpected insights from Measurable Signals
Now that the data is available, it needs to be taken into account by the dev teams, and especially product owners, to make the best prioritization decisions. The prioritization trade-offs are:
Invest in features to increase product effectiveness and / or
Invest in development efficiency and / or
Invest in service reliability
Summary
In summary, if a team optimizes their Product Management Process using Hypotheses, then the team is able to gradually optimize their ways of working in a data-driven way so that over time the team can achieve a state where they build features evidently being used by the users.
Invest in development efficiency and / or
Invest in service reliability
Summary
In summary, if a team optimizes their Product Management Process using Hypotheses, then the team is able to gradually optimize their ways of working in a data-driven way so that over time the team can achieve a state where they build features evidently being used by the users.
Hypotheses help depoliticize and enable transparency in the decision making process of the software delivery organization. Finally, it supports the organization in driving better business results, such as user engagement with the software and revenue.
This article is part of the Data-Driven Decision Making for Software Product Delivery Organizations Series. The Series provides an overview of how the three main activities in the software delivery – Product Management, Development and Operations – can be supported by data-driven decision making. Future articles will shed light on data-driven decision making in Development, Operations and combinations of data-driven decision making in Product Management, Development and Operations.
Data Driven Product Management is an approach to business governance which values decisions and data that can be verified. The approach is gaining popularity within the enterprise as the amount of available data increases in tandem. Data Driven Product Management is normally undertaken in order to acquire competitive advantage.
One of the benefits of data driven product management is that it is used to quantify full economic impact. In fact, data driven product management has driven the impact across various sectors of economy.
Recent studies have revealed that the product management has impacted the job creation and IT. The main core of data management is to save money, increase efficiency and the precious resources which have benefited all the sectors of the economy as well as improving the quality of life. From safety to security, the outcomes of product management have benefited all sectors of the economy.
Another benefit of data driven management is that it ensures that the accounting is done properly. In addition to that, it will enable the data to be done consistently. When you have data which is properly organized, you will get the ability to analyze and apply the third party insights as well as creating an optimal scenario to make high quality data driven management decisions.
Data Product Management Analytics
Analytics is a crucial area which the product managers have to understand. However, it can be intimidating when you are getting started on the product management basics. There are some basics which you need to consider when covering the role of product management in an organization. The following are some of the steps which you need to follow while doing product management analytics:
- Tracking of the email signature clicks
Under analytics, the initial step is to start tracking your email click signature. This is an example of how to introduce the basics. In addition to that, include some unique identifiers. This will ensure that a user has arrived on the site after clicking the exact site.
- Viewing of the clicks
With Google analytics, you will be able to see the number of times which the signature Gmail has been viewed. This is because of the querystring parameter ref which is not used by the developers.
- More superior
By measuring more than 10 clicks, you will require the ability to group and filter certain kinds of referrals. The Google analytics assists you in construction of more complex tracking links with the URL Builder. This will enable you to create separate campaigns while starting to collect data in a systematic way.
- Forward
The Data Driven Management Certification Programs
Data driven management programs teaches the essential methods of management. The first two weeks of the programs covers the following: application of statistical methods of measuring and predicting of consumer preference, development of perceptual maps used for positioning of products which is based on the perceptions of the customers. Other courses which are covered by the certification program includes: use of the management response model in order to predict demand as well as development of the right marketing mix.
The certification is a foundation which is used for harnessing the power of data, sales, margins and marketing share.
Who is eligible for enrollment of data driven management programs?
The certification is design for the marketing managers. Nevertheless, professionals who interact regularly with many customers or product data are also eligible to apply. Another requirement is proficiency in Microsoft excel.
Data driven management accreditation
People who have successfully completed the certification are accredited with data driven management certificate from the graduate school of management.
Keys required in building a data driven management
Analytics and big data have been promising to transform the way in which companies does. The data driven management is used for controlling the performance gains.
Most of the companies have found out that the data driven management requires capabilities which are supportive. With data driven management, companies should be able to identify, manage and combine multiple data sources of data. The following are three keys which should be adhered to while building a data driven management strategy:
Choosing of the right data
Over the past few years, the modeling and universe of data has drastically changed. Hence, the volume of information is rapidly growing. On the other hand, the opportunities for expanding insights have been growing rapidly. As a result, better and bigger data have been giving companies the granular and panoramic views of their businesses. Companies will improve their operations, customer’s experience and strategy.
Data Driven Product Management Tools
One of the strategies which the management should have so as to choose the right data is to have source data creativity. In order to tackle the business problems, managers should be aware of how to use these decisions in order to encourage a more comprehensive look on the data. IT support is also another thing which is necessary for the data storage, sourcing and analysis. The business leaders will therefore address the big data requirements for synchronization and matching of the overlapping data.
Building of model which optimize and predict the outcomes of the business
The data is very important. However, the improvement performance and the competitive advantage arise from the models analytics which enables managers to optimize and predict the customers.
Transformation of the capabilities of the company
This is a data driven management strategy which solves the problems which arise because of the mismatch between the capabilities and culture as well as the emerging tactics which can exploit the analytics successfully.
Model designers should therefore understand the type of business decisions which managers make so as to make their actions align with the goals of the company.
Data Driven Product Management System
Manages also requires transparent methods which can use analytics and big data. Apart from that, sophisticated modeling is needed so that it can be used for sharpening of the risk management and operations.
Conclusion
Data Driven Decision Making
The above, is a highlight of management including the major benefits of the services provided. Hence, you can use the service if you run a business firm. Therefore, you can get data driven management and you will enjoy the service provided.