API KPIs Part 2: The API Product Management KPIs Worksheet
API Product Management KPIs Worksheet: Part 2
This is a continuation from last month’s blog post. Check out part 1 to hear our take on the Business Success Criteria Lens, which applies to all other filters. Last month we also covered the first of three categories under the API Product Catalog lens, which are:
- Public APIs (Covered in Part 1)
- Internal APIs
- Third-Party APIs
KPIs for Internal APIs (Business Capability APIs)
The internal APIs we (at digitalML) call “Business Capability Building Blocks” are also known as Capability APIs, Business Capability APIs, or internal APIs. (For more on these, see Developing with Business Capability APIs).
The lens changes here because of the nature of a capability API’s purpose — these APIs are more about operationality, so any monetization metrics you’re after will contribute more to cost savings than revenue growth.
Capability API KPIs (yes, that’s a mouthful!) will be an important measure for your innovation efforts, as the more coverage you have in your catalog of capability APIs, the more pre-packaged options you have to choose from.
Meaning: higher visibility, more ideas, and quicker launch.
This layer of internal APIs is every bit as important, if not more important, as the other layers combined — and the sheer amount of reuse and discoverability it lends is no small savings.
Growth & Expansion Metrics for internal APIs:
- % of total LoBs represented as internal APIs and % to go.
- # launches per year.
- $ revenue generated by internal APIs.
- % of reuse or extended.
- # users & usage
- LoB with the highest % increase of usage. (This can be helpful to stay on top of spikes.)
- # opportunities created per LoB.
- % publishable as is.
Cost Savings & Operability Metrics for internal APIs:
- Incremental/marginal analysis for each LoB represented. (A highly over-looked route for cost savings)
- % Capability API Availability — this can be measured by tracking capability wishlist requests.
- Number of APIs in and time spent on each stage of a new API Product — Plan, Design, Build, Run.
- Time to launch for a needed internal API. (To calculate employee hours, an interesting metric to stack against increased productivity — Harvard Business Review has a great calculator here.)
- # of launches per year.
- # of microservces refactored from monolith.
Special Use Case: Migration Efforts
- If you’re transitioning from one runtime environment to another, coverage running in one environment versus another and state-to-goal are helpful statistics.
Customer Success Metrics for internal APIs (Note: internal APIs are typically used by internal customers or wrapped into partner APIs, but if you’re really building great internal APIs, they can be published as external APIs.):
- % internal services exposed and relative to customers.
- How many new customers use your services (per capability API)?
- $ direct and indirect revenue associated? (This one gives each LoB a nice business case for ROI.)
- How quickly can the customer access your data? (Again, SLA documentation that allows developers/DevOps teams to stay on top of throttling can help you stay on top of load times).
- #s of bugs & throttling issues.
- How quickly are customers able to onboard your capability API?
- % capabilities publishable as is.
KPIs for Third-Party APIs
Imagine discovering that no one has been managing a third-party API that charges your organization per call. Say it’s a certain maps/location-based third-party API. Your team wrapped it into your Location-Based Loyalties API Product, which backs an MVP digital experience. You sent it out into the world, thinking will check back in 3 months, onto the next project for now. Uh-oh.
In the first month, user activity is on a steady incline, but nothing your teams can’t handle — that much you prepared for. But those 3rd party APIs though. In the second month, you get some unexpected press and it sends a ton of traffic your way… no big deal. But when it comes time to pay the bill, you owe one of those 3rd party APIs tens of thousands of dollars. More than you made from the digital experience in the first place.
This is where third-party API management comes in handy.
Growth & Expansion KPIs for managing third-party APIs:
- ROI associated with each third-party API.
- Users accessed through third-party APIs.
- % usage externally.
- New channels reached.
- New markets presented through third-party APIs integration.
- New opportunities created through third-party APIs.
Cost Savings & Operability Metrics for third-party APIs:
- % ownership — who manages your third-party APIs? (To avoid surprise bills, this is highly encouraged.)
- % of budget used.
- $ internal business process coverage.
- $ per call cost compared to market competitors. — should you choose to switch to a costlier oprtion.
- % API Products supported by third-party APIs.
- # external capabilities covered by 3rd party APIs.
Customer Success Metrics for third-party APIs:
- % of throttling usage.
- Time to access data (could be related to throttling).
- # users, % usage.
- # of channels and opportunities opened through third-party APIs.
- Quality — # of bugs.
- Ease of usage/consumption for customers.
The Stakeholder Lens
As an API Product Manager, you might be reporting to a number of stakeholders; one category might be digital executives — anyone in the c-suite who knows a bit about digital but is predominantly more business-minded. Externally, maybe you’re engaging with an opportunity or onboarding a partner. Internally, maybe you’re reporting on high-level growth of a product or catalog.
You might be helping internal digital teams realize a product. They’re likely a bit more technical than execs (excluding the CIO), and their focus will likely be more on customer and implementation.
Or maybe you’re working with development teams to gather requirements or make iterations. Developer teams, though predominantly technical, are increasingly crossing over into business territory as modern organizations attempt to straddle a bimodal approach to their IT model.
Again, every role varies. For companies in early stages of their API Catalog, one API Product Manager might be assigned to report on the entire portfolio. At more mature companies, API Product Managers are several; one API Product Manager might be assigned per Line of Business.
This changes the scope of a position in a big way, but even if the roles differ we can likely categorize the metrics important to them based on their focus.
Digital Executives —
A stakeholder might be considering API Products to bring speed and quality to your company’s digital experiences, or perhaps they’re building Open APIs to grow new opportunities. In a third use case, maybe they’re looking to partner with your company to wrap up a few of your internal APIs into a whitelabeled Product API.
Whether external or internal, an executive role is pressed for time and likely looking for a high-level overview. Your interaction as an API Product Manager is likely going to be to present a business case for the APIs or Catalog you manage.
For this reason, the digital executive box is under the Growth & Opportunity Column of our API Product Manager KPI Worksheet, as growth & opportunity KPIs present the most tangible value. However, you know your use case better than we do, so you can certainly pull from the other columns to find the right match — particularly the cost saving metrics from the middle Operability column, and any time-to-market or # of user metrics from the Customer-Centricty column should prove helpful. A few examples:
- Time to market to realize a digital experience from idea to delivery.
- $ Total Economic Impact of internal APIs.
- # opportunities created via API Program.
Development Teams — (developer producers, developer consumers, tech leads, developer evangelists)
Again, these interactions might come from external or internal consumers. As these stakeholders are more technical and working directly with the goods you manage, we’ve placed it under the Operabilitycolumn of the worksheet. In truth, there’s a lot there that might interest them, if you have the right set up.
You might find yourself pulling from the Customer-Centricity column for many useful development KPIs as well — Developers are often your customers, after all. They will love your APIs if your documentation is up to snuff; if you’re keeping them informed on the why behind the what. It seems obvious, but you’d be surprised how many APIs fail to provide even the most basic information. How you design, categorize, and map your APIs to your business model can make onboarding a huge time saver (and thereby cost-saver) for everyone involved. A few metrics of interest:
- How many others are using your APIs?
- What is the throttling limit?
- How long does it take to onboard?
- How long does it take to complete a query?
- How many endpoints can it service at one time (if there’s a limit)?
- How often are updates?
Digital Teams — (CX/Innovation/Digital Product/Program Managers) This is more often an internal role than external.
Lastly, the role we loosely label “digital teams” is a stakeholder you’ll often work with as an API Product Manager. This includes, but is not limited to, any role charged to enable digital business and bring the customer journey to life.
The interesting thing about this group is that, as development teams are increasingly moving closer to the business, with any success, these traditionally business roles are increasingly becoming more technical — to the effect that we are seeing, in companies at greater stages of digital maturity, digital roles self-serving on their own development; citizen developers.
More typical is the digital product owner or digital program owner, who has a long backlog of patents and customer experiences to launch. This role is often looking to bring the brand’s best to customers, whereever they are, with speed and ease. And so we have this box under the Customer-Centricity column — though arguably this stakeholder might make use of most of the KPIs on this worksheet. A few examples:
- Which department is seeing the most use by our customers?
- How far along are we on the way to complete the customer journeys we’ve identified?
- What is our time-to-market for a business capability API? API Product?