With approximately 19,000 public APIs and 172 being added to the list every month, the dependency and prominence of the programming interfaces are dramatically increasing. In my previous post, “API usage in networks is proliferating; are these interfaces ready for the unexpected?”, I discussed the role the APIs are playing in machine-to-machine communication and the way they are changing the shape of modern networks. With so much dependency, the reliability of these interfaces is vital to the functioning of the majority of digital businesses and any failures can cause huge losses both financially and by reputation. We have to continuously ensure the reliability of these programming interfaces and make them ready for any unforeseen circumstances. All the providers do test their implementations before deploying them, won’t this be enough to judge the reliability of these interfaces? The range of issues we see post the deployment provides us the answer. In this post, let’s take a look at where the current validation process is going wrong and discuss the best practices of validating the Application Programming Interfaces. This process will help prevent interoperability and functionality issues in the field and ensure the reliability of the interfaces.

An API has three stages in its development cycle, where the first two stages, specification development and implementation, are performed by the same organization or two different organizations and the third stage is implemented by a consumer who develops applications to integrate with the provider. Common-understanding of the API functionality among these organizations is vital for a successful outcome. Deviation of the implementations from the API specifications will result in interoperability issues and they could go undiscovered until the integrated applications raise an alarm in the production environment. One another concern is the incomplete validation of API functionality. Today, we validate the functionality of an API by executing a fixed number of test scenarios that are the most prominent sunny day scenarios and generally verified by the test engineers. How about the unexpected boundary scenarios, which are less anticipated by the testers and may have a high impact on production systems?

Given the wide usage of APIs and the wide variety of possible requests, a limited number of prominent test scenarios do not prove to be sufficient to judge the reliability of an API. Validating the implementations against all possible combinations of input parameters, leaving no stones unturned, with various supported types and ranges, and analyzing their behavior against the specification will provide us answers to the API reliability. But, performing this manually is daunting and costly. Such an elaborate verification will require an automated framework to try various combinations of values the API can take. This will also help us validate the security issues that could result in data theft.

We provide a framework for automatic validation of APIs in our publication, “Measure Your API Complexity And Reliability“. This uses the API specification to automatically generate test scenarios that exhaustively validate the functionality and specification conformity of the interfaces. The framework is intelligent enough to generate and execute the tests, validate them, analyze the failed cases and provide possible reasons for the failure of a test case.

“Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.” — H. James Harrington 

Improving the reliability of any application is a continuous process, but how can we ensure that we are doing better at every stage? Metrics to measure the reliability will help us establish that. There does not exist any publicly available metrics to measure the reliability of APIs. Using the equations proposed in our publication, “Measure Your API Complexity And Reliability“, one can measure the complexity and reliability of the APIs. The complexity metric will help in designing consumer-friendly specifications and the reliability metric will help us understand and improve the reliability of API implementations.

We used the proposed framework to validate the functionality and specification conformity of existing and widely known collaboration cloud APIs and found interesting results. We also measured the reliability of these cloud interfaces using the proposed metrics. In the next article, I will discuss the results and the reliability of these interfaces in detail.