By Bill Bates and Selma Pittman
In today’s health landscape, more data is available than ever before, and stakeholders are still working to find the best ways to access, ingest, and analyze this data. Until recently, healthcare data was only available within the four walls of the hospital, showing an episodic view of a patient’s health and leaving their actions outside of the doctor’s office largely unaccounted for. Now, we have access to health data from consumers around the world, and it’s crucial that this data is available to healthcare providers and other stakeholders as soon as possible.
Validic™ Inform’s new streaming capability allows for the integration of continuous, near real-time patient data from home health devices. This model offers the streaming capability of high-volume data platforms like Twitter or Salesforce, but formulated to offer the most effective, efficient access to patient data. In this post, the second in the series, Validic’s VP of Engineering Bill Bates and Head of Product Selma Pittman are sharing their insight on exactly what that means for the industry and the future of healthcare.
In our last blog post, we shared some insight into the functions of an API and some other key terms and tools in order to understand exactly how Inform works. Today, we’re discussing exactly what a Streaming API means for you and your end user. A Streaming API completely changes the way you ingest and therefore analyze and understand patient data.
Streaming API
This type of API is a method of streaming data directly to Validic customers. Basically, a Validic customer opens a connection to Validic and we push data records to the customer using Server-sent events. A Server-sent event (SSE) is a “technology where a browser receives automatic updates from a server via HTTP connection.” As soon as an event is received by Validic, we de-identify data, transform it to a standardized format and push this data directly to our customers. This method provides near-real time data to our customers, without the overhead of a traditional RESTful API. The data delivered over this API is in the same JSON format as the RESTful API, making it easy to move between the RESTful and Streaming API.
Advantages To Streaming
So why learn about and use the Streaming API when developers already have a firm grasp of the REST API?
Immutable Data
With a REST API, the data delivered is based on the state of that data at the time of the REST call. If one wanted to know how many steps a given end-user took by 8, 9, and 10 a.m. on a given day, he or she would have to make three separate calls to the REST API to know that. If they only make one call at 10 a.m., they’ll only know the value as of that time. Using the Streaming API, without having to initiate calls at those respective times, Validic customers connect to the stream and any values present are sent to the customer across the stream. In this case, even if they connect for the first time at 10 a.m., they’d receive a summary from each of those hours. No value was overwritten with the next.
No Push and Pull
In Validic’s original REST API, we were able to notify customers that new data was present for them to obtain using Push Notifications. This allowed customers to only grab data from Validic when it was new and available, instead of polling every so often. However, this required that Validic know something about the customer’s environment, namely where to send these notifications. If the customer made a change to their environment, this system could break, leaving customers unaware that new data was available. Troubleshooting a system like this can be difficult given the complexity. With a Streaming API, customers can receive data through a stream, without having to wait for Push Notifications.
Reduced Data Latency
Data enters the stream when available, which means if the customer is connected to the stream, the data is available to them almost immediately. Customers are not polling for the data, which inherently means creating a sleep between polling requests. Even customers using Push Notifications have to manage the latency between the time they receive a Push and the time they obtain the data. The Streaming API allows Validic customers to access their data in near real-time, without having to poll for the data or wait for a Push Notification.
Less Customer Coding
First, because the Streaming API uses the same JSON-formatted data model as the REST API, only one parser for this data has to be written. Second, because no controlling mechanism has to be written to manage the receipt of Push notifications, nor the management of polling for new data, communicating with an always-on and available data stream requires fewer “moving parts,” or code for the customer to create. Connection management is generally the trickiest part of SSE (our Streaming technology). However, there are many SSE libraries available for a number of different, common languages, so much of the work is already available in these open-source libraries. With REST APIs, customers have to write code to manage that interaction. With our Streaming API, customers can focus on how to utilize the data coming across the stream instead of how to interact with the stream.
When to Use the REST API
With these advantages of the Streaming API, why would a customer ever use the REST API? In a nutshell, the Streaming API is a firehose. If someone wants to know the latest set of events for a given user, they would have to create a means of interacting with all of the Streaming data. In cases where they use Validic as their storage for this data, having a means of querying for a given end-user’s records for a specific time range and by use of other search/filtering criteria is essential. One can query the REST API on the filters that suit their needs, and therefore receive only the data relevant to their request.
When to Use the Streaming API
While the advantages of the Stream API are clear, it’s important to understand the best use cases for this data. If customers want to store all of the data themselves, perhaps for use in an analytics service, access to all of the data is necessary. When running such an analysis of the data, it would be inefficient to enumerate requests over the entire set of end-users, one request per user. In addition, the REST API only has a limited amount of historical data. If a customer wants to run an analysis of data over a period of several years, they have to store that data themselves — getting the firehose of data from the Streaming API is far more efficient means for receiving it.
Conclusion
While the new Inform API provides beneficial technical advantages over our previous API, we feel there is a clear advantage giving developers the flexibility to craft the right integration solution for them. However, it’s important to recognize that a Streaming API for PGHD signals a shift in the industry to more efficient, effective integration of actionable data that can lead to more insights than ever before. To learn more about Validic Inform, click here.
This is the second post in a two-part series on Streaming APIs. Missed the first post? Learn more about APIs and our Validic technology here.