Getting Real-Time data into Rockset is easy. Querying that data is just as easy. But finding a Visualization or BI tool that allows for a Real-Time view of that data is not as easy as it should be.
Most visualization tools were created and optimized to take in large amounts of data and give powerful manipulations on that locally stored data (i.e. run a query to download 1 TB of data, then manipulate it locally). This can be useful for many BI requirements, but is not conducive to decisions requiring fresh data. To solve the Real-Time visualization problem, a tool will need to have both a Real-Time data source that execute queries quickly and also a way to auto-refresh the data at a low interval rate (<10 seconds).
Rockset has integrations with multiple visualization tools, such as Tableau and Superset, allowing users to leverage Rockset for real-time visualizations. Another great visualization tool that Rockset connects with is Grafana (you can look at this doc to learn how to use the Rockset Plugin), allowing companies that already use the tool for their monitoring to also use it for data analysis. One important point to note is that the current implementation of the Rockset Plugin for Grafana is that it was originally built for a more time-series oriented use case. This means that it currently looks for at least one time-oriented field in the query, and then leveraging one or two other fields for a line-graph visualization.
For those customers who are wanting to see data in other visualization types, or even a non-time-oriented graphing requirement (or even those using Grafana Cloud), the current plugin does not allow for these requirements (although, we are improving on the plugin consistently). But that does not mean that you are not able to use Rockset and Grafana at all. This post will discuss the ways to get Rockset and Grafana connected, whether that is a direct query or a leveraging a Query Lambda, regardless of the visualization type or use case.
To start this connection, make sure that you have either a query or Query Lambda built out in Rockset that delivers the desired data set (if you need to do any field aliasing a.field AS Field, that can be done inside of Grafana instead). Then, you’ll need to go to your Grafana instance and download/enable the JSON API plugin. Once that plugin is ready for you to use, you’ll need to configure it to use the correct Rockset API endpoint based on your Virtual Instance Region:
- Oregon = https://api.rs2.usw2.rockset.com (https://api.rs2.usw2.rockset.com/)
- N. Viriginia = https://api.use1a1.rockset.com (https://api.use1a1.rockset.com/)
You’ll also need to create an API Key in Rockset that has the correct permissions to access the data in the desired collections and workspaces. These will be used in the following plugin configurations:
- Name: Rockset
- URL
- Auth: None
- Custom HTTP Headers
- Header: Content-Type
- Value: application/json
- Header: Authorization
- Value: ApiKey ROCKSET_API_KEY
Once you click Save & Test, you should see a Success message with a green check mark. If you don’t make sure that the HTTP URL is pointing to the correct Rockset API Endpoint, your API key has sufficient privileges, and that your Grafana instance can access the required API Endpoint on the Rockset side. With that connected, you’ll want to start building out your visualization dashboard (we will use the Table Visualization Type for this).
In the Table visualization type, you’ll need to select the JSON API Plugin configuration that you just created:
And then, in the field section, make sure that you have the following setup per field:
-
Body
- Syntax Highlighting: JSON
- Add the desired payload in the Body section
-
Path
- Type: POST
- Value: /v1/orgs/self/ws//lambdas//tags/latest
-
Fields (add a new field for each column in the table)
- Field: `$.results[*].<FIELD_NAME>
- JSON_Type: JSONPath
- Type: Auto
- Alias: User defined column name in Grafana
For each field in the Table, you’ll you want to clone the previous field and point to the next FIELD NAME in the Fields section.
With that, you’ll be able to see all of the desired columns and relevant data from Rockset in Grafana. Additionally, following these steps will also allow you to leverage any of the native Grafana Visualization types with your Rockset data, in Real Time!
- BryFi