Menu

What if your Data Management Application had an actual Data Layer?

hqStore is the Datastore for Data Management Applications, providing access to ALL your data, at ANY point in history, across changes in application logic or even platform, right from a web browser or from directly within your code.

hqStore: A Universal Datastore

Tell me more!

hqStore is the Datastore for Data Management Applications, providing access to ALL your data, at ANY point in history, across changes in application logic or even platform, right from a web browser or from directly within your code.

With hqStore you can…

  • Execute hqStore and rapidly capture the schema and contents of any Table or View.
  • Process hqStore asynchronously to apply compression and expose new data to query.
  • Access hqStore data directly from within your own procedures, functions, and queries.
  • Query hqStore seamlessly across schema changes.
  • Invoke hqStore to generate database tables and populate them with historical data.
  • Navigate hqStore to explore the history of your data via a simple, flexible user interface.

The Program vs. the Datastore

In an Application there is the Program, and there is the Datastore.

The Program processes data, while the Datastore contains it. Objects in the Program change frequently in form and content, while objects in the Datastore have stable forms and change mostly in content. The Datastore is the stable foundation on which the Program is built. In the Program you can change one thing. Changing the Datastore changes everything.

In most modern Applications, the Datastore is a database. The Program rarely intrudes very far into the database, so it is tempting to believe that when one has constructed a database, the Datastore is complete. Mostly this is true.

But Data Management Applications require the rapid processing of data at industrial scales. The best tool for that job is a database, so in Data Management Applications the Program extends deeply into the database. And since part of the Data Management problem is to accommodate frequent changes in data requirements, much of the database changes frequently in shape as well as content. No part is stable in form.

No part is the Datastore.

The Database is not the Datastore

In database design there is a trade-off: designs that process data efficiently contain many internal dependencies, meaning a change in one place requires changes in lots of other places as well. Data Management Applications are designed to process data rapidly at industrial scales, and their database designs generally reflect that.  Without a proper Datastore, user interfaces and other non-pipeline components are compelled to depend on the same data objects. When one thing changes, everything breaks.

A well-crafted Datastore serves as a stable foundation to meet most data access needs, decoupling the shape of the data stream from the machinery that interacts with it, and eliminating much of the effort required to accommodate inevitable changes to the data.

Another trade-off: database designs that process data efficiently store it inefficiently. Maintaining a 90-day rolling history of data processed daily on this kind of system requires somewhat more than ninety times the database capacity needed to process a single day’s worth of data. An Application that performs well when new can easily experience performance problems and present maintenance and logistical challenges when its database grows by two orders of magnitude, only three months after going live.

A well-designed Datastore will often meet the same requirement with a database one-twentieth the size, which is more manageable by far.

hqStore is the Datastore for Data Management applications.