Industrial Data Architecture
Reopen Stored Evidence
Rehearse retrieval.
On this page
Reopening means a separate reader can reproduce the required answer.
Choose the answer before the store
A production supervisor trying to understand an interruption and a manager comparing operations are asking for different kinds of help. Treating both as a request for more storage hides the difference. Start with what the recipient needs to finish: explain a particular incident, decide where to investigate or prepare an operating review.
Historian, time-series database and lake are labels, not guarantees.
The business should choose which questions deserve a dependable answer before choosing how ambitious the service should become. A useful question has a recipient who can explain what they would do with the result. A technically impressive answer that arrives after that decision has been made may add little value.
- Immediate work
- What task is waiting for an explanation?
- Investigation
- What decision would further evidence change?
- Planning
- Who will act on a comparison?
Make disagreement about the purpose visible. An owner seeking fewer interruptions may prefer a different first assignment from an analyst seeking broader research access. Neither preference is a storage capability. Resolving that choice prevents a successful demonstration from being judged against a different, unstated business goal.
A useful sponsor can also decline a request. Information access deserves a purpose, rather than an assumption that every possible question needs a permanent service.
Separate recent access from repeatable history
Gorilla's 2015 design separated recent in-memory monitoring from persistent storage. It does not establish local retention or recoverability.
BTrDB returns stream versions (record states) for repetition. Lookup is separate; retained quality status needs its own evidence.
- Recent
- Which tier?
- Past
- Which range, version and lookup?
Distinguish history from recovery
Delta's paper and manual require retained data and log dependencies for past versions. Cleanup can prevent reconstruction.
- Current rows
- Readable now.
- Past version
- Retained dependencies.
- Recovery
- Separate reader reopens.
For the business, continuity is a separate purpose from everyday convenience. An owner may accept an occasional delay in a management review but need a definite answer about how work would continue after losing the usual service. Agree which consequence matters before accepting a broad assurance that the information is protected.
Do not ask an analyst to make that continuity commitment on behalf of everyone else. The people responsible for the affected work must decide what interruption they can tolerate. A technically narrow result can inform that decision without silently becoming an assurance about the whole organization.
Ask what the analytical reader actually reads
Dremel describes querying data in place, with nested fields stored in columns. A selected field can be read without reading unrelated fields. Its execution tree combines partial answers. This addresses broad analytical questions differently from fetching a narrow stream window.
- Stored objectsHold the records.
- Selected fieldsDetermine what the reader accesses.
- Query executionCombines the answer.
A lake label proves none of these capabilities. Ask which actual reader, fields and query produce the requested comparison. The paper supplies a mechanism, not a local speed guarantee. Storing objects and supplying an analytical answer are distinct responsibilities. Test the actual analytical reader separately. Columns here group the same field, rather than representing departments or different machines.
Broader access also needs a business reason. A central team may want to compare sites, while local managers want explanations they can use in their own operation. Decide who benefits from the comparison and who must respond to the questions it raises. Creating an audience for more information also creates work for the people expected to explain it.
Our published sensor-integration project describes connecting readings with assets, locations, units and time. Edge processing and historical storage supported reporting and maintenance follow-up; local control remained separate. Missing readings were not presented as normal conditions or zeros. The useful work was not simply collecting numbers. It was giving operators information they could investigate with its operating context attached. Virtual agents organized information for responsible people; they did not independently control equipment. These are the published project's stated outcomes, not a recovery-test result. The case gives a concrete reason to preserve the meaning alongside a reading, while keeping the project claim separate from a capability that still needs evidence. Reporting did not treat different sampling periods or changed equipment states as automatically comparable merely because each yielded numbers.
Compare the finished job, not the scaling story
McSherry, Isard and Murray define COST as the configuration that outperforms a competent single-threaded implementation. Their point: good scaling can conceal overhead. Their graph-processing measurements are not industrial database rankings.
- Baseline
- A competent simpler path.
- Same job
- The same required answer.
- Comparison
- Elapsed completion and resources, not scaling alone.
For an authorized future rehearsal, compare the actual retrieval job against its simpler supported alternative. A larger machine count is not itself a successful result. Keep the required output constant; changing the question would compare different jobs. Record resources alongside completion time, rather than reporting a speedup curve alone. Prefer observed completion over advertised scale. Configuration here means resources used, not a promise about monetary savings.
Keep the business's willingness to pay for attention separate from excitement about capacity. A request that interrupts a specialist whenever it arrives carries a different operating burden from familiar work colleagues can finish themselves. That burden deserves an explicit decision even when the technical result is satisfactory.
Likewise, distinguish time people spend working from time a requester waits. The remedies may involve staffing or scheduling rather than another platform. Include the effort of explaining the answer to its recipient. A new screen has not removed work if someone must accompany every use with the same lengthy explanation.
The purpose of this discussion is to make a supportable commitment, not to manufacture a saving. An honest account of existing effort can justify leaving a useful service alone.
Test a reader beyond the original service
SQLite documents a consistent completed database copy, not all application dependencies.
- CopyCompleted backup.
- OpenSeparate reader.
- AskRequired request.
BagIt makes another distinction. Its payload manifests list files and checksums. A complete bag includes the required elements and listed files; a valid bag also passes every manifest checksum. The payload is treated as uninterpreted bytes. Consequently, a valid package does not establish that an application can correctly interpret its contents, that measurements are qualified or that an omitted business dependency was included. This informational specification addresses packaging, not database transactions or a tested industrial recovery procedure. Use this distinction when someone presents a successful file check as evidence that the whole service is ready. Checksums alone do not establish protection against active attacks. A manifest cannot prove that all application dependencies were included.
The business question is who can continue the work when the usual route is unavailable. A person who can demonstrate an application may not be the person who supports it after an interruption. Give the continuity assignment to someone who can speak for the service being promised.
Consider the recipient too. An analyst may be comfortable asking a maintainer for help, while a supervisor expects a familiar route that works without that conversation. State which relationship the organization is prepared to support. Neither is inherently wrong, but they are different commitments to the people doing the work.
Assign the duty the rehearsal supports
Time identifies the observation moment; value is the retained reading. Quality is recorded status, not a new measurement qualification. Version identifies the record state; order links provide approved context.
- Request
- Ten minutes of earlier line-speed records.
- Comparison
- Export matches expected rows. Without original storage, the isolated restored reader lacks the map locating their series.
- Decision
- Keep existing trends; do not qualify recovery. Investigate the lookup map; retest the same request. Alternatives untested.
Missing fields stay unknown.
End with a commitment the organization can keep: a named recipient, a useful purpose and a supported next action. More access is valuable when it helps someone finish important work. It is not a substitute for deciding what work the business actually needs.
Sources and further guidance
References checked October 26, 2026.