Work · a Central European component manufacturer
Machine data that arrives twice, late, or not at all
The problem
Line telemetry was written straight into the reporting tables. Duplicate messages inflated output figures, and a network drop left gaps nobody noticed until a plant manager questioned a shift total.
The approach
We put an idempotent ingestion layer in front of reporting, keyed on machine and event time, with explicit gap detection that raises rather than interpolating. Reporting now reconciles to the counts the plant already trusted, which is what made adoption possible.
What was technically hard
Event time is the only key that makes a duplicate detectable, and machine clocks drift. A line that resets its clock, or reports local time across a daylight change, produces readings that look new when they are repeats and repeats when they are new — so deduplication cannot simply trust the timestamp it is handed. Defining what counts as the same reading, and refusing to decide when the evidence is ambiguous, was most of the work.
One decision we made, and why
Gap detection raises instead of interpolating. A filled-in value makes a chart look complete and a shift total wrong, and once somebody has planned against it the error is indistinguishable from a real reading. An acknowledged hole is a question the plant can answer from its own paperwork; an invented number is one nobody thinks to ask.
More work
- Reconciliation that reports breaks the day they happen →
Time to detect a reconciliation break fell from weeks to one day.
- Population statistics for any polygon a user can draw →
- A training record registry auditors can trust →
$ tell us what you're building and where it's stuck
Book a 30-min discovery call
We reply within one business day.