Unified Namespace vs Data Historian: Why Neither Replaces the Other

People type unified namespace vs data historian into a search box for one reason: a vendor told them one replaces the other. Neither claim survives ten minutes with the two designs.

A unified namespace is a live topic space. Publishers write once, subscribers read what they care about and nothing holds a queue of the past. A historian is the opposite kind of object. It is a time-series archive whose entire economic case rests on discarding data on purpose, under a rule precise enough to defend to an auditor.

They answer different questions on different time horizons. The confusion is commercially convenient. Broker vendors would like the historian line item to disappear. Historian vendors would like the broker to look like middleware you already own. I have watched plants in Guangdong specify this three times now and the same error shows up in the architecture diagram every time: one box where there should be two, with an arrow that quietly assumes storage.

A unified namespace is a topic space with a schema, not a database

Strip the marketing and a UNS is an MQTT broker plus a naming discipline. The naming discipline is the part that matters. Topics are shaped like the plant, usually from the ISA-95 equipment hierarchy standardised as IEC 62264: enterprise, site, area, work centre, work unit. A subscriber asks for a subtree and receives everything under it, without knowing who publishes.

Sparkplug B, published by the Eclipse Sparkplug Working Group and transposed as ISO/IEC 20237:2023 in October 2023, adds the part that makes this workable in an OT context., adds the part that makes this workable in an OT context. The topic structure is fixed: namespace/group_id/message_type/edge_node_id/device_id, with the namespace element spBv1.0. Nine message types, protobuf payloads, report by exception.

The useful bit is the birth certificate. The useful bit is the birth certificate. NBIRTH must declare every metric the node will ever report on. Aliases are optional in Sparkplug, but a node that uses them must put both the name and the alias in NBIRTH, after which NDATA and DDATA carry the alias with the name excluded. The schema is announced once and the data messages stay small. The schema is announced once and the data messages stay small. Session state rides on two counters: bdSeq, an INT64 metric registered in the MQTT Will message and incremented on each connect, and seq, a per-message counter between 0 and 255 that wraps.

Then the part most summaries skip. When a host application receives an NDEATH, the specification requires it to mark the edge node offline and to mark every metric from that node and its devices STALE, timestamped with the host's own UTC clock. That is a quality flag, not a stored value. The broker is telling you it no longer knows, which is a more honest answer than a database returning the last row it happens to hold.

The broker is cheap because it forgets. The historian is affordable because it forgets selectively, on a rule you can defend to an auditor.

A historian is an archive engineered around throwing data away

A historian collects tags, filters them twice and writes the survivors to a compressed archive.

The first filter is exception reporting, a deadband applied at the interface before anything is sent. The second is compression, applied at the archive. In the AVEVA PI System that second filter is the swinging door algorithm, patented by Edgar H. Bristol in 1987 as US 4,669,097 and refined with a rolling pivot in US 5,774,385. Competing historians filter at the archive too, though not all of them by swinging door., patented by Edgar H. Bristol in 1987 as US 4,669,097 and refined with variable offsets in US 5,774,385.

Swinging door works on a promise rather than a sample rate. It holds an open corridor around a linear fit from the last archived point. A new reading is stored only at the moment a straight line from the last stored point would misrepresent some intermediate reading by more than the configured compression deviation. Everything inside the corridor is dropped and reconstructed later by interpolation.

Read that again, because it is the whole difference. The historian does not promise you every value. It promises a bounded error. Set the compression deviation to one percent of span and the archive holds any reconstructed point within one percent of the values that reached it, while storing a fraction of the events that arrived. The exception deadband at the interface adds its own error before that, so the end to end bound is the two deviations summed, which is why the usual advice is to set exception at roughly half of compression., while storing a fraction of the events that arrived. That guarantee is what turns twenty years of a hundred thousand tags into a line item instead of a capital project.

What a unified namespace does not give you is the record

A broker's memory is one retained message per topic. Publish a new one and it overwrites the old. Publish a zero-byte payload and it deletes. That is the complete storage model in the OASIS MQTT specification, and it is a cache, not an archive.

The defaults are worse than most buyers assume. EMQX 5 stores retained messages with a RAM backend by default, with max_retained_messages set to 0 for unlimited and a 1 MB payload threshold above which a message is treated as ordinary. Restart that broker and the current-state view is empty until every publisher republishes. HiveMQ queues at most 1000 messages per client by default and its default strategy discards newly arriving messages once that queue is full, so a subscriber offline through a long shift keeps the first 1000 messages and loses every one after them.

Worth checking on your own bench rather than taking from me: publish ten thousand retained values across a realistic topic tree, stop the broker, start it again and count what comes back before any publisher reconnects. On stock EMQX settings the count you should expect is zero. I am describing the check, not reporting a run of it.

Then there is the record itself. Under 40 CFR 63.10(b)(1), affected sources must keep records for at least five years, and the most recent two years must be retained on site. Pharmaceutical batch records under 21 CFR 211.180(a) run at least one year past the batch expiry date. Where 21 CFR Part 11 applies, 11.10(e) requires secure computer-generated time-stamped audit trails, retained at least as long as the electronic records they cover. A retained message has no timestamp history, no audit trail and no retention policy. It is not the thing you hand an inspector.

What a historian does not give you is loose coupling

Run the argument the other way and the historian loses just as clearly.

A historian is a poll, store and query engine. Its interface is a query API and its natural client is a reporting tool. Every new consuming system means another connector configured against the historian, another credential, another polling schedule. Ten consuming systems and four data sources is fourteen separately configured and separately credentialed connections against one query API, every one of them bespoke and every one of them breaking on its own. Wire the same systems directly to each other with no hub and the count becomes the product rather than the sum, which is the trap a topic space is there to avoid.

It also holds no current-state view across systems, only across the tags it collects. The production order from the MES, the pass or fail from the lab system and the schedule change from ERP are not tags. They arrive as events on their own timelines and they are exactly the context that makes a temperature reading mean something. Asset frameworks help, but they organise what the historian already collects inside one vendor's model, and the write path stays point to point.

There is a latency story too. Historians are tuned for write throughput and range queries. Asking one to serve a line-side display refreshing every 200 milliseconds is asking a filing cabinet to behave like a window.

Unified namespace vs data historian: the retention arithmetic settles it

Concrete numbers end this argument faster than principles do.

On AVEVA’s own PI Square community forum, an engineer describes a system of around 1.5 million tags with compression and exception switched off on most of them, so every one minute value lands in the archive and it fills at roughly 1 GB per hour. That is the uncompressed case and it is the right one to price, because it is what a historian costs the moment you stop letting it forget. Do the division yourself. That is 90 million events an hour, so about 12 bytes per stored event, 24 GB a day and 8.76 TB a year. The five year window in 40 CFR 63.10(b)(1) works out near 44 TB, of which the most recent two years, roughly 17.5 TB, has to sit on site.

The same thread carries a sizing rule of thumb, archive file size in MB around (licensed point count x 3) / 1024, though the AVEVA engineer notes it assumes some compression and so does not fit this system. Run through AVEVA's own sizing tool instead, the recommendation for 1.5 million tags came back at about 6 GB per archive file.

Now price the broker side of the same plant. The same 1.5 million signals as retained topics is 1.5 million retained messages holding one value each, comfortably inside EMQX's unlimited default. Megabytes, not terabytes. Storage is a rounding error precisely because there is no history in it.

That is the economics in two paragraphs. The broker is cheap because it forgets. The historian is affordable because it forgets selectively, on a rule you can defend to an auditor. Anything claiming to do both at once is either paying full price for storage it never compresses or quietly dropping data it never told you it dropped.

The honest architecture gives each layer exactly one job

Edge nodes publish once, Sparkplug B, report by exception, into a broker cluster. Anything needing current state subscribes: SCADA, dashboards, MES, the anomaly detector on the line.

The historian is one more subscriber. Not a privileged one. It holds a durable session with a queue sized for the longest outage you are willing to survive, and it writes what it receives through exception and compression filters tuned per tag class rather than globally. Flow and temperature loops get a wide compression deviation. Anything feeding a regulatory report has compression switched off outright rather than set to a deviation of zero, because a zero deviation still lets the swinging door drop every point sitting exactly on the line, which is most of a flatlined tag. You pay for that deliberately across a few hundred tags instead of accidentally across a hundred thousand., and you pay for that deliberately across a few hundred tags instead of accidentally across a hundred thousand.

Two rules keep it honest. The broker is never the query interface, so no application asks it for yesterday. The historian is never the integration bus, so no application polls it for live values. The only loop back is derived: values computed over history are published to the broker as new topics, which keeps the current-state view complete without letting the archive turn into middleware.

Naming is the last piece and the one that decides whether any of this survives contact with a second plant. Derive the topic path and the historian's asset path from the same ISA-95 hierarchy. If they diverge you have built a mapping table, and mapping tables are where these projects die.

The strongest argument against building a unified namespace at all

Take the counter-case seriously, because parts of it are correct.

No standards body defines a unified namespace. MQTT is OASIS. Sparkplug is ISO/IEC 20237. ISA-95 is IEC 62264. UNS is a pattern named and popularised by a consultant, Walker Reynolds of 4.0 Solutions, and everything sold under the label is somebody's product carrying a conformance claim nobody can test. That is a real difference in kind from the standards it sits on.

Second, Sparkplug is not as loosely coupled as the pitch implies. bdSeq, a rolling seq between 0 and 255, primary host STATE messages, a mandatory obligation to mark metrics STALE on death. That is a stateful protocol with ordering requirements. You have not removed coupling, you have relocated it onto a broker and its session semantics, and when the broker misbehaves every subscriber finds out simultaneously.

Third, scale honestly. A single-site plant with one SCADA, one historian and no roadmap for a second consumer gets very little from a UNS while inheriting a cluster to run, patch and secure. Under roughly five consuming systems the historian's asset framework plus a handful of connectors is cheaper, and I would say so in the room.

The argument flips when the fifth, sixth and seventh consumer arrive, because point-to-point integration cost grows with the product of producers and consumers while a topic space grows with their sum.

China built a national naming layer and still needed the archives

There is a large natural experiment on this question that almost nobody outside China cites.

MIIT's industrial internet identifier resolution system has completed its 5+2 national top-level nodes and, MIIT's industrial internet identifier resolution system has completed its 5+2 national top-level nodes and, on China Academy of Information and Communications Technology figures reported by Economic Daily on 5 December 2024, carries 370 secondary nodes, more than 450,000 connected enterprise nodes and a cumulative 613 billion registered identifiers spanning 31 provinces and 47 industries. [ spanning 31 provinces and 47 industries. It is a naming and resolution layer built at national scale, with state backing and genuine adoption.

It stores no time series. It resolves an identifier to a service endpoint and stops there. Every factory hanging off it still runs a historian, because resolution answers where and what, never when and how much. A country spent a decade demonstrating that the naming layer and the archive are separate concerns, and the plants that treated the identifier registry as a database learned it the same way everyone else does.

The two boxes are not a compromise you settle for. A namespace that remembers everything is a slow database with a topic tree bolted on. An archive that every system talks to directly is a bottleneck with a compliance obligation attached. Anyone selling you one box is selling you the one they already built.

Sources

Eclipse Sparkplug 3.0.0 Specification: https://sparkplug.eclipse.org/specification/version/3.0/documents/sparkplug-specification-3.0.0.pdf

Sparkplug normative operational behaviour requirements (Eclipse GitHub): https://github.com/eclipse-sparkplug/sparkplug/blob/master/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc

OASIS MQTT Version 5.0 specification: https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html

EMQX 5.0 retained message configuration defaults: https://docs.emqx.com/en/emqx/v5.0/mqtt/mqtt-retained-messages.html

HiveMQ broker configuration defaults: https://docs.hivemq.com/hivemq/latest/user-guide/configuration.html

US Patent 4,669,097, Bristol, data compression for display and storage (swinging door): https://patents.google.com/patent/US4669097

40 CFR 63.10 recordkeeping and reporting requirements: https://www.law.cornell.edu/cfr/text/40/63.10

AVEVA PI Square: archive sizing on a large system: https://community.aveva.com/pi-square-community/f/forum/88241/archive-sizing-on-a-large-system

Economic Daily, 5 December 2024,: https://economy.gmw.cn/2024-12/05/content_37720846.htm]