State, Memory, and Queues

Overview

State, Memory, and Queues are the HML structures that preserve information across time.

If HIFs are the semantic units that flow through the system, and Semantic Cells are the architectural units that process them, then state and memory structures are the components that allow the architecture to remember, reuse, delay, resume, and reason across multiple interaction moments.

These structures support:

semantic continuity
persistent world knowledge
session context
caching
asynchronous interaction
delayed execution
fact-triggered resumption
temporal reasoning
explainable memory
human correction
policy-aware state updates

They are essential for moving from immediate command-response behavior to socially aware long-term interaction.

Why State Structures Are Needed

A HIF pipeline alone is not enough for HRI.

Human-Robot Interaction is often incomplete, delayed, ambiguous, or dependent on future context.

Examples:

A user says: Bring this bottle to Bob, but Bob's location is unknown.
A robot needs a short time window of skeletons to learn a gesture.
A task should wait until the user is available.
A previous fact affects how a new instruction is interpreted.
A query should be answered from memory before escalating to an LLM.
A fact may become stale unless reinforced.
A robot may need to remember that it already asked a clarification question.
A social request may expire if the opportunity window closes.

State, memory, and queue structures allow HML architectures to preserve semantic continuity across these situations.

Three Families of State Structures

HML distinguishes between three broad families of state structures.

Family	Purpose	Examples
Semantic Memory	Persistent or semi-persistent knowledge about the world	HRI_DB, facts, beliefs, preferences
Operational Memory	Temporary context used during an interaction	session context, short-term memory, recent references
Flow-Control Structures	Structures that manage time, ordering, delay, or readiness	queues, buffers, time-series windows, pending queues

These structures may appear as side inputs, side outputs, or explicit storage elements in HML diagrams.

General Memory and Flow View

A typical HML architecture allows HIFs to flow through Semantic Cells while interacting with state structures.

This diagram shows that state structures are not necessarily part of the main left-to-right HIF flow.

They can support, modify, delay, or resume that flow.

Semantic Objects and Dictionaries

A Semantic Object is a JSON-like structured representation of meaning.

It may appear inside a HIF, inside a policy file, inside HRI_DB, or as the result of a query.

Example:

{
  "person": "Bob",
  "location": "kitchen",
  "confidence": 0.92,
  "source": "vision_tracker"
}

Semantic Objects are used to represent:

HIF properties
facts
query results
task parameters
policy parameters
configuration values
validation outcomes
world-state entities

The visual notation {} or {:} may be used to indicate a structured semantic object.

Important distinction:

A semantic object is a representation format.
It is not necessarily a database, cache, or persistent store.

A dictionary-like structure may be used inside many different HML components.

HRI_DB — Persistent Semantic World Model

HRI_DB is the persistent semantic world model of the robot.

It represents what the robot currently believes, remembers, or assumes about the world, people, objects, tasks, social context, and itself.

It may contain:

entities
facts
beliefs
people
object states
locations
user preferences
social context
policies
autonomy state
integrity state
interaction history
unresolved assumptions
source and confidence metadata

Example:

{
  "people": {
    "bob": {
      "type": "person",
      "location": "kitchen",
      "last_seen": "2026-05-11T12:34:01Z",
      "source": "vision_tracker",
      "confidence": 0.86
    }
  }
}

HRI_DB is not merely a technical database.

It is the robot's transparent semantic world model.

It supports:

explainability
consistency evaluation
human correction
deterministic querying
LLM-assisted reasoning
persistence across sessions
context continuity
fact-triggered task resumption

HRI_DB is important enough to receive a dedicated documentation page.

This page introduces it only as one of the central memory structures in HML.

HRI_DB is discussed in depth in the dedicated HRI_DB page. This page treats it only as one member of the broader family of state and memory structures. The dedicated page explains its transparent JSON-like reference implementation, deterministic querying strategy, confidence decay, human correction, and LLM escalation over structured state.

Cache vs HRI_DB

A cache is not the same as HRI_DB.

A cache stores reusable computation or interpretation results.

HRI_DB stores semantic world knowledge.

Example:

Cache:
  "do you know Bob's whereabouts?" → query intent: where_is(Bob)

HRI_DB:
  Bob is in the kitchen.

In other words:

Cache = reuse of interpretation or computation
HRI_DB = semantic memory of the world

Caches support:

latency reduction
token saving
consistency of repeated interpretation
cheaper fallback chains
reduced LLM calls

HRI_DB supports:

persistent state
fact lookup
consistency checking
explainability
human correction
world-model reasoning

Both may be used together.

Semantic Cache

A Semantic Cache stores reusable semantic results.

Examples:

Text phrase → parsed intent
Query pattern → structured query
LLM answer → reusable interpretation
Spatial phrase → resolved pattern

A Semantic Cache is especially useful inside escalation patterns.

For example:

The ES may first try deterministic code, then the semantic cache, and only then escalate to a more expensive generative model.

Short-Term Memory and Session Context

Not all memory should be persistent.

Short-Term Memory or Session Context stores information that is relevant to the current interaction but may not belong in long-term HRI_DB.

Examples include:

current user
active speaker
current topic
last referenced object
recent clarification
pronoun context
active task
temporary assumptions
dialogue state
currently open social window

Example:

{
  "current_user": "Alice",
  "last_referenced_object": "bottle_5",
  "current_topic": "medicine delivery",
  "last_clarification_question": "Where is Bob?"
}

Session context supports natural dialogue and short-horizon reasoning.

It should not automatically become persistent world knowledge.

Buffers

A Buffer is bounded temporary storage.

It stores HIFs or data items for short-term processing.

Examples:

Buffer<HIF>
Buffer<AudioSnippet>
Buffer<VideoFrame>
Buffer<PersonState>

Buffers are useful for:

smoothing noisy signals
waiting for synchronization
collecting evidence
short-term temporal reasoning
managing asynchronous streams
rate matching between components

A buffer may be bounded by:

time
number of items
memory size
confidence
source
interaction state

Time-Series Windows

A Time-Series Window is a buffer with explicit temporal meaning.

It stores a sequence of frames, poses, gestures, audio snippets, or other HIFs over a time interval.

Example:

TimeSeriesWindow<SkeletonHIF>

Typical uses:

gesture learning
gesture classification
motion pattern detection
temporal smoothing
repeated behavior detection
engagement estimation
activity recognition

This structure is central to patterns such as the Adaptive Signature Learner.

The window gives the system enough temporal context to classify or learn behavior that cannot be understood from a single HIF.

Queues

A Queue is an ordered collection of HIFs, tasks, requests, or actions waiting for processing.

General form:

Queue<T> = ordered collection of items waiting for processing

Queues are used when processing is:

asynchronous
delayed
prioritized
resource-dependent
interaction-dependent
waiting for missing information
waiting for social opportunity

Common queue types include:

HIF stream queue
task queue
request queue
prioritized queue
pending queue
action queue
social request queue

A queue is not only a data structure.

In HML, it often represents part of the lifecycle of a task or interaction request.

Prioritized Queue

A Prioritized Queue orders tasks or requests according to a policy.

Possible priority criteria include:

urgency
safety
social relevance
user importance
task age
explicit instruction priority
autonomy level
resource availability
estimated execution cost
decay factor

Example:

PrioritizedQueue<TaskHIF>

A Prioritized Queue is useful when several candidate tasks compete for attention or execution.

Examples:

user instructions
proactive social requests
system maintenance tasks
safety alerts
reminders
delayed actions

Pending Queue

A Pending Queue stores tasks or requests that are not executable yet but should not be discarded.

Reasons for pending status include:

missing information
unavailable person
closed social window
low confidence
insufficient resources
missing permission
unresolved ambiguity
future timing condition
safety constraint
task dependency

Example:

Bring bottle_5 to Bob
→ Bob location unknown
→ Pending Queue

Another example:

Ask Bob for help
→ Bob is busy
→ Pending Request Queue

Pending queues are essential for moving from command-response systems to goal-oriented agents.

A robot should not fail immediately when a task is incomplete.

It should hold the task, seek missing information, and resume when the required context becomes available.

Pending Queue Lifecycle

The following diagram shows a common pending task lifecycle.

This lifecycle appears in patterns such as:

Task Prerequisite Resolver
Social Opportunity TPR
delayed clarification handlers
proactivity engines
pending social requests

A pending task should usually include an expiration policy or decay factor to prevent outdated tasks from executing later in an inappropriate context.

State Update vs HIF Flow

Some information flows through the system as HIFs.

Other information is stored as state.

For example:

User says: "Bob is in the kitchen."

This may become:

Text HIF → Intent / Fact Extraction ST → Fact HIF → Consistency Evaluator → HRI_DB update

Later, another query may retrieve that fact:

Query HIF → HRI_DB Handler → Answer HIF

This means that state structures may appear as:

side inputs to Semantic Cells
side outputs from Semantic Cells
persistent memory stores
triggers for pending task re-evaluation
sources of newly created HIFs

The HIF flow and the state model are therefore connected, but they are not the same thing.

Visual Notation

The following table summarizes suggested visual notation for state and memory structures.

Symbol	Structure	Meaning
	Semantic Object	Structured JSON-like semantic data
	HRI_DB	Persistent semantic world model
	Semantic Cache	Reusable computation or interpretation store
	Buffer	Bounded temporary storage
	Time-Series Window	Temporally scoped buffer
	Queue	Ordered waiting items
	Prioritized Queue	Queue governed by urgency or relevance policy
	Pending Queue	Delayed tasks or requests
	Session Context	Current interaction context or short-term memory

The exact drawing style may vary, but the semantic meaning should remain explicit.

Relationship to HRI Design Patterns

Many HRI Design Patterns depend on state, memory, or queue structures.

Pattern	State / Memory Structure
Tiered Semantic Cache	Semantic Cache
Adaptive Signature Learner	Time-Series Window
Context Novelty Extractor	Previous Unified Context
HRI_DB Handler	HRI_DB
Spatial Based Reasoning	HRI_DB + spatial object state
Consistency Evaluator & Updater	HRI_DB + fact validation
Query Social Handler	HRI_DB + Session Context
Deep Social Insight Extractor	HRI_DB updates
System Integrity & Agency Handler	Integrity state + Autonomy state
Task Prerequisite Resolver	Prioritized Queue + Pending Queue
Social Opportunity TPR	Pending Request Queue
Late-Binding Behavioral Choreographer	delayed request state + current context

This page defines the vocabulary needed to read those patterns.

Design Guidelines

Do Not Use HRI_DB for Everything

Not every temporary value belongs in persistent memory.

Use session context or short-term memory for temporary dialogue state.

Use HRI_DB for facts, preferences, entities, and state that may matter beyond the immediate moment.

Do Not Treat Cache as Truth

A cache stores reusable computation.

It should not be treated as the authoritative world model.

If a cached interpretation conflicts with current context or HRI_DB, the system should validate or invalidate it.

Preserve Provenance

Stored facts should include source information whenever possible.

Example:

{
  "fact": "Bob is in the kitchen",
  "source": "vision_tracker",
  "confidence": 0.86,
  "timestamp": "2026-05-11T12:34:01Z"
}

Provenance supports explainability and correction.

Support Confidence Decay and Forgetting

Some facts become less reliable over time.

For example:

Bob was in the kitchen 30 minutes ago.

may be less reliable than:

Bob was detected in the kitchen 10 seconds ago.

HRI_DB implementations should support confidence decay, expiration, or reinforcement policies where appropriate.

Use Pending Queues Carefully

Pending tasks should not live forever.

They should include:

creation time
reason for pending status
required condition
priority
decay factor
expiration policy
re-evaluation trigger

This prevents outdated tasks from executing in the wrong context.

Separate Session Context from Persistent Memory

Session context is useful for pronouns, dialogue continuity, and temporary assumptions.

Persistent memory is useful for stable facts, preferences, and world state.

Mixing them can cause stale or accidental assumptions to become long-term beliefs.

Document Memory Side Effects

Any Semantic Cell that reads or writes memory should make that behavior explicit.

This is important for:

debugging
reproducibility
safety review
auditability
explainability

Prefer Human-Readable State

When possible, use readable structured formats.

This supports:

human inspection
operator correction
LLM-assisted reasoning
academic reproducibility
safety audits

Validate Conflicting Updates

A new fact should not silently overwrite a conflicting old fact.

Conflicts should be routed through a consistency mechanism, such as a Consistency Evaluator and Updater.

Conclusion

State, Memory, and Queues allow HML architectures to reason across time.

They support memory, caching, delayed execution, asynchronous interaction, and fact-triggered resumption.

They also help the robot remain explainable by making its remembered state inspectable and its delayed tasks explicit.

In HML:

HIFs carry semantic information.
Semantic Cells process semantic information.
State structures preserve semantic information across time.
Queues manage semantic work that cannot be completed immediately.

Together, these structures allow socially intelligent robots to move beyond immediate reaction and toward persistent, context-aware, and explainable interaction.

Overview​

Why State Structures Are Needed

Three Families of State Structures

General Memory and Flow View

Semantic Objects and Dictionaries

HRI_DB — Persistent Semantic World Model

Cache vs HRI_DB

Semantic Cache

Short-Term Memory and Session Context

Buffers

Time-Series Windows

Queues

Prioritized Queue

Pending Queue

Pending Queue Lifecycle

State Update vs HIF Flow

Visual Notation

Relationship to HRI Design Patterns

Design Guidelines

Do Not Use HRI_DB for Everything​

Do Not Treat Cache as Truth​

Preserve Provenance​

Support Confidence Decay and Forgetting​

Use Pending Queues Carefully​

Separate Session Context from Persistent Memory​

Document Memory Side Effects​

Prefer Human-Readable State​

Validate Conflicting Updates​

Conclusion

Overview

Do Not Use HRI_DB for Everything

Do Not Treat Cache as Truth

Preserve Provenance

Support Confidence Decay and Forgetting

Use Pending Queues Carefully

Separate Session Context from Persistent Memory

Document Memory Side Effects

Prefer Human-Readable State

Validate Conflicting Updates