Skip to main content

HRI Design Patterns Overview

Purpose​

The HRI Design Patterns define reusable architectural solutions for socially intelligent Human-Robot Interaction systems.

They are not implementations, APIs, ROS graphs, or fixed software modules.

They are reusable HML-based architectural compositions that describe how HIFs, Semantic Cells, Semantic Operators, policies, state structures, queues, gates, and executors may be combined to solve recurring HRI design problems.

In this documentation, each pattern should be understood as:

a reusable semantic architecture for a recurring HRI problem

rather than:

a required implementation recipe

This distinction is important.

The concrete examples in this documentation illustrate possible compositions. They do not claim that every robot must implement the pattern in exactly the same way.

Relationship to SOCIAL and HML

The design patterns are the third layer of the framework.

SOCIAL principles
      ↓
HML modeling language
      ↓
HRI Design Patterns
      ↓
Concrete HRI architectures
  • SOCIAL defines the architectural commitments.
  • HML defines the modeling language and visual grammar.
  • HRI Design Patterns define reusable ways to compose HML primitives.
  • Concrete architectures instantiate those patterns for a specific robot, task, deployment, or research system.

The patterns therefore serve as the bridge between abstract principles and concrete HRI architecture.

What Counts as a Pattern?

A design pattern should satisfy four criteria.

1. It Solves a Recurring HRI Problem​

The pattern should address a problem that appears repeatedly in socially intelligent robotics.

Examples include:

  • synchronizing multiple perceptual experts
  • managing limited attention or compute resources
  • resolving intent with bounded latency
  • learning a user-specific gesture
  • detecting meaningful context changes
  • updating a transparent semantic world model
  • resolving incomplete tasks
  • validating social acceptability
  • styling an action for a social context

2. It Can Be Expressed in HML​

A pattern should be representable using HML primitives:

  • HIFs
  • HC / ST / SG / ES / HE cells
  • λ operators
  • policies
  • decision diamonds
  • state structures
  • queues
  • caches
  • HRI_DB
  • semantic objects

3. It Is Reusable Across Layers​

A pattern may be introduced in one cognitive layer but reused later.

For example, a Synchronous Multi-Extractor may first appear in Human Context processing, but the same pattern can also be reused for Scene Context extraction.

4. It Has Tradeoffs​

A pattern is not merely a diagram.

It should expose engineering tradeoffs such as:

  • latency
  • robustness
  • resource cost
  • explainability
  • synchronization complexity
  • policy complexity
  • risk of stale state
  • risk of over-escalation
  • failure behavior

Pattern Container Notation

When a diagram needs to show that several HML elements form a reusable pattern, the pattern may be wrapped in a clipped-corner container.

This convention follows the presentation style used for layer examples.

The clipped container means:

The enclosed elements form one reusable design pattern or pattern instance.

It does not mean:

The enclosed elements must be implemented as one software component.
Pattern Container Notation

A real HML composition can be wrapped as a reusable pattern instance. The wrapped composition may then be referenced by a shorthand clipped-corner container such as SME.

Input HIFOutput HIFPattern Instanceexample of a reusable wrapped HML compositionST₁ST₂ST₃SGexample wrapped composition: ST₁ / ST₂ / ST₃ → SG=SMEpattern shorthand

Use this container sparingly.

It is most useful in layer examples, where the same diagram contains several reusable patterns connected together.

Documentation Strategy

The design pattern documentation follows a layer-oriented reading path while preserving each pattern as a reusable catalog entry.

This gives two complementary views.

1. Layer-Oriented Reading Path​

The reader can follow the construction of a socially intelligent HRI architecture layer by layer:

Human Context
Scene Context
Robot Context
Context Management and Reasoning
Social Planning and Behavioral Synthesis
Actuation

This path explains why each pattern appears, what role it plays in a larger architecture, and how later layers reuse earlier patterns.

2. Reusable Pattern Catalog​

Each pattern is documented as an independent reusable architectural solution.

This allows a designer to ask:

Which pattern should I use to solve this specific HRI problem?

For example:

  • Use SME when several experts must process the same source while preserving temporal alignment.
  • Use EAG when attention or compute must be allocated according to social relevance.
  • Use TSC/TSP when language or intent resolution should be fast, cached, and escalation-aware.
  • Use CNE when the system should reason about meaningful change rather than all raw state.
  • Use TPR when a task is meaningful but not yet executable.
  • Use SCV when physically possible actions must be checked for social acceptability.

Cognitive Layer Map

The following organization is recommended for the documentation.

Human Context​

Human Context introduces patterns that convert raw human-facing signals into structured human context.

Introduced patterns:

  • Synchronous Multi-Extractor (SME)
  • Elastic Attention Governor (EAG)
  • Tiered Semantic Cache / Proxy (TSC/TSP)
  • Adaptive Signature Learner (ASL)

Layer examples may include:

  • video-based person, skeleton, gaze, and facial expression extraction
  • audio-based speech, azimuth, and sentiment extraction
  • resource-aware 2D-to-3D skeleton processing
  • verbal grounding of a learned gesture

These are examples of possible compositions, not required implementations.

Scene Context​

Scene Context reuses foundational extraction and attention patterns for the physical environment.

Reused patterns:

  • SME
  • EAG

Layer examples may include:

  • object detection
  • door detection
  • stair detection
  • obstacle detection
  • slope detection
  • SLAM integration
  • 2D-to-3D scene grounding

The goal is to show how the same pattern language generalizes beyond human perception.

Robot Context​

Robot Context models the robot's own body, resources, actuation interfaces, and internal state as HIF streams.

Reused HML structures:

  • HC
  • SG
  • HE
  • Semantic Cells
  • HIF streams
  • structured resource state

Layer examples may include:

  • battery HIFs
  • CPU / memory / disk HIFs
  • motion request HIFs
  • speech request HIFs
  • domain-specific execution requests

This layer is less about introducing new patterns and more about showing that the robot itself is also part of the semantic context.

Context Management and Reasoning​

Context Management connects Human Context, Scene Context, Robot Context, memory, and reasoning.

Introduced patterns:

  • Context Novelty Extractor (CNE)
  • HRI_DB Handler Pattern

Commonly reused or derived patterns:

  • Spatial-Based Reasoning (SBR)
  • Consistency Evaluator and Updater (CEU)
  • Query Social Handler (QSH)
  • Deep Social Insight Extractor (DSIE)
  • System Integrity and Agency Handler (SIAH)
  • Task Prerequisite Resolver (TPR)
  • Social Convention Validator (SCV)

This layer is where many pieces become connected.

It should show how unified context becomes delta context, how delta context updates or queries HRI_DB, and how specialized handlers reason over transparent state.

Social Planning and Behavioral Synthesis​

Social Planning turns validated context and pending intentions into socially appropriate action candidates.

Introduced or emphasized patterns:

  • Social Opportunity TPR
  • Social Action Stylist (SAS)
  • Late-Binding Behavioral Choreographer

Layer examples may include:

  • waiting for the right social moment
  • choosing whether to speak, gesture, move, or combine modalities
  • styling an action based on politeness, privacy, urgency, and context
  • binding behavior as late as possible to preserve adaptability

This layer should make clear that planning is not only about what to do, but also:

when to do it
how to do it
whether it is socially appropriate to do it now

Actuation​

The actuation layer remains open for future documentation.

It will likely involve patterns that connect executable HIFs to embodied effects through HE cells, actuator policies, safety gates, action monitoring, and rollback behavior.

Pattern Reuse Principle

A pattern should be introduced where it first becomes necessary, but it should remain reusable elsewhere.

For example:

SME is introduced in Human Context.
SME is reused in Scene Context.
SME may also appear in Social Action Styling when several stylists produce candidate modalities.

Similarly:

EAG is introduced for human attention and resource allocation.
EAG may also be reused for scene processing, social prioritization, or actuation bandwidth.

This avoids duplicating pattern definitions while still allowing concrete examples in each layer.

Documentation pages should use this wording:

First introduced in:
Reused in:
Example composition:

Concrete Examples Are Illustrative

Concrete examples are important because HRI patterns can otherwise remain too abstract.

However, each example should be framed as an illustration of possible connections.

Use language such as:

One possible instantiation is...
A layer example may connect...
In this example composition...
This does not require every implementation to...

Avoid language such as:

The system must...
The architecture is...
The robot always...

The purpose is to document a reusable pattern language, not to document one specific robot project.

Recommended Pattern Page Structure

Each design pattern page should use the same general structure.

# Pattern Name

## Intent
What the pattern is trying to achieve.

## Problem
The recurring HRI problem addressed by the pattern.

## Context
When the pattern is useful.

## HML Structure
A diagram using the HML visual language.

## Participants
HIFs, Semantic Cells, λ operators, policies, state structures, queues, gates.

## Flow
Step-by-step explanation of the pattern.

## Example Composition
A concrete example showing one possible connection of the pattern.

## Reuse
Where else the pattern may appear.

## SOCIAL Principles Supported
How the pattern supports S/O/C/I/A/L.

## Tradeoffs
Benefits, costs, risks, and failure modes.

## Implementation Notes
Practical considerations for deployment or experimentation.

This structure should remain stable across pattern pages.

Foundational and Composite Patterns

Some patterns are foundational.

They provide basic reusable mechanisms.

Examples:

  • Synchronous Multi-Extractor
  • Elastic Attention Governor
  • Tiered Semantic Cache / Proxy
  • Adaptive Signature Learner
  • Context Novelty Extractor
  • HRI_DB Handler

Other patterns are composite or orchestration-oriented.

They combine earlier mechanisms into higher-level reasoning or planning roles.

Examples:

  • Spatial-Based Reasoning
  • Consistency Evaluator and Updater
  • Query Social Handler
  • Deep Social Insight Extractor
  • System Integrity and Agency Handler
  • Task Prerequisite Resolver
  • Social Convention Validator
  • Social Opportunity TPR
  • Social Action Stylist
  • Late-Binding Behavioral Choreographer

This distinction is not rigid.

A pattern may begin as a composite in one context and later become a reusable building block in another.

Example Mapping of HRI Design Patterns to Architectural Layers

Layer-to-Pattern MapDesign patterns are introduced layer by layer and reused in later cognitive layers.Human ContextSME / EAG / TSC / ASLScene Contextreuse SME / EAGRobot ContextHIF streams / HC / SG / HEContext Management and ReasoningCNE / HRI_DB Handler / SBR / CEU / QSH / DSIE / SIAH / TPR / SCVconnects multiple contexts, memory, validation, and reasoningSocial Planning and Behavioral SynthesisSocial Opportunity TPR / SAS / Late-Binding Choreographer

This map is intentionally schematic.

It shows the recommended learning path and reuse structure, not a mandatory architecture.

Writing Style for Pattern Pages

Use engineering-academic language.

Prefer:

The pattern supports...
The pattern can be instantiated as...
A common implementation strategy is...
One possible HML composition is...
The pattern trades latency for synchronization integrity...

Avoid:

The robot must...
The system always...
This is the implementation...

The documentation should support academic publication by presenting the patterns as generalizable architecture, while still remaining concrete enough for engineers to apply.

Conclusion

The HRI Design Patterns document reusable architectural solutions built from HML primitives and governed by SOCIAL principles.

They should be organized by the cognitive layer in which they first become necessary, while remaining reusable across later layers.

This organization supports both:

a readable architectural learning path

and:

a reusable pattern catalog

The next pages define the individual patterns, beginning with the Human Context layer and the Synchronous Multi-Extractor.