AimDB v0.3.0 Release Notes

Major update with RecordId/RecordKey architecture and buffer metrics!

🎯 What's New in v0.3.0

This release introduces a complete architectural overhaul of AimDB's internal record storage system, enabling multi-instance records (same type, different keys), stable O(1) indexing, and comprehensive buffer metrics. The new RecordId/RecordKey system provides both the performance benefits of numeric indexing and the usability of human-readable keys.

✨ Major Features

🔑 RecordId + RecordKey Architecture

Complete rewrite of internal storage for stable record identification!

AimDB now supports multiple records of the same type with unique keys, enabling patterns like:

Multiple sensors of the same type (Temperature) with different keys ("sensors.indoor", "sensors.outdoor")
Multi-tenant configurations ("tenant.a.config", "tenant.b.config")
Regional data streams ("region.us.metrics", "region.eu.metrics")

Key Components:

RecordId: u32 index wrapper for O(1) Vec-based hot-path access
RecordKey: Hybrid &'static str / Arc<str> with zero-alloc static keys and flexible dynamic keys
O(1) key resolution via HashMap<RecordKey, RecordId>
Type introspection via HashMap<TypeId, Vec<RecordId>>

New API:

use aimdb_core::{AimDbBuilder, buffer::BufferCfg};
use aimdb_tokio_adapter::TokioAdapter;
use serde::{Serialize, Deserialize};
use std::sync::Arc;

#[derive(Clone, Debug, Serialize, Deserialize)]
struct Temperature {
    celsius: f32,
    sensor_id: String,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let runtime = Arc::new(TokioAdapter::new()?);
    
    let mut builder = AimDbBuilder::new().runtime(runtime);

    // Register MULTIPLE records of the same type with different keys
    builder.configure::<Temperature>("sensors.indoor", |reg| {
        reg.buffer(BufferCfg::SingleLatest);
    });

    builder.configure::<Temperature>("sensors.outdoor", |reg| {
        reg.buffer(BufferCfg::SingleLatest);
    });

    let db = builder.build().await?;

    // Key-based access for multi-instance records
    let indoor_producer = db.producer_by_key::<Temperature>("sensors.indoor")?;
    let outdoor_producer = db.producer_by_key::<Temperature>("sensors.outdoor")?;

    indoor_producer.produce(Temperature { celsius: 22.5, sensor_id: "indoor-1".into() }).await?;
    outdoor_producer.produce(Temperature { celsius: 15.2, sensor_id: "outdoor-1".into() }).await?;

    // Introspection
    let temp_ids = db.records_of_type::<Temperature>();  // Returns &[RecordId] with 2 IDs
    let id = db.resolve_key("sensors.indoor");           // Returns Option<RecordId>

    Ok(())
}

Key Naming Conventions:

Use dot-separated hierarchical names: "sensors.indoor", "config.app"
Keys must be unique across all records (duplicate keys panic at registration)
Static string literals ("key") are zero-allocation via &'static str
Dynamic keys (String::from("key")) use Arc<str> for efficient cloning

📊 Buffer Metrics API (Feature-Gated)

Comprehensive buffer introspection for monitoring and debugging!

Enable buffer metrics with the metrics feature flag:

[dependencies]
aimdb-core = { version = "0.3.0", features = ["metrics"] }
aimdb-tokio-adapter = { version = "0.3.0", features = ["metrics"] }

New Metrics:

use aimdb_core::buffer::BufferMetricsSnapshot;

// Get metrics from any record
let metadata = db.record_metadata::<Temperature>("sensors.indoor")?;

if let Some(metrics) = metadata.buffer_metrics {
    println!("Produced: {}", metrics.produced_count);
    println!("Consumed: {}", metrics.consumed_count);
    println!("Dropped: {}", metrics.dropped_count);
    println!("Occupancy: {}/{}", metrics.occupancy.0, metrics.occupancy.1);
}

Available Metrics:

produced_count: Total items pushed to the buffer
consumed_count: Total items consumed across all readers
dropped_count: Total items dropped due to lag (per-reader semantics documented)
occupancy: Current buffer fill level as (current, capacity) tuple

Supported Buffers:

✅ SPMC Ring Buffer: Full metrics support
✅ SingleLatest: Full metrics support
✅ Mailbox: Full metrics support

Tokio Adapter: Full implementation with atomic counters
Embassy Adapter: Feature flag present (API consistency), but metrics not functional on embedded targets (requires std)

🔍 Enhanced Introspection API

New methods for exploring records at runtime:

// Find all records of a specific type
let temperature_records = db.records_of_type::<Temperature>();
for record_id in temperature_records {
    let metadata = db.record_metadata_by_id(*record_id)?;
    println!("Temperature record: {} (key: {})", record_id.0, metadata.record_key);
}

// Resolve key to RecordId
if let Some(record_id) = db.resolve_key("sensors.indoor") {
    println!("Found record with ID: {}", record_id.0);
}

// Key-bound producers/consumers
let producer = db.producer_by_key::<Temperature>("sensors.indoor")?;
let consumer = db.consumer_by_key::<Temperature>("sensors.outdoor")?;

println!("Producer key: {}", producer.key());
println!("Consumer key: {}", consumer.key());

🏗️ Internal Architecture Improvements

Optimized storage for sub-50ms latency:

Before (v0.2.0):

BTreeMap<TypeId, Box<dyn AnyRecord>>  // O(log n) lookups

After (v0.3.0):

Vec<Box<dyn AnyRecord>>                      // O(1) hot-path access by RecordId
HashMap<RecordKey, RecordId>                 // O(1) name lookups
HashMap<TypeId, Vec<RecordId>>               // O(1) type introspection

Performance Benefits:

✅ O(1) hot-path access (was O(log n))
✅ Stable RecordId across application lifetime
✅ Zero-allocation static keys
✅ Efficient multi-instance type lookups

🔨 Breaking Changes

1. Record Registration API

All records now require a key parameter:

// Before (v0.2.x)
builder.configure::<Temperature>(|reg| {
    reg.buffer(BufferCfg::SingleLatest);
});

// After (v0.3.0)
builder.configure::<Temperature>("sensor.temperature", |reg| {
    reg.buffer(BufferCfg::SingleLatest);
});

2. Type-Based Lookup Ambiguity

If you register multiple records of the same type, type-based methods return AmbiguousType error:

// With multiple Temperature records registered...
db.produce(temp).await  // ❌ Returns Err(AmbiguousType { count: 2, ... })

// Use key-based methods instead:
db.produce_by_key("sensors.indoor", temp).await  // ✅ Works correctly

Migration Strategy:

Single-instance records: Type-based API still works (produce(), subscribe(), etc.)
Multi-instance records: Use key-based API (produce_by_key(), subscribe_by_key(), etc.)

3. DynBuffer Implementation

Custom buffer implementations must now explicitly implement DynBuffer<T>:

// Before (v0.2.x) - automatic via blanket impl
impl<T: Clone + Send> Buffer<T> for MyBuffer<T> { ... }
// DynBuffer was automatically implemented

// After (v0.3.0) - explicit implementation required
impl<T: Clone + Send> Buffer<T> for MyBuffer<T> { ... }

impl<T: Clone + Send + 'static> DynBuffer<T> for MyBuffer<T> {
    fn push(&self, value: T) {
        <Self as Buffer<T>>::push(self, value)
    }
    
    fn subscribe_boxed(&self) -> Box<dyn BufferReader<T> + Send> {
        Box::new(self.subscribe())
    }
    
    fn as_any(&self) -> &dyn core::any::Any {
        self
    }
    
    // Optional: implement metrics_snapshot() if you support metrics
    #[cfg(feature = "metrics")]
    fn metrics_snapshot(&self) -> Option<BufferMetricsSnapshot> {
        None // or Some(...) if you track metrics
    }
}

Why this change? Enables adapters to provide metrics_snapshot() when the metrics feature is enabled.

4. RecordMetadata Changes

New fields added to RecordMetadata:

pub struct RecordMetadata {
    pub record_id: u32,           // ← NEW: Stable numeric identifier
    pub record_key: String,       // ← NEW: Human-readable key
    pub type_id: u64,
    pub type_name: String,
    pub buffer_type: String,
    pub buffer_capacity: Option<usize>,
    pub producer_count: usize,
    pub consumer_count: usize,
    pub outbound_connector_count: usize,
    pub inbound_connector_count: usize,
    #[cfg(feature = "metrics")]
    pub buffer_metrics: Option<BufferMetricsSnapshot>,  // ← NEW: Buffer metrics
}

📦 Published Crates

Updated to v0.3.0

✅ aimdb-core@0.3.0 - RecordId/RecordKey architecture + buffer metrics
✅ aimdb-tokio-adapter@0.3.0 - Buffer metrics implementation + multi-instance tests
✅ aimdb-embassy-adapter@0.3.0 - Explicit DynBuffer implementation + metrics feature flag
✅ aimdb-client@0.3.0 - Updated for new RecordMetadata fields
✅ aimdb-sync@0.3.0 - Updated for key-based registration API
✅ aimdb-mqtt-connector@0.3.0 - Updated for key-based registration + rumqttc 0.25
✅ aimdb-cli@0.3.0 - Updated formatters for RecordId/RecordKey display
✅ aimdb-mcp@0.3.0 - Updated tools for RecordId/RecordKey introspection

Updated to v0.2.0

✅ aimdb-knx-connector@0.2.0 - Updated for key-based registration (first stable release)

Unchanged

aimdb-executor@0.1.0 - No changes (still compatible)

🚀 Quick Start

Multi-Instance Records Example

use aimdb_core::{AimDbBuilder, buffer::BufferCfg};
use aimdb_tokio_adapter::TokioAdapter;
use serde::{Serialize, Deserialize};
use std::sync::Arc;

#[derive(Clone, Debug, Serialize, Deserialize)]
struct SensorReading {
    value: f32,
    timestamp: u64,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let runtime = Arc::new(TokioAdapter::new()?);
    let mut builder = AimDbBuilder::new().runtime(runtime);

    // Register multiple sensors of the same type
    builder.configure::<SensorReading>("sensors.temperature", |reg| {
        reg.buffer(BufferCfg::SpmcRing { capacity: 100 });
    });

    builder.configure::<SensorReading>("sensors.humidity", |reg| {
        reg.buffer(BufferCfg::SpmcRing { capacity: 100 });
    });

    builder.configure::<SensorReading>("sensors.pressure", |reg| {
        reg.buffer(BufferCfg::SingleLatest);
    });

    let db = builder.build().await?;

    // Key-based producers for each sensor
    let temp_producer = db.producer_by_key::<SensorReading>("sensors.temperature")?;
    let humidity_producer = db.producer_by_key::<SensorReading>("sensors.humidity")?;
    let pressure_producer = db.producer_by_key::<SensorReading>("sensors.pressure")?;

    // Each produces to its own record
    temp_producer.produce(SensorReading { value: 22.5, timestamp: 1000 }).await?;
    humidity_producer.produce(SensorReading { value: 65.0, timestamp: 1001 }).await?;
    pressure_producer.produce(SensorReading { value: 1013.25, timestamp: 1002 }).await?;

    // Introspection: Find all SensorReading records
    let sensor_ids = db.records_of_type::<SensorReading>();
    println!("Found {} sensor records", sensor_ids.len());  // Prints: Found 3 sensor records

    for record_id in sensor_ids {
        let metadata = db.record_metadata_by_id(*record_id)?;
        println!("  - {} ({})", metadata.record_key, metadata.buffer_type);
    }

    Ok(())
}

Buffer Metrics Example

use aimdb_core::{AimDbBuilder, buffer::BufferCfg};
use aimdb_tokio_adapter::TokioAdapter;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let runtime = Arc::new(TokioAdapter::new()?);
    let mut builder = AimDbBuilder::new().runtime(runtime);

    builder.configure::<String>("app.logs", |reg| {
        reg.buffer(BufferCfg::SpmcRing { capacity: 1000 });
    });

    let db = builder.build().await?;
    let producer = db.producer_by_key::<String>("app.logs")?;

    // Produce some data
    for i in 0..50 {
        producer.produce(format!("Log entry {}", i)).await?;
    }

    // Check metrics
    let metadata = db.record_metadata::<String>("app.logs")?;
    
    #[cfg(feature = "metrics")]
    if let Some(metrics) = metadata.buffer_metrics {
        println!("Buffer Metrics:");
        println!("  Produced: {}", metrics.produced_count);
        println!("  Consumed: {}", metrics.consumed_count);
        println!("  Dropped:  {}", metrics.dropped_count);
        println!("  Occupancy: {}/{}", metrics.occupancy.0, metrics.occupancy.1);
    }

    Ok(())
}

📚 Migration Guide

Step 1: Update Dependencies

[dependencies]
aimdb-core = "0.3.0"
aimdb-tokio-adapter = "0.3.0"
aimdb-mqtt-connector = "0.3.0"
aimdb-knx-connector = "0.2.0"  # Note: KNX connector at v0.2.0

# Optional: Enable metrics
# aimdb-core = { version = "0.3.0", features = ["metrics"] }
# aimdb-tokio-adapter = { version = "0.3.0", features = ["metrics"] }

Step 2: Add Keys to Record Registration

Find all .configure::<T>() calls and add a key parameter:

# Search for patterns to update
rg "\.configure::<" --type rust

Update each call:

// Before
builder.configure::<Temperature>(|reg| {
    reg.buffer(BufferCfg::SingleLatest);
});

// After - Add a descriptive key
builder.configure::<Temperature>("sensors.temperature", |reg| {
    reg.buffer(BufferCfg::SingleLatest);
});

Step 3: Handle Multi-Instance Scenarios

If you have multiple records of the same type, switch to key-based API:

// Before (v0.2.x) - Only one Temperature record allowed
let producer = db.producer::<Temperature>()?;

// After (v0.3.0) - Multiple Temperature records supported
let indoor_producer = db.producer_by_key::<Temperature>("sensors.indoor")?;
let outdoor_producer = db.producer_by_key::<Temperature>("sensors.outdoor")?;

Step 4: Update Custom Buffers (If Applicable)

If you've implemented custom Buffer<T> types, add explicit DynBuffer<T> implementation:

See "Breaking Changes" section above for implementation template.

Step 5: Optional - Add Metrics

Enable metrics feature and update monitoring code:

#[cfg(feature = "metrics")]
{
    let metadata = db.record_metadata::<MyType>("my.record")?;
    if let Some(metrics) = metadata.buffer_metrics {
        // Log or export metrics
        log::info!("Buffer produced: {}, consumed: {}, dropped: {}",
            metrics.produced_count,
            metrics.consumed_count,
            metrics.dropped_count
        );
    }
}

Step 6: Update Tests

Test code needs keys too:

// Before
#[tokio::test]
async fn test_producer() {
    let mut builder = AimDbBuilder::new().runtime(runtime);
    builder.configure::<Data>(|reg| reg.buffer(BufferCfg::Mailbox));
    let db = builder.build().await.unwrap();
    let producer = db.producer::<Data>().unwrap();
}

// After
#[tokio::test]
async fn test_producer() {
    let mut builder = AimDbBuilder::new().runtime(runtime);
    builder.configure::<Data>("test.data", |reg| reg.buffer(BufferCfg::Mailbox));
    let db = builder.build().await.unwrap();
    let producer = db.producer::<Data>().unwrap();  // Still works for single-instance
    // Or: let producer = db.producer_by_key::<Data>("test.data").unwrap();
}

🐛 Dependency Updates

rumqttc Upgrade (0.24 → 0.25)

The MQTT connector now uses rumqttc@0.25, which includes:

Improved connection stability
Better error handling
Enhanced TLS support

License Updates:

Added Zlib license allowance (used by foldhash, hashbrown's default hasher)
Added OpenSSL license allowance (transitive via rumqttc/rustls)
Ignored RUSTSEC-2025-0134 advisory (rustls-pemfile unmaintained but not vulnerable)

📖 Examples

All examples have been updated for v0.3.0:

git clone https://github.com/aimdb-dev/aimdb.git
cd aimdb

# Multi-instance record example
cargo run --example tokio-mqtt-connector-demo

# Buffer metrics example (requires --features metrics)
cargo build --features metrics
cargo run --example remote-access-demo --features metrics

# Embedded examples
cd examples/embassy-mqtt-connector-demo
cargo build --release

🎯 Performance Characteristics

Latency Targets (Maintained):

✅ Sub-50ms record access (O(1) via RecordId)
✅ Lock-free buffer operations
✅ Zero-copy type-safe routing

New Optimizations:

O(1) hot-path access (improved from O(log n) BTreeMap)
O(1) key resolution via HashMap
Zero-allocation static keys (&'static str)

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

Quick start:

git clone https://github.com/aimdb-dev/aimdb.git
cd aimdb
make check  # Format + clippy + test + embedded cross-compile

Test with metrics:

make test-metrics

📄 License

Licensed under Apache License 2.0 - see LICENSE for details.

🙏 Acknowledgments

Special thanks to:

Community members who requested multi-instance record support
Contributors who helped test the RecordId/RecordKey architecture
The Rust async ecosystem maintainers (Tokio, Embassy)

💬 Community

Issues: GitHub Issues
Discussions: GitHub Discussions

🎉 Upgrade Today!

# Update all AimDB dependencies to v0.3.0
cargo update

# Or specify v0.3.0 explicitly
cargo add aimdb-core@0.3.0 aimdb-tokio-adapter@0.3.0

# Enable metrics feature
cargo add aimdb-core@0.3.0 --features metrics

Build multi-instance, metrics-enabled data pipelines across your entire infrastructure!

📋 Release Checklist

For detailed technical changes, see individual crate changelogs:

v0.3.0