Crypto Engine

The crypto engine manages the key lifecycle - creation, retrieval, caching, rotation, and deletion. Users interact with these operations through ITayra; ICryptoEngine is a public but low-level interface that applications rarely use directly.

User-Facing Operations on ITayra

ITayra Method	What Happens Internally
`EncryptAsync<T>()`	Gets or creates a key via the engine, then encrypts
`DecryptAsync<T>()`	Retrieves a key (returns replacements if shredded)
`ShredAsync(subjectId)`	Deletes the subject's base key plus all rotated versions and group keys - data becomes permanently unreadable
`ShredByPrefixAsync(prefix)`	Bulk deletes all matching keys
`RotateKeyAsync(keyIdBase)`	Creates a new versioned key, preserves the old one
`ReEncryptAsync<T>()`	Decrypts with old key, re-encrypts with new key
`KeyExistsAsync(keyId)`	Checks if a key is still active
`ListKeysAsync(prefix)`	Lists all key IDs matching a prefix

How It Works

Internal implementation - provided for clarity

ICryptoEngine Interface

/// <summary>
/// Central abstraction for key lifecycle management. Composes key generation,
/// storage (<see cref="IKeyStore"/>), and encryption/decryption operations.
/// </summary>
public interface ICryptoEngine
{
    /// <summary>
    /// Gets an existing key or generates and stores a new one for the given key ID.
    /// </summary>
    Task<byte[]> GetOrCreateKeyAsync(string keyId, CancellationToken ct = default);

    /// <summary>
    /// Gets an existing key. Returns null if the key does not exist (shredded).
    /// </summary>
    Task<byte[]?> GetKeyAsync(string keyId, CancellationToken ct = default);

    /// <summary>
    /// Warms the in-memory key cache for the given resolved key IDs in a single batched round trip
    /// to the key store, so a subsequent per-document decrypt path hits the cache instead of the store.
    /// Intended for large Marten projection rebuilds that would otherwise issue one key lookup per subject.
    /// </summary>
    /// <remarks>
    /// Best-effort: prefetch is a pure optimization. A key-store failure here must NOT throw or break the
    /// caller's rebuild; the authoritative per-key path remains fail-closed and will surface any real error.
    /// The default implementation is a no-op so engines that do not implement it simply skip warming.
    /// </remarks>
    Task PrefetchKeysAsync(IReadOnlyCollection<string> keyIds, CancellationToken ct = default)
        => Task.CompletedTask;

    /// <summary>
    /// Evicts the given resolved key IDs from the in-memory key cache, without touching the key store.
    /// This is the inverse of <see cref="PrefetchKeysAsync"/>: it lets a bounded, rolling-window prefetch
    /// release keys it warmed ahead of a large rebuild once the daemon has moved past them, so the warmed
    /// working set stays bounded for rebuilds spanning tens of millions of subjects.
    /// </summary>
    /// <remarks>
    /// Cache-only and synchronous (an in-memory operation, no I/O). This is NOT crypto-shredding: it never
    /// deletes a key from the store, so an evicted key is simply re-loaded on next access. Use
    /// <see cref="DeleteKeyAsync"/> to actually delete a key. The default implementation is a no-op so
    /// engines that keep no cache simply skip it.
    /// </remarks>
    /// <param name="keyIds">The resolved key IDs to remove from the cache.</param>
    void EvictCachedKeys(IReadOnlyCollection<string> keyIds)
    {
    }

    /// <summary>
    /// Gets the latest versioned key for the given base key ID, including its version number.
    /// Used by the encrypt path to stamp the key version into the ciphertext.
    /// Falls back to the base key with version 0 if no versioned keys exist.
    /// </summary>
    Task<ResolvedKey> GetOrCreateLatestKeyAsync(string keyIdBase, CancellationToken ct = default)
        => throw new NotSupportedException();

    /// <summary>
    /// Gets a specific versioned key for decryption. Version 0 means the base (unversioned) key.
    /// Returns null if the key does not exist (shredded).
    /// </summary>
    Task<byte[]?> GetKeyByVersionAsync(string keyIdBase, int version, CancellationToken ct = default)
        => throw new NotSupportedException();

    /// <summary>
    /// Deletes a key - this IS the crypto-shredding operation.
    /// After deletion, any data encrypted with this key becomes permanently unreadable.
    /// </summary>
    Task DeleteKeyAsync(string keyId, CancellationToken ct = default);

    /// <summary>
    /// Checks whether a key exists in the store.
    /// </summary>
    Task<bool> KeyExistsAsync(string keyId, CancellationToken ct = default);

    /// <summary>
    /// Deletes all keys matching the given subject prefix.
    /// Used for bulk crypto-shredding across all groups for a data subject.
    /// </summary>
    Task DeleteAllKeysAsync(string subjectPrefix, CancellationToken ct = default);

    /// <summary>
    /// Rotates the key for the given key ID base by creating a new versioned key.
    /// Returns the new key ID. The old key is preserved for decrypting existing data.
    /// </summary>
    Task<string> RotateKeyAsync(string keyIdBase, CancellationToken ct = default)
        => throw new NotSupportedException("This crypto engine does not support key rotation.");
}

^anchor

The built-in DefaultCryptoEngine wraps an IKeyStore with an in-memory MemoryCache.

Key Retrieval Flow

GetOrCreateKeyAsync("cust-abc123")
    │
    ├─ Check MemoryCache
    │   ├─ HIT → return cached key
    │   └─ MISS ↓
    │
    ├─ Acquire per-key-id lock (serializes in-process creation)
    │
    ├─ Call IKeyStore.GetAsync("cust-abc123")
    │   ├─ Key exists → cache it, return
    │   └─ Key not found ↓
    │
    ├─ Generate new AES key (RandomNumberGenerator)
    ├─ Call IKeyStore.StoreAsync("cust-abc123", newKey)
    ├─ Read back: IKeyStore.GetAsync("cust-abc123")
    │   └─ StoreAsync is first-writer-wins, so the store's bytes
    │      are authoritative (a concurrent creator may have won)
    ├─ Cache the stored key
    └─ Return the stored key

The miss path is serialized per key ID within the process, and the post-store read-back covers cross-process races: if another instance stored the key first, the engine uses that key rather than the one it generated locally. This guarantees data is never encrypted with a key that was not actually persisted. The same store-then-read-back pattern is applied during key rotation and by the blind-index HMAC key provider.

Decryption Flow

GetKeyAsync("cust-abc123")
    │
    ├─ Check MemoryCache
    │   ├─ HIT → return cached key
    │   └─ MISS ↓
    │
    ├─ Call IKeyStore.GetAsync("cust-abc123")
    │   ├─ Key exists → cache it, return
    │   └─ Key not found → return null (key was shredded)
    │
    └─ null triggers replacement value logic

Cache Behavior

Cache duration is controlled by TayraOptions.KeyCacheDuration (default: 5 minutes).
ShredAsync evicts the subject's keys from the local cache immediately, then deletes them from the key store. It performs two deletes: an exact-match delete of the base key, followed by a prefix delete of {subjectId}: that removes rotated key versions and group keys.
ShredByPrefixAsync delegates to the engine's DeleteAllKeysAsync, which lists matching key IDs (where the store supports ListKeyIdsAsync) and evicts them from the cache before calling IKeyStore.DeleteByPrefixAsync. If the store cannot list keys, cached copies expire naturally.

Key Rotation

RotateKeyAsync creates a new versioned key:

Discovers existing versioned keys via IKeyStore.ListKeyIdsAsync.
Determines the current maximum version number.
Generates a new key with the next version (e.g., cust-abc123:v2).
Stores and caches the new key.
Returns a KeyRotationResult with old and new key IDs.

The old key is preserved so that existing data can still be decrypted. Call ReEncryptAsync<T>() to migrate data to the new key.

Telemetry

The engine emits:

OpenTelemetry activities via TayraActivitySource for each operation.
Metrics via TayraMetrics: cache hits, cache misses, key store latency, keys created, keys deleted.
Audit events via ITayraAuditLogger for key creation, access, deletion, and bulk deletion.

Distributed Deployments

In a multi-instance deployment, calling ShredAsync on one instance evicts the key from that instance's cache. Other instances will continue using their cached copy until it expires. Keep KeyCacheDuration short (1-5 minutes) to minimize this window.

Crypto Engine ​

User-Facing Operations on ITayra ​

How It Works ​

ICryptoEngine Interface ​

Key Retrieval Flow ​

Decryption Flow ​

Cache Behavior ​

Key Rotation ​

Telemetry ​

See Also ​