Wire Format Specification

This page is the normative description of every byte Tayra writes. It exists so that data encrypted by Tayra can be recovered with nothing but your key store and any AES-256-GCM implementation, in any language, with no Tayra assemblies, no license key, and no involvement from Radarleaf.

It documents decryption of existing data only. Nothing here describes how to produce new ciphertext, and you should not write your own encryptor: nonce management and context binding are easy to get wrong in ways that are silent until they are catastrophic. Recovery is a read operation.

This is a stability commitment

The format version byte is the compatibility mechanism. Existing payloads stay readable by the decoder that matches their version byte, and new versions are additive. A payload written today will decrypt with the rules on this page indefinitely.

What you need to decrypt

Three things, all of which are yours:

1. The ciphertext, as stored in your database.
2. The data encryption key (DEK) for the subject, from your key store.
3. The field context (entity type name, member name, group), which is bound into the authentication tag and must be reproduced exactly.

If you use envelope encryption you also need the master key, and the DEK must be unwrapped before use.

Format versions

Byte	Layout	Written by
0x01	[version][nonce][ciphertext][tag]	The no-AAD primitive AesGcmEncryptor.Encrypt only. Never emitted by field encryption.
0x02	[version][nonce][ciphertext][tag], AAD bound	The AAD primitive only. Never emitted by field encryption.
0x03	[version][key_version][nonce][ciphertext][tag]	Encrypted string fields, wrapped DEKs, and serialized fields written before 2.4.0.
0x04	Identical to 0x03	Serialized (non-string) fields. Differs from 0x03 only in the plaintext encoding it carries.
0xE1	Envelope wrapper	Envelope-encrypted DEK blobs in the key store.

0xE1 sits deliberately outside the 0x01 to 0x04 range so an envelope blob can be told apart from raw ciphertext by inspecting one byte.

Recovery code needs 0x03, 0x04 and, if you use master keys, 0xE1. It does not need 0x01 or 0x02, which no Tayra write path produces.

Payload layout: 0x03 and 0x04

These two are cryptographically identical. Same header, same AAD construction, same tag. They differ only in how the plaintext was encoded before encryption.

┌───────────┬──────────────┬──────────┬──────────────┬──────────────┐
│ Version   │ Key version  │ Nonce    │ Ciphertext   │ Auth tag     │
│ (1 byte)  │ (2 bytes BE) │ (12 B)   │ (N bytes)    │ (16 bytes)   │
└───────────┴──────────────┴──────────┴──────────────┴──────────────┘
  offset 0     1..2           3..14      15..15+N       last 16

Field	Size	Notes
Version	1 byte	0x03 or 0x04.
Key version	2 bytes	Unsigned big-endian. Selects which rotation of the DEK to fetch. 0 means the unversioned base key.
Nonce	12 bytes	Random per operation, 96 bits as recommended for GCM.
Ciphertext	N bytes	Same length as the plaintext.
Auth tag	16 bytes	128-bit GCM tag.

Fixed overhead is 31 bytes. Most AEAD libraries (including Python's cryptography and Go's crypto/cipher) expect the tag appended to the ciphertext, which is exactly this layout, so you can pass ciphertext || tag as one slice.

Legacy 0x01 and 0x02 layout

Identical minus the key version field: [version][nonce(12)][ciphertext(N)][tag(16)], 29 bytes of overhead, ciphertext starting at offset 13. Documented for completeness only.

Associated data

Versions 0x02 and above bind context into the authentication tag. AAD is authenticated but not encrypted, and a decryptor that does not reconstruct it byte for byte gets an authentication failure with no other diagnostic. This is the part that cannot be guessed, and the most common reason a hand-written decryptor fails.

For field encryption, the AAD is this UTF-8 string:

tayra:aad:v1|type:{TypeFullName}|field:{MemberName}|group:{Group}

• {TypeFullName} is the .NET type's FullName (namespace plus type name) of the object that owned the field, falling back to the simple name if FullName is null.
• {MemberName} is the property or field name carrying the attribute.
• {Group} is the field's group, or the empty string when the field has no group. The separator is always present, so an ungrouped field ends in a trailing group:.

For 0x03 and 0x04, the 2-byte big-endian key version is then appended to those UTF-8 bytes to form the final AAD. This is what protects the key version from tampering:

final_aad = utf8("tayra:aad:v1|type:...|field:...|group:...") || key_version_be16

Ciphertext is bound to .NET type identity

The AAD embeds the CLR type full name and member name recorded at encrypt time. Renaming a class, moving it to another namespace, or renaming a member changes the AAD and breaks decryption of data already written. Recovery outside .NET therefore requires knowing the original type and member names, so keep them alongside your backups if the code may not survive.

Plaintext encoding

Once the AEAD layer is peeled off, what you have depends on the version and the field.

Source	Version	Stored as	Plaintext
[PersonalData] on a string	0x03	Base64 string in the same property	UTF-8 bytes of the original string
[SerializedPersonalData] on any type	0x04	Raw byte[]	UTF-8 JSON
[SerializedPersonalData], written before 2.4.0	0x03	Raw byte[]	Legacy binary encoding

String fields are Base64-encoded after encryption, so decode Base64 before reading the version byte. Byte-array fields are stored raw.

The JSON in a 0x04 payload is produced by System.Text.Json with pinned options: not indented, nulls written rather than skipped, strict number handling, and enums written as numbers. A null value serializes to zero bytes, so an empty plaintext means null.

Dispatch on the version byte, not on the field type: a field re-encrypted after an upgrade moves from 0x03 to 0x04, so both can coexist in one column.

Masking prefix

A field configured with a masking strategy does not store bare Base64. It stores the redacted display value first, then a newline, then the payload:

TAYRA_M:{redacted}\n{base64_payload}

If a stored string starts with TAYRA_M:, strip everything up to and including the first \n before Base64-decoding. Miss this and Base64 decoding fails on data that is otherwise perfectly intact.

Key identification

Key ids in the store are built from the subject:

{prefix}{subjectId}          when the subject has no group
{prefix}{subjectId}:{group}  when it does

Rotation appends a version suffix. Version 0 is the unversioned base id, and every later rotation is explicit:

{baseKeyId}        version 0
{baseKeyId}:v1     version 1
{baseKeyId}:v2     version 2

Take the key version from bytes 1 and 2 of the payload and resolve the matching id.

You do not have to derive ids to recover data. The key store holds them as literal strings, so enumerating the store is authoritative and immune to any mistake in reproducing the derivation.

Keys prefixed bi: are HMAC keys for blind indexes. They never decrypt anything. Blind indexes are one-way by construction and are not recoverable to plaintext.

Key store layout

Every key store holds the same logical mapping of key id to key bytes. The PostgreSQL store uses:

CREATE TABLE tayra_keys
(
    key_id      VARCHAR(255) NOT NULL,
    secret_key  BYTEA        NOT NULL,
    created_at  TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    PRIMARY KEY (key_id)
);

secret_key holds the DEK. Without envelope encryption those are the raw AES-256 key bytes, usable directly. With envelope encryption they are 0xE1 blobs and must be unwrapped first. Check the first byte to tell which.

The table name is configurable, so confirm yours against your own configuration.

Envelope format: 0xE1

When a master key is configured, DEKs are themselves encrypted before storage.

┌───────────┬──────────┬──────────────┬───────────────────┬──────────────────┐
│ 0xE1      │ Flags    │ Id length    │ Master key id     │ Wrapped DEK      │
│ (1 byte)  │ (2 B)    │ (2 bytes BE) │ (UTF-8, L bytes)  │ (0x03 payload)   │
└───────────┴──────────┴──────────────┴───────────────────┴──────────────────┘
  offset 0     1..2       3..4           5..5+L              5+L..end

Flags are reserved and written as 0x0000. The wrapped DEK is an ordinary 0x03 payload carrying the master key's own version in its key version field, so it decodes with the rules above once you have the master key bytes.

Its AAD is a different string:

tayra:envelope:v1|{masterKeyId}|{keyId}

again with the 2-byte big-endian master key version appended. {keyId} is the id of the DEK being wrapped, which binds each wrapped key to its own slot and makes a cross-tenant splice fail.

Unwrapping is therefore a two-stage decrypt: master key opens the envelope to yield the DEK, and the DEK opens the field payload.

Decryption procedure

1. Read the stored value. If it is a string starting with TAYRA_M:, drop through the first newline. If it is a string, Base64-decode it.
2. Read byte 0. Expect 0x03 or 0x04.
3. Read the key version from bytes 1 and 2, big-endian.
4. Fetch the key for that subject and version from the key store. If its first byte is 0xE1, unwrap it with the master key as described above.
5. Rebuild the AAD string for the field and append the 2-byte key version.
6. Nonce is bytes 3 to 14. Ciphertext and tag are byte 15 to the end.
7. AES-256-GCM decrypt with a 16-byte tag.
8. Interpret the plaintext: UTF-8 text for 0x03 string fields, UTF-8 JSON for 0x04.

An authentication failure at step 7 means the key, the key version, or the AAD is wrong. It is far more often the AAD than the key.

Reference implementations

Complete working decryptors in two languages. Neither depends on any Tayra component: the .NET one uses only the base class library, the Python one only the cryptography package. Copy either into a recovery tool as-is.

Both are maintained as single runnable files (a .NET 10 file-based app needing no project or restore, and a Python 3 script) and both are executed on every CI build against the test vectors, so the code printed here cannot drift from the format.

Fetching the DEK is a single lookup against your key store, for example SELECT secret_key FROM tayra_keys WHERE key_id = $1.

Unwrap the DEK first if your key store holds envelope blobs, then decrypt each field value with it.

Python

Requires only the cryptography package. Being outside .NET entirely, it is the clearest demonstration that the format stands on its own.

import base64

from cryptography.hazmat.primitives.ciphers.aead import AESGCM

FORMAT_V3 = 0x03        # string fields, wrapped DEKs, legacy serialized fields
FORMAT_V4 = 0x04        # serialized fields (JSON plaintext)
ENVELOPE_MARKER = 0xE1  # envelope-wrapped DEK blob

NONCE_SIZE = 12
TAG_SIZE = 16
KEY_VERSION_SIZE = 2
HEADER_SIZE = 1 + KEY_VERSION_SIZE + NONCE_SIZE  # version + key version + nonce

MASKING_PREFIX = "TAYRA_M:"


def decrypt_field(stored, dek, type_full_name, member_name, group=""):
    """Decrypt one Tayra-encrypted field value.

    `stored` is either the string held in the entity (Base64, optionally carrying the TAYRA_M:
    masking prefix) or the raw bytes of a serialized field. Pass the CLR type full name and member
    name recorded at encrypt time, plus the field group (empty string when the field has no group).

    Returns UTF-8 text: the original string for a v3 payload, JSON for a v4 payload.
    """
    if isinstance(stored, str):
        # A masked field stores the redacted display value, a newline, then the payload.
        if stored.startswith(MASKING_PREFIX):
            stored = stored.split("\n", 1)[1]
        stored = base64.b64decode(stored)

    context = f"tayra:aad:v1|type:{type_full_name}|field:{member_name}|group:{group}"
    return _decrypt(stored, dek, _build_aad(context, stored)).decode("utf-8")


def unwrap_dek(stored_key, master_key, key_id):
    """Unwrap an envelope-encrypted DEK.

    Key store values that do not begin with 0xE1 are already raw DEKs and are returned unchanged,
    so this is safe to call unconditionally.
    """
    if not stored_key or stored_key[0] != ENVELOPE_MARKER:
        return stored_key

    # [0xE1][flags(2)][master_key_id_len(2 BE)][master_key_id(UTF-8)][wrapped DEK: v3 payload]
    id_length = int.from_bytes(stored_key[3:5], "big")
    master_key_id = stored_key[5:5 + id_length].decode("utf-8")
    wrapped = stored_key[5 + id_length:]

    context = f"tayra:envelope:v1|{master_key_id}|{key_id}"
    return _decrypt(wrapped, master_key, _build_aad(context, wrapped))


def read_key_version(payload):
    """Read the key version from bytes 1 and 2, which selects the DEK rotation to fetch."""
    return -1 if len(payload) < 3 else int.from_bytes(payload[1:3], "big")


def _build_aad(context, payload):
    """Append the 2-byte big-endian key version to the UTF-8 context string.

    Reproduce this exactly: a mismatched AAD fails authentication with no other diagnostic.
    """
    return context.encode("utf-8") + payload[1:3]


def _decrypt(payload, key, aad):
    if len(payload) < HEADER_SIZE + TAG_SIZE:
        raise ValueError("Payload is too short to be valid Tayra ciphertext.")

    version = payload[0]
    if version not in (FORMAT_V3, FORMAT_V4):
        raise ValueError(f"Unsupported format version 0x{version:02x}. Expected 0x03 or 0x04.")

    # [version(1)][key_version(2)][nonce(12)][ciphertext(N)][tag(16)]
    # AESGCM.decrypt expects the tag appended to the ciphertext, which is this layout exactly.
    nonce = payload[1 + KEY_VERSION_SIZE:HEADER_SIZE]
    return AESGCM(key).decrypt(nonce, payload[HEADER_SIZE:], aad)

^anchor

.NET

Uses System.Security.Cryptography.AesGcm from the base class library, with no reference to any Tayra package.

using System.Security.Cryptography;
using System.Text;

/// <summary>
/// Decrypts Tayra-encrypted values using nothing but the .NET base class library.
/// </summary>
public static class TayraRecovery
{
    private const byte FormatV3 = 0x03;       // string fields, wrapped DEKs, legacy serialized fields
    private const byte FormatV4 = 0x04;       // serialized fields (JSON plaintext)
    private const byte EnvelopeMarker = 0xE1; // envelope-wrapped DEK blob

    private const int NonceSize = 12;
    private const int TagSize = 16;
    private const int KeyVersionSize = 2;
    private const int HeaderSize = 1 + KeyVersionSize + NonceSize; // version + key version + nonce

    private const string MaskingPrefix = "TAYRA_M:";

    /// <summary>
    /// Decrypts a string field as stored in the entity, handling the Base64 wrapper and the
    /// <c>TAYRA_M:</c> masking prefix. Pass the CLR type full name and member name recorded at
    /// encrypt time, plus the field group (the empty string when the field has no group).
    /// </summary>
    public static string DecryptStringField(
        string stored,
        byte[] dek,
        string typeFullName,
        string memberName,
        string group = "")
    {
        ArgumentNullException.ThrowIfNull(stored);

        // A masked field stores the redacted display value, a newline, then the payload.
        if (stored.StartsWith(MaskingPrefix, StringComparison.Ordinal))
        {
            var newline = stored.IndexOf('\n');
            if (newline < 0)
            {
                throw new InvalidOperationException(
                    "Masked value is missing the newline separating the redaction from the payload.");
            }

            stored = stored[(newline + 1)..];
        }

        var plaintext = DecryptPayload(
            Convert.FromBase64String(stored), dek, typeFullName, memberName, group);

        return Encoding.UTF8.GetString(plaintext);
    }

    /// <summary>
    /// Decrypts a serialized (non-string) field, stored as a raw byte array. For a v4 payload the
    /// result is UTF-8 JSON.
    /// </summary>
    public static byte[] DecryptSerializedField(
        byte[] stored,
        byte[] dek,
        string typeFullName,
        string memberName,
        string group = "")
        => DecryptPayload(stored, dek, typeFullName, memberName, group);

    /// <summary>
    /// Unwraps an envelope-encrypted DEK. Key store values that do not begin with <c>0xE1</c> are
    /// already raw DEKs and are returned unchanged, so this is safe to call unconditionally.
    /// </summary>
    public static byte[] UnwrapDek(byte[] storedKey, byte[] masterKey, string keyId)
    {
        ArgumentNullException.ThrowIfNull(storedKey);

        if (storedKey.Length == 0 || storedKey[0] != EnvelopeMarker)
        {
            return storedKey;
        }

        // [0xE1][flags(2)][master_key_id_len(2 BE)][master_key_id(UTF-8)][wrapped DEK: v3 payload]
        var idLength = (storedKey[3] << 8) | storedKey[4];
        var masterKeyId = Encoding.UTF8.GetString(storedKey, 5, idLength);
        var wrapped = storedKey.AsSpan(5 + idLength).ToArray();

        var context = $"tayra:envelope:v1|{masterKeyId}|{keyId}";
        return Decrypt(wrapped, masterKey, BuildAad(context, wrapped));
    }

    /// <summary>
    /// Reads the key version from bytes 1 and 2, which selects the DEK rotation to fetch.
    /// </summary>
    public static int ReadKeyVersion(byte[] payload)
    {
        ArgumentNullException.ThrowIfNull(payload);
        return payload.Length < 3 ? -1 : (payload[1] << 8) | payload[2];
    }

    private static byte[] DecryptPayload(
        byte[] payload, byte[] dek, string typeFullName, string memberName, string group)
    {
        var context = $"tayra:aad:v1|type:{typeFullName}|field:{memberName}|group:{group}";
        return Decrypt(payload, dek, BuildAad(context, payload));
    }

    // The 2-byte big-endian key version is appended to the UTF-8 context string. Reproduce this
    // exactly: a mismatched AAD fails authentication with no other diagnostic.
    private static byte[] BuildAad(string context, byte[] payload)
    {
        var contextBytes = Encoding.UTF8.GetBytes(context);
        var aad = new byte[contextBytes.Length + KeyVersionSize];
        contextBytes.CopyTo(aad, 0);
        aad[^2] = payload[1];
        aad[^1] = payload[2];
        return aad;
    }

    private static byte[] Decrypt(byte[] payload, byte[] key, byte[] aad)
    {
        if (payload.Length < HeaderSize + TagSize)
        {
            throw new InvalidOperationException("Payload is too short to be valid Tayra ciphertext.");
        }

        var version = payload[0];
        if (version is not (FormatV3 or FormatV4))
        {
            throw new InvalidOperationException(
                $"Unsupported format version 0x{version:x2}. Expected 0x03 or 0x04.");
        }

        // [version(1)][key_version(2)][nonce(12)][ciphertext(N)][tag(16)]
        var ciphertextLength = payload.Length - HeaderSize - TagSize;
        var nonce = payload.AsSpan(1 + KeyVersionSize, NonceSize);
        var ciphertext = payload.AsSpan(HeaderSize, ciphertextLength);
        var tag = payload.AsSpan(HeaderSize + ciphertextLength, TagSize);

        var plaintext = new byte[ciphertextLength];
        using var aes = new AesGcm(key, TagSize);
        aes.Decrypt(nonce, ciphertext, tag, plaintext, aad);
        return plaintext;
    }
}

^anchor

Note the difference in tag handling

Python's AESGCM.decrypt expects the tag appended to the ciphertext, which matches the wire layout directly. .NET's AesGcm.Decrypt takes ciphertext and tag as separate spans, so the payload has to be split. Getting this wrong is the second most common cause of an authentication failure, after a mismatched AAD.

Test vectors

Known-good ciphertext produced by Tayra's own encryptor, with the keys and expected plaintexts beside it. Use it to validate a reimplementation in any language: unwrap the envelope, confirm you get the published DEK, then decrypt each field and compare against expected.

The same vectors are asserted by Tayra's test suite and by both reference implementations on every CI build, so agreement here means your implementation agrees with the encryptor.

dek, masterKey and envelope are Base64. So is stored, except on the masked vector, which carries its TAYRA_M: prefix and newline verbatim as the entity would hold it. storedIsBase64String records how the value lives in the entity: true for a string field held as Base64, false for a serialized field held as raw bytes and shown Base64-encoded here only so it can be written down.

These keys are fixed test data

They protect nothing and must never be used for anything.

{
  "comment": "Canonical test vectors for the Tayra wire format. The keys here are fixed test data: they protect nothing and must never be reused.",
  "dek": "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8=",
  "masterKey": "oKGio6SlpqeoqaqrrK2ur7CxsrO0tba3uLm6u7y9vr8=",
  "keyId": "cust:42",
  "masterKeyId": "master-eu",
  "masterKeyVersion": 3,
  "envelope": "4QAAAAltYXN0ZXItZXUDAAO5DFY/1bQ0ur+nVjbsbOm+UvUE5Ae1OnXCi6EJ6a6lzeVbOJxL2P6/SDDcZU1wAowDDTWhp8ciD7rXio8=",
  "fields": [
    {
      "name": "v3 string field",
      "formatVersion": 3,
      "typeFullName": "Acme.Billing.Customer",
      "memberName": "Email",
      "group": "",
      "keyVersion": 7,
      "stored": "AwAHBlmlATaie0KmzZAO/7uKCRwagfC5FCFoV85J1CFpt9nf9TScHEe/+oJ27g==",
      "storedIsBase64String": true,
      "expected": "ada@example.com"
    },
    {
      "name": "v3 string field with a group",
      "formatVersion": 3,
      "typeFullName": "Acme.Billing.Customer",
      "memberName": "Iban",
      "group": "billing",
      "keyVersion": 0,
      "stored": "AwAATpjcCKa6aCiWE74SiEhIFqWjjx3btoJMCr2t98F8xv81BwjC25VfSuQn/HKsRp/KGR8=",
      "storedIsBase64String": true,
      "expected": "GB33BUKB20201555555555"
    },
    {
      "name": "v3 string field behind a masking prefix",
      "formatVersion": 3,
      "typeFullName": "Acme.Billing.Customer",
      "memberName": "Email",
      "group": "",
      "keyVersion": 7,
      "stored": "TAYRA_M:a**@example.com\nAwAHBlmlATaie0KmzZAO/7uKCRwagfC5FCFoV85J1CFpt9nf9TScHEe/+oJ27g==",
      "storedIsBase64String": true,
      "expected": "ada@example.com"
    },
    {
      "name": "v4 serialized field (JSON plaintext)",
      "formatVersion": 4,
      "typeFullName": "Acme.Billing.Customer",
      "memberName": "Address",
      "group": "",
      "keyVersion": 7,
      "stored": "BAAHzQr5zUFPJf0TSYJW6D7R4HgcwzBqDKhMCmM6Zkbofe4b5UMjhLgF2Ap0dJqjxV38XDgMqQsyuGVklSzoDHKW3lEHAkQhSv9c8P9X",
      "storedIsBase64String": false,
      "expected": "{\"Line1\":\"22 Acacia Ave\",\"Postcode\":\"SW1A 1AA\"}"
    }
  ]
}

^anchor

Between them these cover a v3 string field, a grouped field (which pins the non-empty group: case in the AAD), a masked field, a v4 serialized field, and an envelope-wrapped DEK.

Not covered here

• Blind indexes. HMAC-SHA256 and one-way by design. They support search, never recovery.
• Encryption. Decrypt only, as above.
• Audit records and inventory data. Not encrypted, so no format is needed to read them.

See also

• Encryption for the design rationale behind these choices
• Key Store for how keys are stored and rotated
• Crypto Engine for key resolution and caching