coracle-rust/book/03-encryption.md

Encryption

Nostr is publicity technology. Every event a client publishes is broadcast, in the clear, to every relay it talks to, and from there to anyone who cares to subscribe. That's a feature — it's what makes nostr resilient — but it means that two people who want a private conversation have to arrange for privacy themselves, on top of a medium that was designed to have none.

An older format, NIP-04, is still in use in some places on the network — unauthenticated AES-256-CBC over the raw ECDH x-coordinate, with the IV appended to a base64 blob. It has a failure mode that NIP-44 does not (a corrupted ciphertext is indistinguishable from a wrong key), and it leaks plaintext length. It has been deprecated, and so this library does not implement it.

The modern standard for doing that is NIP-44 v2: it uses authenticated encryption, pads messages to hide their length, and versions itself so the scheme can evolve without breaking old ciphertexts. It's what this chapter covers and what new code should reach for.

The module

pub mod encryption;

//! NIP-44 v2 encryption for nostr events.
//!
//! Starts from an ECDH shared secret between a sender's
//! [`SecretKey`](crate::keys::SecretKey) and a recipient's
//! [`PublicKey`](crate::keys::PublicKey), feeds it through HKDF to derive a
//! reusable [`nip44::ConversationKey`], and then derives per-message keys from
//! there.

use std::fmt;

use base64::engine::general_purpose::STANDARD as BASE64;
use base64::Engine;
use chacha20::cipher::{KeyIvInit, StreamCipher};
use chacha20::ChaCha20;
use hkdf::Hkdf;
use hmac::{Hmac, Mac};
use rand::RngCore;
use sha2::Sha256;

use crate::keys::{PublicKey, SecretKey};

Errors

Encryption has a handful of distinct failure modes that a caller might reasonably want to distinguish: a corrupted payload, a MAC mismatch, a message that decoded to non-UTF-8 bytes. One enum covers them all.

/// Errors that can occur when encrypting or decrypting a nostr payload.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum EncryptionError {
    /// The payload was not well-formed: bad base64, wrong length, missing
    /// separator, or similar structural issues.
    InvalidPayload,
    /// The NIP-44 version byte was not one this library understands.
    UnsupportedVersion(u8),
    /// The NIP-44 HMAC did not match the ciphertext — either the payload was
    /// tampered with or the wrong conversation key was used.
    InvalidMac,
    /// The NIP-44 padding was malformed: a bad length prefix or non-zero
    /// padding bytes.
    InvalidPadding,
    /// The plaintext was empty. NIP-44 forbids zero-length messages.
    MessageEmpty,
    /// The plaintext exceeded NIP-44's 65535-byte limit.
    MessageTooLong,
    /// The decrypted bytes were not valid UTF-8.
    InvalidUtf8,
}

impl fmt::Display for EncryptionError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            EncryptionError::InvalidPayload => write!(f, "invalid encrypted payload"),
            EncryptionError::UnsupportedVersion(v) => {
                write!(f, "unsupported NIP-44 version byte: {v:#04x}")
            }
            EncryptionError::InvalidMac => write!(f, "HMAC verification failed"),
            EncryptionError::InvalidPadding => write!(f, "invalid NIP-44 padding"),
            EncryptionError::MessageEmpty => write!(f, "message is empty"),
            EncryptionError::MessageTooLong => write!(f, "message exceeds 65535 bytes"),
            EncryptionError::InvalidUtf8 => write!(f, "decrypted bytes are not valid UTF-8"),
        }
    }
}

impl std::error::Error for EncryptionError {}

ECDH

Given my secret key and your public key, I multiply them on the secp256k1 curve to get a point that only the two of us can compute — you can reach the same point with your secret key and my public key, because a·G·b == b·G·a. That point, reduced to its 32-byte x-coordinate, is the shared secret NIP-44 builds on.

There are two ways to spell this with the secp256k1 crate. SharedSecret::new returns a SHA-256 hash of the x-coordinate, which is a sensible default for general-purpose ECDH — but it is not what NIP-44 specifies. We want the raw x-coordinate, so we reach for shared_secret_point, which returns the full 64-byte point, and take the first 32 bytes.

There's one wrinkle: our PublicKey is x-only, but shared_secret_point wants a full secp256k1::PublicKey with a parity byte. We pick the even-parity lift — prepend 0x02 to the 32-byte x-coordinate — which matches every other nostr implementation. ECDH only cares about the x-coordinate of the resulting point, so the choice of input parity doesn't affect the output.

/// Compute the raw 32-byte ECDH shared secret between two nostr keys.
///
/// This is the x-coordinate of `sk · pk` on secp256k1. It is *not* hashed;
/// NIP-44 feeds it through HKDF before using it as key material.
pub fn shared_secret(sk: &SecretKey, pk: &PublicKey) -> [u8; 32] {
    // Lift the x-only pubkey to a full secp256k1 point by prepending the
    // even-parity byte. The ECDH output's x-coordinate is independent of the
    // chosen parity, so this always matches what the counterparty computes.
    let mut lifted = [0u8; 33];
    lifted[0] = 0x02;
    lifted[1..].copy_from_slice(&pk.as_bytes());
    let full_pk = secp256k1::PublicKey::from_slice(&lifted)
        .expect("even-parity lift of a valid x-only key is always a valid point");

    let point = secp256k1::ecdh::shared_secret_point(&full_pk, sk.as_secp256k1());
    let mut out = [0u8; 32];
    out.copy_from_slice(&point[..32]);
    out
}

The call to sk.as_secp256k1() is a small accessor we need to add to SecretKey. It returns a reference to the wrapped secp256k1::SecretKey without handing the bytes out in any form that's easier to misuse than what's already available.

impl SecretKey {
    /// Borrow the wrapped `secp256k1::SecretKey` for use with lower-level APIs.
    ///
    /// Exposed so that other modules in this crate (signing, encryption) can
    /// feed the key into `secp256k1` primitives without an extra copy.
    pub(crate) fn as_secp256k1(&self) -> &secp256k1::SecretKey {
        &self.0
    }
}

NIP-44 v2

NIP-44 v2 encrypts with ChaCha20, authenticates with HMAC-SHA256, and pads the plaintext to a fixed set of sizes so length analysis can't tell a one-word reply from a short paragraph. It layers HKDF on top of the ECDH shared secret twice: once to derive a conversation key from the key pair, and once per message to derive fresh ChaCha20 and HMAC keys from that conversation key and a random nonce.

The wire format

A NIP-44 v2 payload is the base64 encoding of this byte sequence:

Offset	Length	Field
0	1	version byte, always 0x02 for v2
1	32	nonce (random, also feeds per-message key derivation)
33	≥	ciphertext (padded plaintext)
—	32	HMAC-SHA256 over nonce ‖ ciphertext

The nonce is transmitted unencrypted and authenticated by the HMAC. The HMAC covers both the nonce and the ciphertext, so flipping either breaks verification.

ConversationKey

The expensive part of NIP-44 is the ECDH plus the first HKDF extract. It depends only on the key pair, so it can be computed once and reused for every message in a conversation. We expose it as a dedicated type.

pub mod nip44 {
    //! NIP-44 v2 authenticated encryption.
    //!
    //! The hot path is:
    //!
    //! 1. Derive a [`ConversationKey`] from an `(sk, pk)` pair — this runs
    //!    ECDH plus HKDF-Extract and is the expensive step.
    //! 2. For each message, call [`ConversationKey::encrypt`] /
    //!    [`ConversationKey::decrypt`], which runs HKDF-Expand with a fresh
    //!    random nonce, encrypts with ChaCha20, and signs with HMAC-SHA256.
    //!
    //! The free [`encrypt`] and [`decrypt`] functions are one-shot helpers
    //! that derive a conversation key, use it once, and throw it away.

    use super::*;

    const VERSION: u8 = 0x02;
    const MIN_PLAINTEXT_LEN: usize = 1;
    const MAX_PLAINTEXT_LEN: usize = 65535;
    const NONCE_LEN: usize = 32;
    const MAC_LEN: usize = 32;
    const HKDF_SALT: &[u8] = b"nip44-v2";

    type HmacSha256 = Hmac<Sha256>;

HKDF-Extract with salt "nip44-v2" and the ECDH output as input keying material is — by definition — HMAC-SHA256 over the shared secret, keyed by the salt. We could spell it out as a direct HMAC call, but using the hkdf crate's extract step makes the intent explicit and keeps the two KDF stages adjacent in the code.

    /// A reusable NIP-44 conversation key.
    ///
    /// Derived once per `(secret key, public key)` pair. Callers that send
    /// or receive many messages with the same counterparty should hold one
    /// of these and call [`encrypt`](Self::encrypt) / [`decrypt`](Self::decrypt)
    /// directly instead of going through the free [`encrypt`] / [`decrypt`]
    /// helpers, which re-run ECDH on every call.
    #[derive(Clone)]
    pub struct ConversationKey([u8; 32]);

    impl ConversationKey {
        /// Derive a conversation key from a sender's secret key and a
        /// recipient's public key.
        pub fn derive(sk: &SecretKey, pk: &PublicKey) -> Self {
            let shared = shared_secret(sk, pk);
            let (prk, _) = Hkdf::<Sha256>::extract(Some(HKDF_SALT), &shared);
            let mut out = [0u8; 32];
            out.copy_from_slice(&prk);
            ConversationKey(out)
        }

        /// Borrow the raw 32 conversation-key bytes.
        pub fn as_bytes(&self) -> &[u8; 32] {
            &self.0
        }
    }

    impl fmt::Debug for ConversationKey {
        fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
            f.write_str("ConversationKey(<redacted>)")
        }
    }

A conversation key is almost as sensitive as a secret key — anyone who holds it can decrypt every message in the conversation, in either direction. We apply the same redacted-Debug treatment we used for SecretKey in the previous chapter. We do not provide a Display impl for the same reason.

Per-message key derivation

Every call to encrypt picks a fresh 32-byte random nonce and feeds it, along with the conversation key, into HKDF-Expand. The output is 76 bytes, sliced into three parts: a 32-byte ChaCha20 key, a 12-byte ChaCha20 nonce, and a 32-byte HMAC key. The spec calls this the message keys.

    struct MessageKeys {
        chacha_key: [u8; 32],
        chacha_nonce: [u8; 12],
        hmac_key: [u8; 32],
    }

    impl MessageKeys {
        fn derive(ck: &ConversationKey, nonce: &[u8; NONCE_LEN]) -> Self {
            let hkdf = Hkdf::<Sha256>::from_prk(&ck.0)
                .expect("conversation key has correct length");
            let mut okm = [0u8; 76];
            hkdf.expand(nonce, &mut okm)
                .expect("76 bytes is within HKDF-Expand limits");

            let mut chacha_key = [0u8; 32];
            let mut chacha_nonce = [0u8; 12];
            let mut hmac_key = [0u8; 32];
            chacha_key.copy_from_slice(&okm[..32]);
            chacha_nonce.copy_from_slice(&okm[32..44]);
            hmac_key.copy_from_slice(&okm[44..76]);

            MessageKeys { chacha_key, chacha_nonce, hmac_key }
        }
    }

from_prk is the HKDF entry point that skips the extract step, treating its input as a pseudorandom key already. That's what we want here — the conversation key is the extract output, so running it through extract again would be wrong.

Padding

Length is one of the easiest things for a passive observer to see even when content is encrypted. NIP-44 addresses it by padding every plaintext to one of a small number of allowed sizes: 32 bytes for short messages, then doubling, then chunks of the next power of two divided by eight for longer messages. The full definition from the spec:

Plaintext length	Padded length
1	32
32	32
33	64
100	128
320	384
1000	1024

Before padding, we prepend a 2-byte big-endian length prefix so that decryption can recover the original length. The padded plaintext then looks like [len_hi, len_lo, plaintext..., zeros...].

    /// Compute the padded length for a plaintext of `unpadded` bytes, per the
    /// NIP-44 padding scheme.
    fn padded_len(unpadded: usize) -> usize {
        if unpadded <= 32 {
            return 32;
        }
        // Next power of two strictly greater than (unpadded - 1).
        let next_power = 1usize << (usize::BITS - (unpadded - 1).leading_zeros());
        let chunk = if next_power <= 256 { 32 } else { next_power / 8 };
        chunk * ((unpadded - 1) / chunk + 1)
    }

    fn pad(plaintext: &[u8]) -> Result<Vec<u8>, EncryptionError> {
        let len = plaintext.len();
        if len < MIN_PLAINTEXT_LEN {
            return Err(EncryptionError::MessageEmpty);
        }
        if len > MAX_PLAINTEXT_LEN {
            return Err(EncryptionError::MessageTooLong);
        }
        let padded_len = padded_len(len);
        let mut out = Vec::with_capacity(2 + padded_len);
        out.extend_from_slice(&(len as u16).to_be_bytes());
        out.extend_from_slice(plaintext);
        out.resize(2 + padded_len, 0);
        Ok(out)
    }

    fn unpad(padded: &[u8]) -> Result<Vec<u8>, EncryptionError> {
        if padded.len() < 2 {
            return Err(EncryptionError::InvalidPadding);
        }
        let len = u16::from_be_bytes([padded[0], padded[1]]) as usize;
        if len < MIN_PLAINTEXT_LEN || 2 + len > padded.len() {
            return Err(EncryptionError::InvalidPadding);
        }
        // The stated length must correspond to the padded length we would have
        // produced for it — otherwise the sender is lying about the real size.
        if padded.len() - 2 != padded_len(len) {
            return Err(EncryptionError::InvalidPadding);
        }
        Ok(padded[2..2 + len].to_vec())
    }

The unpad check that padded.len() - 2 == padded_len(len) closes a subtle attack: without it, a crafter could claim a shorter length than the plaintext actually is and use the difference to smuggle data. Insisting that the padded size be exactly what we would have produced means the length prefix is authenticated against the padding scheme itself.

Encrypt and decrypt

With the key derivation, padding, and cipher all in place, the actual encrypt and decrypt functions are mostly plumbing.

    impl ConversationKey {
        /// Encrypt `plaintext` under this conversation key with a freshly
        /// random 32-byte nonce.
        pub fn encrypt(&self, plaintext: &str) -> Result<String, EncryptionError> {
            let mut nonce = [0u8; NONCE_LEN];
            rand::thread_rng().fill_bytes(&mut nonce);
            self.encrypt_with_nonce(plaintext, &nonce)
        }

        /// Encrypt with a caller-supplied nonce.
        ///
        /// Mainly useful for verifying against the official NIP-44 test
        /// vectors, which fix the nonce to produce a deterministic output.
        /// Production code should use [`encrypt`](Self::encrypt) so that
        /// every message gets a freshly random nonce.
        pub fn encrypt_with_nonce(
            &self,
            plaintext: &str,
            nonce: &[u8; NONCE_LEN],
        ) -> Result<String, EncryptionError> {
            let padded = pad(plaintext.as_bytes())?;
            let keys = MessageKeys::derive(self, nonce);

            let mut ciphertext = padded;
            ChaCha20::new(&keys.chacha_key.into(), &keys.chacha_nonce.into())
                .apply_keystream(&mut ciphertext);

            let mut mac = <HmacSha256 as Mac>::new_from_slice(&keys.hmac_key)
                .expect("HMAC accepts any key length");
            mac.update(nonce);
            mac.update(&ciphertext);
            let tag = mac.finalize().into_bytes();

            let mut payload = Vec::with_capacity(1 + NONCE_LEN + ciphertext.len() + MAC_LEN);
            payload.push(VERSION);
            payload.extend_from_slice(nonce);
            payload.extend_from_slice(&ciphertext);
            payload.extend_from_slice(&tag);
            Ok(BASE64.encode(&payload))
        }

        /// Decrypt a NIP-44 v2 payload previously encrypted under this
        /// conversation key.
        pub fn decrypt(&self, payload: &str) -> Result<String, EncryptionError> {
            let bytes = BASE64
                .decode(payload)
                .map_err(|_| EncryptionError::InvalidPayload)?;
            // Smallest valid payload: version + nonce + min-padded + mac.
            let min_len = 1 + NONCE_LEN + 2 + 32 + MAC_LEN;
            if bytes.len() < min_len {
                return Err(EncryptionError::InvalidPayload);
            }
            if bytes[0] != VERSION {
                return Err(EncryptionError::UnsupportedVersion(bytes[0]));
            }

            let nonce: &[u8; NONCE_LEN] = bytes[1..1 + NONCE_LEN]
                .try_into()
                .expect("slice length checked above");
            let ct_end = bytes.len() - MAC_LEN;
            let ciphertext = &bytes[1 + NONCE_LEN..ct_end];
            let mac_bytes = &bytes[ct_end..];

            let keys = MessageKeys::derive(self, nonce);

            let mut mac = <HmacSha256 as Mac>::new_from_slice(&keys.hmac_key)
                .expect("HMAC accepts any key length");
            mac.update(nonce);
            mac.update(ciphertext);
            mac.verify_slice(mac_bytes)
                .map_err(|_| EncryptionError::InvalidMac)?;

            let mut plaintext = ciphertext.to_vec();
            ChaCha20::new(&keys.chacha_key.into(), &keys.chacha_nonce.into())
                .apply_keystream(&mut plaintext);

            let unpadded = unpad(&plaintext)?;
            String::from_utf8(unpadded).map_err(|_| EncryptionError::InvalidUtf8)
        }
    }

Two points about decrypt are worth lingering on. First, the HMAC is verified before the ciphertext is decrypted — a tampered message is rejected without ever being fed through ChaCha20. Second, the MAC check uses verify_slice, which the hmac crate implements in constant time; a hand-rolled byte-by-byte comparison would leak timing information about how many bytes matched.

One spec-level requirement sits above this module: when a NIP-44 payload travels inside a signed event, the spec says the outer event signature MUST be verified before the payload is decrypted. The signature authenticates the full event — author, timestamp, tags, and the encrypted content together — and skipping it means trusting a ciphertext whose provenance hasn't been checked. That check belongs to the event and signer layers, not to this module, but callers are on the hook for doing it.

One-shot helpers

Most of the time callers want to encrypt or decrypt a single message without thinking about conversation keys at all. Free functions wrap the derivation step for them.

    /// Encrypt `plaintext` to `pk` with a freshly derived conversation key.
    ///
    /// Re-runs ECDH and HKDF-Extract on every call. Callers sending many
    /// messages to the same recipient should hold a [`ConversationKey`]
    /// directly.
    pub fn encrypt(
        sk: &SecretKey,
        pk: &PublicKey,
        plaintext: &str,
    ) -> Result<String, EncryptionError> {
        ConversationKey::derive(sk, pk).encrypt(plaintext)
    }

    /// Decrypt a NIP-44 v2 payload from `pk` with a freshly derived
    /// conversation key.
    pub fn decrypt(
        sk: &SecretKey,
        pk: &PublicKey,
        payload: &str,
    ) -> Result<String, EncryptionError> {
        ConversationKey::derive(sk, pk).decrypt(payload)
    }
}

Methods on SecretKey

The free functions above are the primitives. For the common case — "encrypt this string to that pubkey" — it's nicer to write sk.nip44_encrypt(pk, msg) than to reach into a module. We hang two convenience methods off SecretKey that just forward to the free functions.

impl SecretKey {
    /// Encrypt `plaintext` to `pk` using NIP-44 v2.
    pub fn nip44_encrypt(
        &self,
        pk: &PublicKey,
        plaintext: &str,
    ) -> Result<String, EncryptionError> {
        nip44::encrypt(self, pk, plaintext)
    }

    /// Decrypt a NIP-44 v2 payload from `pk`.
    pub fn nip44_decrypt(
        &self,
        pk: &PublicKey,
        payload: &str,
    ) -> Result<String, EncryptionError> {
        nip44::decrypt(self, pk, payload)
    }
}

These live in encryption.rs rather than keys.rs because they belong to this chapter's story, not the previous one. Rust doesn't mind: impl blocks can be added to a type from any module in the crate that defines it.

What's next

The two methods above cover the common path: encrypt a string to a pubkey, decrypt one back. Hiding who is talking to whom is a separate problem that we'll tackle when we get to gift wrap (NIP-59). Before that, there are signers to build: the encryption primitives here take a &SecretKey directly, but most real applications want to delegate that to a browser extension, a remote bunker, or a hardware device without ever touching the raw bytes.