coracle/coracle-rust
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
coracle-rust/book/03-encryption.md
T
Jon Staab 2f334fa3be Add encryption chapter
2026-04-13 21:21:52 -07:00

549 lines
22 KiB
Markdown
Raw Permalink Blame History

# Encryption
Nostr is publicity technology. Every event a client publishes is broadcast, in the
clear, to every relay it talks to, and from there to anyone who cares to subscribe.
That's a feature — it's what makes nostr resilient — but it means that two people
who want a private conversation have to arrange for privacy themselves, on top of a
medium that was designed to have none.
An older format,
[NIP-04](https://github.com/nostr-protocol/nips/blob/master/04.md), is still in
use in some places on the network — unauthenticated AES-256-CBC over the raw ECDH
x-coordinate, with the IV appended to a base64 blob. It has a failure mode that
NIP-44 does not (a corrupted ciphertext is indistinguishable from a wrong key),
and it leaks plaintext length. It has been deprecated, and so this library does not
implement it.
The modern standard for doing that is
[NIP-44](https://github.com/nostr-protocol/nips/blob/master/44.md) v2: it uses
authenticated encryption, pads messages to hide their length, and versions itself
so the scheme can evolve without breaking old ciphertexts. It's what this chapter
covers and what new code should reach for.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod encryption;
```
```rust {file=coracle-lib/src/encryption.rs}
//! NIP-44 v2 encryption for nostr events.
//!
//! Starts from an ECDH shared secret between a sender's
//! [`SecretKey`](crate::keys::SecretKey) and a recipient's
//! [`PublicKey`](crate::keys::PublicKey), feeds it through HKDF to derive a
//! reusable [`nip44::ConversationKey`], and then derives per-message keys from
//! there.
use std::fmt;
use base64::engine::general_purpose::STANDARD as BASE64;
use base64::Engine;
use chacha20::cipher::{KeyIvInit, StreamCipher};
use chacha20::ChaCha20;
use hkdf::Hkdf;
use hmac::{Hmac, Mac};
use rand::RngCore;
use sha2::Sha256;
use crate::keys::{PublicKey, SecretKey};
```
## Errors
Encryption has a handful of distinct failure modes that a caller might
reasonably want to distinguish: a corrupted payload, a MAC mismatch, a message
that decoded to non-UTF-8 bytes. One enum covers them all.
```rust {file=coracle-lib/src/encryption.rs}
/// Errors that can occur when encrypting or decrypting a nostr payload.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum EncryptionError {
/// The payload was not well-formed: bad base64, wrong length, missing
/// separator, or similar structural issues.
InvalidPayload,
/// The NIP-44 version byte was not one this library understands.
UnsupportedVersion(u8),
/// The NIP-44 HMAC did not match the ciphertext — either the payload was
/// tampered with or the wrong conversation key was used.
InvalidMac,
/// The NIP-44 padding was malformed: a bad length prefix or non-zero
/// padding bytes.
InvalidPadding,
/// The plaintext was empty. NIP-44 forbids zero-length messages.
MessageEmpty,
/// The plaintext exceeded NIP-44's 65535-byte limit.
MessageTooLong,
/// The decrypted bytes were not valid UTF-8.
InvalidUtf8,
}
impl fmt::Display for EncryptionError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
EncryptionError::InvalidPayload => write!(f, "invalid encrypted payload"),
EncryptionError::UnsupportedVersion(v) => {
write!(f, "unsupported NIP-44 version byte: {v:#04x}")
}
EncryptionError::InvalidMac => write!(f, "HMAC verification failed"),
EncryptionError::InvalidPadding => write!(f, "invalid NIP-44 padding"),
EncryptionError::MessageEmpty => write!(f, "message is empty"),
EncryptionError::MessageTooLong => write!(f, "message exceeds 65535 bytes"),
EncryptionError::InvalidUtf8 => write!(f, "decrypted bytes are not valid UTF-8"),
}
}
}
impl std::error::Error for EncryptionError {}
```
## ECDH
Given my secret key and your public key, I multiply them on the secp256k1 curve
to get a point that only the two of us can compute — you can reach the same
point with your secret key and my public key, because `a·G·b == b·G·a`. That
point, reduced to its 32-byte x-coordinate, is the shared secret NIP-44 builds
on.
There are two ways to spell this with the `secp256k1` crate. `SharedSecret::new`
returns a SHA-256 hash of the x-coordinate, which is a sensible default for
general-purpose ECDH — but it is *not* what NIP-44 specifies. We want the raw
x-coordinate, so we reach for `shared_secret_point`, which returns the full
64-byte point, and take the first 32 bytes.
There's one wrinkle: our [`PublicKey`](crate::keys::PublicKey) is x-only, but
`shared_secret_point` wants a full `secp256k1::PublicKey` with a parity byte. We
pick the even-parity lift — prepend `0x02` to the 32-byte x-coordinate — which
matches every other nostr implementation. ECDH only cares about the
x-coordinate of the resulting point, so the choice of input parity doesn't
affect the output.
```rust {file=coracle-lib/src/encryption.rs}
/// Compute the raw 32-byte ECDH shared secret between two nostr keys.
///
/// This is the x-coordinate of `sk · pk` on secp256k1. It is *not* hashed;
/// NIP-44 feeds it through HKDF before using it as key material.
pub fn shared_secret(sk: &SecretKey, pk: &PublicKey) -> [u8; 32] {
// Lift the x-only pubkey to a full secp256k1 point by prepending the
// even-parity byte. The ECDH output's x-coordinate is independent of the
// chosen parity, so this always matches what the counterparty computes.
let mut lifted = [0u8; 33];
lifted[0] = 0x02;
lifted[1..].copy_from_slice(&pk.as_bytes());
let full_pk = secp256k1::PublicKey::from_slice(&lifted)
.expect("even-parity lift of a valid x-only key is always a valid point");
let point = secp256k1::ecdh::shared_secret_point(&full_pk, sk.as_secp256k1());
let mut out = [0u8; 32];
out.copy_from_slice(&point[..32]);
out
}
```
The call to `sk.as_secp256k1()` is a small accessor we need to add to
`SecretKey`. It returns a reference to the wrapped `secp256k1::SecretKey` without
handing the bytes out in any form that's easier to misuse than what's already
available.
```rust {file=coracle-lib/src/keys.rs}
impl SecretKey {
/// Borrow the wrapped `secp256k1::SecretKey` for use with lower-level APIs.
///
/// Exposed so that other modules in this crate (signing, encryption) can
/// feed the key into `secp256k1` primitives without an extra copy.
pub(crate) fn as_secp256k1(&self) -> &secp256k1::SecretKey {
&self.0
}
}
```
## NIP-44 v2
NIP-44 v2 encrypts with ChaCha20, authenticates with HMAC-SHA256, and pads the
plaintext to a fixed set of sizes so length analysis can't tell a one-word
reply from a short paragraph. It layers HKDF on top of the ECDH shared secret
twice: once to derive a **conversation key** from the key pair, and once per
message to derive fresh ChaCha20 and HMAC keys from that conversation key and a
random nonce.
### The wire format
A NIP-44 v2 payload is the base64 encoding of this byte sequence:
| Offset | Length | Field |
| -----: | ------: | --------------------------------------------------- |
| 0 | 1 | version byte, always `0x02` for v2 |
| 1 | 32 | nonce (random, also feeds per-message key derivation) |
| 33 | ≥ | ciphertext (padded plaintext) |
| — | 32 | HMAC-SHA256 over nonce ‖ ciphertext |
The nonce is transmitted unencrypted and authenticated by the HMAC. The HMAC
covers both the nonce and the ciphertext, so flipping either breaks
verification.
### `ConversationKey`
The expensive part of NIP-44 is the ECDH plus the first HKDF extract. It
depends only on the key pair, so it can be computed once and reused for every
message in a conversation. We expose it as a dedicated type.
```rust {file=coracle-lib/src/encryption.rs}
pub mod nip44 {
//! NIP-44 v2 authenticated encryption.
//!
//! The hot path is:
//!
//! 1. Derive a [`ConversationKey`] from an `(sk, pk)` pair — this runs
//! ECDH plus HKDF-Extract and is the expensive step.
//! 2. For each message, call [`ConversationKey::encrypt`] /
//! [`ConversationKey::decrypt`], which runs HKDF-Expand with a fresh
//! random nonce, encrypts with ChaCha20, and signs with HMAC-SHA256.
//!
//! The free [`encrypt`] and [`decrypt`] functions are one-shot helpers
//! that derive a conversation key, use it once, and throw it away.
use super::*;
const VERSION: u8 = 0x02;
const MIN_PLAINTEXT_LEN: usize = 1;
const MAX_PLAINTEXT_LEN: usize = 65535;
const NONCE_LEN: usize = 32;
const MAC_LEN: usize = 32;
const HKDF_SALT: &[u8] = b"nip44-v2";
type HmacSha256 = Hmac<Sha256>;
```
HKDF-Extract with salt `"nip44-v2"` and the ECDH output as input keying
material is — by definition — HMAC-SHA256 over the shared secret, keyed by the
salt. We could spell it out as a direct HMAC call, but using the `hkdf` crate's
extract step makes the intent explicit and keeps the two KDF stages adjacent in
the code.
```rust {file=coracle-lib/src/encryption.rs}
/// A reusable NIP-44 conversation key.
///
/// Derived once per `(secret key, public key)` pair. Callers that send
/// or receive many messages with the same counterparty should hold one
/// of these and call [`encrypt`](Self::encrypt) / [`decrypt`](Self::decrypt)
/// directly instead of going through the free [`encrypt`] / [`decrypt`]
/// helpers, which re-run ECDH on every call.
#[derive(Clone)]
pub struct ConversationKey([u8; 32]);
impl ConversationKey {
/// Derive a conversation key from a sender's secret key and a
/// recipient's public key.
pub fn derive(sk: &SecretKey, pk: &PublicKey) -> Self {
let shared = shared_secret(sk, pk);
let (prk, _) = Hkdf::<Sha256>::extract(Some(HKDF_SALT), &shared);
let mut out = [0u8; 32];
out.copy_from_slice(&prk);
ConversationKey(out)
}
/// Borrow the raw 32 conversation-key bytes.
pub fn as_bytes(&self) -> &[u8; 32] {
&self.0
}
}
impl fmt::Debug for ConversationKey {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("ConversationKey(<redacted>)")
}
}
```
A conversation key is almost as sensitive as a secret key — anyone who holds it
can decrypt every message in the conversation, in either direction. We apply
the same redacted-`Debug` treatment we used for `SecretKey` in the previous
chapter. We do not provide a `Display` impl for the same reason.
### Per-message key derivation
Every call to `encrypt` picks a fresh 32-byte random nonce and feeds it, along
with the conversation key, into HKDF-Expand. The output is 76 bytes, sliced into
three parts: a 32-byte ChaCha20 key, a 12-byte ChaCha20 nonce, and a 32-byte
HMAC key. The spec calls this the *message keys*.
```rust {file=coracle-lib/src/encryption.rs}
struct MessageKeys {
chacha_key: [u8; 32],
chacha_nonce: [u8; 12],
hmac_key: [u8; 32],
}
impl MessageKeys {
fn derive(ck: &ConversationKey, nonce: &[u8; NONCE_LEN]) -> Self {
let hkdf = Hkdf::<Sha256>::from_prk(&ck.0)
.expect("conversation key has correct length");
let mut okm = [0u8; 76];
hkdf.expand(nonce, &mut okm)
.expect("76 bytes is within HKDF-Expand limits");
let mut chacha_key = [0u8; 32];
let mut chacha_nonce = [0u8; 12];
let mut hmac_key = [0u8; 32];
chacha_key.copy_from_slice(&okm[..32]);
chacha_nonce.copy_from_slice(&okm[32..44]);
hmac_key.copy_from_slice(&okm[44..76]);
MessageKeys { chacha_key, chacha_nonce, hmac_key }
}
}
```
`from_prk` is the HKDF entry point that skips the extract step, treating its
input as a pseudorandom key already. That's what we want here — the conversation
key is the extract output, so running it through extract again would be wrong.
### Padding
Length is one of the easiest things for a passive observer to see even when
content is encrypted. NIP-44 addresses it by padding every plaintext to one of
a small number of allowed sizes: 32 bytes for short messages, then doubling, then
chunks of the next power of two divided by eight for longer messages. The full
definition from the spec:
| Plaintext length | Padded length |
| ---------------: | ------------: |
| 1 | 32 |
| 32 | 32 |
| 33 | 64 |
| 100 | 128 |
| 320 | 384 |
| 1000 | 1024 |
Before padding, we prepend a 2-byte big-endian length prefix so that
decryption can recover the original length. The padded plaintext then looks
like `[len_hi, len_lo, plaintext..., zeros...]`.
```rust {file=coracle-lib/src/encryption.rs}
/// Compute the padded length for a plaintext of `unpadded` bytes, per the
/// NIP-44 padding scheme.
fn padded_len(unpadded: usize) -> usize {
if unpadded <= 32 {
return 32;
}
// Next power of two strictly greater than (unpadded - 1).
let next_power = 1usize << (usize::BITS - (unpadded - 1).leading_zeros());
let chunk = if next_power <= 256 { 32 } else { next_power / 8 };
chunk * ((unpadded - 1) / chunk + 1)
}
fn pad(plaintext: &[u8]) -> Result<Vec<u8>, EncryptionError> {
let len = plaintext.len();
if len < MIN_PLAINTEXT_LEN {
return Err(EncryptionError::MessageEmpty);
}
if len > MAX_PLAINTEXT_LEN {
return Err(EncryptionError::MessageTooLong);
}
let padded_len = padded_len(len);
let mut out = Vec::with_capacity(2 + padded_len);
out.extend_from_slice(&(len as u16).to_be_bytes());
out.extend_from_slice(plaintext);
out.resize(2 + padded_len, 0);
Ok(out)
}
fn unpad(padded: &[u8]) -> Result<Vec<u8>, EncryptionError> {
if padded.len() < 2 {
return Err(EncryptionError::InvalidPadding);
}
let len = u16::from_be_bytes([padded[0], padded[1]]) as usize;
if len < MIN_PLAINTEXT_LEN || 2 + len > padded.len() {
return Err(EncryptionError::InvalidPadding);
}
// The stated length must correspond to the padded length we would have
// produced for it — otherwise the sender is lying about the real size.
if padded.len() - 2 != padded_len(len) {
return Err(EncryptionError::InvalidPadding);
}
Ok(padded[2..2 + len].to_vec())
}
```
The `unpad` check that `padded.len() - 2 == padded_len(len)` closes a subtle
attack: without it, a crafter could claim a shorter length than the plaintext
actually is and use the difference to smuggle data. Insisting that the padded
size be exactly what we would have produced means the length prefix is
authenticated against the padding scheme itself.
### Encrypt and decrypt
With the key derivation, padding, and cipher all in place, the actual encrypt
and decrypt functions are mostly plumbing.
```rust {file=coracle-lib/src/encryption.rs}
impl ConversationKey {
/// Encrypt `plaintext` under this conversation key with a freshly
/// random 32-byte nonce.
pub fn encrypt(&self, plaintext: &str) -> Result<String, EncryptionError> {
let mut nonce = [0u8; NONCE_LEN];
rand::thread_rng().fill_bytes(&mut nonce);
self.encrypt_with_nonce(plaintext, &nonce)
}
/// Encrypt with a caller-supplied nonce.
///
/// Mainly useful for verifying against the official NIP-44 test
/// vectors, which fix the nonce to produce a deterministic output.
/// Production code should use [`encrypt`](Self::encrypt) so that
/// every message gets a freshly random nonce.
pub fn encrypt_with_nonce(
&self,
plaintext: &str,
nonce: &[u8; NONCE_LEN],
) -> Result<String, EncryptionError> {
let padded = pad(plaintext.as_bytes())?;
let keys = MessageKeys::derive(self, nonce);
let mut ciphertext = padded;
ChaCha20::new(&keys.chacha_key.into(), &keys.chacha_nonce.into())
.apply_keystream(&mut ciphertext);
let mut mac = <HmacSha256 as Mac>::new_from_slice(&keys.hmac_key)
.expect("HMAC accepts any key length");
mac.update(nonce);
mac.update(&ciphertext);
let tag = mac.finalize().into_bytes();
let mut payload = Vec::with_capacity(1 + NONCE_LEN + ciphertext.len() + MAC_LEN);
payload.push(VERSION);
payload.extend_from_slice(nonce);
payload.extend_from_slice(&ciphertext);
payload.extend_from_slice(&tag);
Ok(BASE64.encode(&payload))
}
/// Decrypt a NIP-44 v2 payload previously encrypted under this
/// conversation key.
pub fn decrypt(&self, payload: &str) -> Result<String, EncryptionError> {
let bytes = BASE64
.decode(payload)
.map_err(|_| EncryptionError::InvalidPayload)?;
// Smallest valid payload: version + nonce + min-padded + mac.
let min_len = 1 + NONCE_LEN + 2 + 32 + MAC_LEN;
if bytes.len() < min_len {
return Err(EncryptionError::InvalidPayload);
}
if bytes[0] != VERSION {
return Err(EncryptionError::UnsupportedVersion(bytes[0]));
}
let nonce: &[u8; NONCE_LEN] = bytes[1..1 + NONCE_LEN]
.try_into()
.expect("slice length checked above");
let ct_end = bytes.len() - MAC_LEN;
let ciphertext = &bytes[1 + NONCE_LEN..ct_end];
let mac_bytes = &bytes[ct_end..];
let keys = MessageKeys::derive(self, nonce);
let mut mac = <HmacSha256 as Mac>::new_from_slice(&keys.hmac_key)
.expect("HMAC accepts any key length");
mac.update(nonce);
mac.update(ciphertext);
mac.verify_slice(mac_bytes)
.map_err(|_| EncryptionError::InvalidMac)?;
let mut plaintext = ciphertext.to_vec();
ChaCha20::new(&keys.chacha_key.into(), &keys.chacha_nonce.into())
.apply_keystream(&mut plaintext);
let unpadded = unpad(&plaintext)?;
String::from_utf8(unpadded).map_err(|_| EncryptionError::InvalidUtf8)
}
}
```
Two points about `decrypt` are worth lingering on. First, the HMAC is verified
*before* the ciphertext is decrypted — a tampered message is rejected without
ever being fed through ChaCha20. Second, the MAC check uses `verify_slice`,
which the `hmac` crate implements in constant time; a hand-rolled byte-by-byte
comparison would leak timing information about how many bytes matched.
One spec-level requirement sits above this module: when a NIP-44 payload
travels inside a signed event, the spec says the outer event signature MUST be
verified before the payload is decrypted. The signature authenticates the full
event — author, timestamp, tags, and the encrypted content together — and
skipping it means trusting a ciphertext whose provenance hasn't been checked.
That check belongs to the event and signer layers, not to this module, but
callers are on the hook for doing it.
### One-shot helpers
Most of the time callers want to encrypt or decrypt a single message without
thinking about conversation keys at all. Free functions wrap the derivation
step for them.
```rust {file=coracle-lib/src/encryption.rs}
/// Encrypt `plaintext` to `pk` with a freshly derived conversation key.
///
/// Re-runs ECDH and HKDF-Extract on every call. Callers sending many
/// messages to the same recipient should hold a [`ConversationKey`]
/// directly.
pub fn encrypt(
sk: &SecretKey,
pk: &PublicKey,
plaintext: &str,
) -> Result<String, EncryptionError> {
ConversationKey::derive(sk, pk).encrypt(plaintext)
}
/// Decrypt a NIP-44 v2 payload from `pk` with a freshly derived
/// conversation key.
pub fn decrypt(
sk: &SecretKey,
pk: &PublicKey,
payload: &str,
) -> Result<String, EncryptionError> {
ConversationKey::derive(sk, pk).decrypt(payload)
}
}
```
## Methods on `SecretKey`
The free functions above are the primitives. For the common case — "encrypt
this string to that pubkey" — it's nicer to write `sk.nip44_encrypt(pk, msg)`
than to reach into a module. We hang two convenience methods off `SecretKey`
that just forward to the free functions.
```rust {file=coracle-lib/src/encryption.rs}
impl SecretKey {
/// Encrypt `plaintext` to `pk` using NIP-44 v2.
pub fn nip44_encrypt(
&self,
pk: &PublicKey,
plaintext: &str,
) -> Result<String, EncryptionError> {
nip44::encrypt(self, pk, plaintext)
}
/// Decrypt a NIP-44 v2 payload from `pk`.
pub fn nip44_decrypt(
&self,
pk: &PublicKey,
payload: &str,
) -> Result<String, EncryptionError> {
nip44::decrypt(self, pk, payload)
}
}
```
These live in `encryption.rs` rather than `keys.rs` because they belong to this
chapter's story, not the previous one. Rust doesn't mind: `impl` blocks can be
added to a type from any module in the crate that defines it.
## What's next
The two methods above cover the common path: encrypt a string to a pubkey,
decrypt one back. Hiding *who* is talking to whom is a separate problem that
we'll tackle when we get to gift wrap (NIP-59). Before that, there are signers
to build: the encryption primitives here take a `&SecretKey` directly, but most
real applications want to delegate that to a browser extension, a remote
bunker, or a hardware device without ever touching the raw bytes.
Reference in New Issue View Git Blame Copy Permalink