coracle-rust/book/plan/03-encryption.md

Plan: 03-encryption

Topic Summary

Covers nostr encryption primitives: ECDH shared key derivation over secp256k1, NIP-04 (legacy AES-256-CBC), and NIP-44 v2 (ChaCha20 + HKDF + HMAC-SHA256). Introduces a ConversationKey type for reusable NIP-44 shared state and hangs convenience nip04_*/nip44_* methods off SecretKey. Builds directly on the keys chapter — this is the first chapter where PublicKey and SecretKey do something interesting with each other.

Chapter Outline

1. Framing. Nostr is publicity tech first; encryption is opt-in. Two live standards: NIP-04 (deprecated, ubiquitous) and NIP-44 v2 (modern, recommended). Why both exist, what each is good for, and the metadata caveat (encrypted content hides content, not routing — gift wrap is a later chapter).

2. ECDH: the shared foundation. Both NIPs start from the same step: an ECDH shared secret between my secret key and your public key. Explain the operation, why we take the x-coordinate of the shared point, and how it depends only on the two keys (so both parties independently derive the same 32 bytes). Ship shared_secret(sk, pk) -> [u8; 32] as the public primitive.

3. NIP-04. Smaller, deprecated, but still the most common DM format on the network. AES-256-CBC with a random 16-byte IV, key = raw ECDH x-coordinate, payload = base64(ct) ?iv= base64(iv). Show encrypt and decrypt.

4. NIP-44 v2. The modern scheme.

   • Motivation: authenticated encryption, forward-compatible versioning, length-hiding padding, no reliance on CBC.
   • Message format on the wire: [version=2][nonce(32)][ciphertext][hmac(32)], base64-encoded.
   • ConversationKey: derive once per (sk, pk) pair with HKDF-Extract(salt="nip44-v2", ikm=shared_secret). Reusable across messages.
   • Per-message keys: HKDF-Expand(conversation_key, info=nonce, len=76) split into (ChaCha20 key 32, ChaCha20 nonce 12, HMAC key 32).
   • Padding: 2-byte big-endian length prefix, zero-pad to a power-of-2 chunk. Show the padding function with a small table of examples.
   • Encrypt pipeline, decrypt pipeline, MAC verification (constant-time).
   • Size and length constraints (1..=65535 plaintext bytes).

5. Wiring onto SecretKey. Convenience methods: nip04_encrypt/_decrypt, nip44_encrypt/_decrypt. These just call into the module-level functions and discard the conversation key. Note that callers doing many messages to one recipient should hold a ConversationKey directly.

6. Tests. A round-trip test for each scheme and at least one official NIP-44 test vector for confidence.

7. What's next. Gift wrap (NIP-59) and signer-side encryption will come later; the free functions here are the load-bearing primitives.

API Design

All in coracle-lib/src/encryption.rs:

pub enum EncryptionError {
    InvalidPayload,       // base64 / size / format
    UnsupportedVersion,   // NIP-44 version byte not recognized
    InvalidMac,           // NIP-44 HMAC mismatch
    InvalidPadding,       // NIP-44 length prefix / padding bytes wrong
    MessageEmpty,         // zero-length plaintext
    MessageTooLong,       // > 65535 bytes
    InvalidUtf8,          // decrypted bytes not valid UTF-8
    Crypto,               // underlying primitive failure (should be unreachable)
}
impl Display + Error for EncryptionError

pub fn shared_secret(sk: &SecretKey, pk: &PublicKey) -> [u8; 32];

pub mod nip04 {
    pub fn encrypt(sk: &SecretKey, pk: &PublicKey, plaintext: &str) -> String;
    pub fn decrypt(sk: &SecretKey, pk: &PublicKey, payload: &str)
        -> Result<String, EncryptionError>;
}

pub mod nip44 {
    pub struct ConversationKey([u8; 32]);
    impl ConversationKey {
        pub fn derive(sk: &SecretKey, pk: &PublicKey) -> Self;
        pub fn encrypt(&self, plaintext: &str) -> Result<String, EncryptionError>;
        pub fn decrypt(&self, payload: &str) -> Result<String, EncryptionError>;
    }
    pub fn encrypt(sk: &SecretKey, pk: &PublicKey, plaintext: &str)
        -> Result<String, EncryptionError>;
    pub fn decrypt(sk: &SecretKey, pk: &PublicKey, payload: &str)
        -> Result<String, EncryptionError>;
}

Added to SecretKey in keys.rs (or inside encryption.rs via impl SecretKey):

impl SecretKey {
    pub fn nip04_encrypt(&self, pk: &PublicKey, msg: &str) -> String;
    pub fn nip04_decrypt(&self, pk: &PublicKey, payload: &str)
        -> Result<String, EncryptionError>;
    pub fn nip44_encrypt(&self, pk: &PublicKey, msg: &str)
        -> Result<String, EncryptionError>;
    pub fn nip44_decrypt(&self, pk: &PublicKey, payload: &str)
        -> Result<String, EncryptionError>;
}

Place these in the encryption chapter file (cross-crate method additions via impl block in a different file), so the narrative stays together.

Code Organization

• New module coracle-lib/src/encryption.rs containing EncryptionError, shared_secret, and nip04/nip44 submodules.
• pub mod encryption; added to coracle-lib/src/lib.rs.
• The impl SecretKey { nip04_*, nip44_* } block lives in the encryption chapter as well (tangles into a new file like coracle-lib/src/keys_encryption_ext.rs, or simply back into keys.rs via a fresh {file=coracle-lib/src/keys.rs} block after the keys chapter's blocks — tangle concatenates in document order across the whole book). Pick the latter: cleaner file layout, and SUMMARY.md order guarantees 02-keys.md is tangled before 03-encryption.md.
• Tests in coracle-lib/tests/encryption.rs (round-trip + one official vector).

Dependencies

Add to coracle-lib/Cargo.toml:

• aes = "0.8" — AES-256 block cipher for NIP-04
• cbc = "0.1" — CBC mode wrapper
• chacha20 = "0.9" — raw ChaCha20 stream cipher for NIP-44 (note: chacha20poly1305 is already a dep for NIP-49, but that's the AEAD; NIP-44 uses unauthenticated stream ChaCha20 with a separate HMAC)
• hmac = "0.12" — HMAC-SHA256
• hkdf = "0.12" — HKDF extract/expand
• base64 = "0.22" — payload encoding

sha2 and secp256k1 are already present. secp256k1 0.29 exposes secp256k1::ecdh::shared_secret_point(&PublicKey, &SecretKey) -> [u8; 64] which is what we need (not SharedSecret::new, which SHA-256-hashes the result).

For the shared_secret_point call we need a full secp256k1::PublicKey (not x-only). Reconstruct one from the stored x-only key by prepending 0x02 (even parity). This matches nostr-tools and rust-nostr.

Narrative Notes

• Open the chapter with a sober framing. Don't pitch encryption as privacy; pitch it as content-hiding with known metadata leaks. Briefly mention NIP-17/gift wrap as the thing that will fix metadata, deferred to a later chapter.
• Tell ECDH as a story: "we both do the same math and end up with the same 32 bytes no one else can reproduce." Avoid jargon like "Diffie-Hellman key agreement" in the first sentence; introduce the name after the concept.
• Motivate x-coordinate extraction: a curve point is (x, y); both parties compute the same point, so taking either coordinate works, and by convention everybody takes x. This is a 30-second paragraph, not a math lecture.
• NIP-04 is short; treat it as a warm-up. The code is 20-ish lines per direction and gives the reader something concrete before NIP-44's complexity.
• For NIP-44, show the wire format first (a table or diagram), then walk through encrypt and decrypt step by step. Readers retain the pipeline better when they can see the output shape before the transformations.
• Show a padding table: 1 → 32, 32 → 32, 33 → 64, 100 → 128, 1000 → 1024 — this makes the algorithm click faster than a prose description of power-of-2 chunking.
• Emphasize ConversationKey reuse: for chat-like workloads the ECDH+HKDF-Extract step is the expensive part, and it only depends on the key pair. The one-shot nip44::encrypt helper is for convenience, not the default.
• Call out constant-time HMAC verification. Use hmac's Mac::verify_slice or the RustCrypto subtle::ConstantTimeEq — don't hand-roll a comparison.
• Do not mention gift wrap (NIP-59), NIP-49, NIP-46, or signer abstractions beyond a forward reference. Scope creep.

Design Decisions

1. Module lives in coracle-lib, next to keys. Rationale: NIP-04/44 are stateless primitives, no async, no relay coupling. Signer-based delegation (the applesauce/welshman pattern) belongs in coracle-signer — a later chapter.

2. Free functions + first-class ConversationKey type. This mirrors rust-nostr. Free functions read well in examples; the type enables efficient reuse. Welshman caches conversation keys in an LRU; we don't — applications can build caches on top of ConversationKey::derive() themselves.

3. Convenience methods on SecretKey. The user explicitly asked for this. They discard the conversation key after a single use, which is fine for one-shot calls.

4. Single EncryptionError enum, not split NIP-04/NIP-44 enums. Teaching simplicity; a handful of variants covers both.

5. Use unreachable! sparingly. For invariants the crypto libraries promise (e.g., AES key length is correct by construction), prefer .expect("…") with a clear message over a new error variant. Matches the NIP-49 code in keys.rs.

6. RustCrypto ecosystem across the board (aes, cbc, chacha20, hmac, hkdf, sha2). Consistent idioms, well-maintained, already partially used. Don't mix in bitcoin_hashes or ring.

7. secp256k1 shared_secret_point, not SharedSecret::new. The latter SHA-256-hashes the x-coordinate and is not what nostr specifies. The former returns the raw 64-byte point and we take the first 32 bytes.

8. Reconstruct full PublicKey from x-only by prepending 0x02. Matches every other reference. We're just picking the even-parity lift of the x-only point.

9. Tests live in coracle-lib/tests/encryption.rs (not inline). Consistent with CLAUDE.md guidance that tests are hand-written and not tangled. Round-trip both directions, plus one official NIP-44 v2 vector (hardcode the vector inline — no need to pull in the full nip44.vectors.json).

Open Questions

• Do we hex- or struct-verify in the NIP-44 vector test? Use at least one vector where conversation key, nonce, plaintext, and ciphertext are all known, so we can verify that our encryption produces a byte-for-byte match given a fixed nonce. Resolve during writing by pulling one encrypt_decrypt vector from the official spec.

• Should encrypt accept a caller-supplied nonce for determinism in tests? Yes, add a crate-internal encrypt_with_nonce helper used only by the vector test; public API stays nonce-free. Mirrors rust-nostr's encrypt_with_rng.

• Keep the nip04::encrypt signature as &str → String or accept bytes? Stick with &str → String for parity with nostr-tools and to match how DMs are actually used (JSON text). Document the limitation.

Plan complete. Proceed with /write-chapter 03-encryption.