Compare commits

..

16 Commits

Author SHA1 Message Date
Jon Staab 2d50fa13b4 Add nip 91 tags 2026-05-22 15:22:02 -07:00
Jon Staab 75381b653e Add search chapter 2026-05-20 16:07:58 -07:00
Jon Staab d0709e1811 Use traits to keep event methods dry 2026-05-20 14:27:26 -07:00
Jon Staab 8ad0bc393b Add protected chapter 2026-05-20 12:27:22 -07:00
Jon Staab f0d49c6fb8 Add expiration chapter 2026-05-20 11:07:12 -07:00
Jon Staab 8773017198 Add some utility methods to tags 2026-04-22 10:39:03 -07:00
Jon Staab a8a57a3d77 Add filters chapter 2026-04-22 09:33:33 -07:00
Jon Staab c8f6bc1652 Add pow 2026-04-21 06:18:49 -07:00
Jon Staab b96ad44d3a Add addresses chapter 2026-04-17 16:43:56 -07:00
Jon Staab c7ec9f18fa Add kinds chapter 2026-04-16 17:37:21 -07:00
Jon Staab e42d7efd6e Add coracle-wasm bindings 2026-04-16 16:55:15 -07:00
Jon Staab fe05d540d4 Update readme 2026-04-16 16:55:03 -07:00
Jon Staab 1267477081 Swap tags and events chapters 2026-04-16 16:54:28 -07:00
Jon Staab 8a29ff39d6 Add tags chapter 2026-04-16 16:49:18 -07:00
Jon Staab 2553cff300 Add events chapter 2026-04-14 14:25:54 -07:00
Jon Staab 7c6909a791 Add nostrlib to agents stuff 2026-04-14 14:06:59 -07:00
50 changed files with 9402 additions and 248 deletions
+2
View File
@@ -0,0 +1,2 @@
--ignore-dir=target
--ignore-dir=.claude
+7
View File
@@ -0,0 +1,7 @@
Refer to @book/SUMMARY.md for a table of contents. Find a chapter that has not yet been written, then execute your /research-chapter, /plan-chapter, and /write-chapter skills for the chapter.
Validation after this process is complete:
- Make sure the chapter is concise, correct, and makes sense in context of the book
- Add tests to the tests directory
- Make sure that the project compiles and the tests complete
-2
View File
@@ -33,8 +33,6 @@ Draw on the research to make your questions specific and informed. Areas to expl
introduced? What prior knowledge can we assume from earlier chapters? introduced? What prior knowledge can we assume from earlier chapters?
- **Code Organization**: Which crate(s) should this code live in? How should modules be - **Code Organization**: Which crate(s) should this code live in? How should modules be
structured? structured?
- **Teaching Approach**: Are there concepts that need careful explanation? Diagrams?
Worked examples?
Ask questions in batches of 2-3 at a time. After each round of answers, follow up on Ask questions in batches of 2-3 at a time. After each round of answers, follow up on
anything that needs clarification before moving on. Continue until you have enough detail anything that needs clarification before moving on. Continue until you have enough detail
+5 -1
View File
@@ -40,10 +40,11 @@ Each sub-agent should:
3. Return a detailed summary focusing **only on functionality relevant to the chapter topic** 3. Return a detailed summary focusing **only on functionality relevant to the chapter topic**
— ignore unrelated parts of the library — ignore unrelated parts of the library
The six code reference implementations to analyze: The seven code reference implementations to analyze:
- `ref/applesauce` (TypeScript — noStrudel ecosystem) - `ref/applesauce` (TypeScript — noStrudel ecosystem)
- `ref/ndk` (TypeScript — Nostr Development Kit) - `ref/ndk` (TypeScript — Nostr Development Kit)
- `ref/nostr-gadgets` (TypeScript — high-level utilities, JSR) - `ref/nostr-gadgets` (TypeScript — high-level utilities, JSR)
- `ref/nostrlib` (Go — fiatjaf's comprehensive Go library)
- `ref/nostr-tools` (TypeScript — low-level tools, minimal deps) - `ref/nostr-tools` (TypeScript — low-level tools, minimal deps)
- `ref/rust-nostr` (Rust — full implementation, multiple crates) - `ref/rust-nostr` (Rust — full implementation, multiple crates)
- `ref/welshman` (TypeScript — extracted from Coracle client) - `ref/welshman` (TypeScript — extracted from Coracle client)
@@ -77,6 +78,9 @@ The research file should have this structure:
### nostr-gadgets ### nostr-gadgets
<detailed findings> <detailed findings>
### nostrlib
<detailed findings>
### nostr-tools ### nostr-tools
<detailed findings> <detailed findings>
+3
View File
@@ -0,0 +1,3 @@
.claude
CLAUDE.md
target
+2 -1
View File
@@ -75,6 +75,7 @@ of chapters, but is not a source of implementation patterns.
| `ref/applesauce` | TypeScript | Libraries for building nostr web clients (noStrudel) | | `ref/applesauce` | TypeScript | Libraries for building nostr web clients (noStrudel) |
| `ref/ndk` | TypeScript | Nostr Development Kit with framework integrations | | `ref/ndk` | TypeScript | Nostr Development Kit with framework integrations |
| `ref/nostr-gadgets` | TypeScript | High-level nostr client utilities (JSR) | | `ref/nostr-gadgets` | TypeScript | High-level nostr client utilities (JSR) |
| `ref/nostrlib` | Go | Comprehensive Go library for relays, clients, and hybrid apps (fiatjaf) |
| `ref/nostr-tools` | TypeScript | Low-level nostr tools, minimal dependencies | | `ref/nostr-tools` | TypeScript | Low-level nostr tools, minimal dependencies |
| `ref/rust-nostr` | Rust | Full Rust implementation with multiple crates | | `ref/rust-nostr` | Rust | Full Rust implementation with multiple crates |
| `ref/welshman` | TypeScript | Nostr toolkit extracted from the Coracle client | | `ref/welshman` | TypeScript | Nostr toolkit extracted from the Coracle client |
@@ -96,7 +97,7 @@ Chapters are developed in three phases using slash commands:
## Code Style ## Code Style
- Rust code should be idiomatic and well-documented - Rust code should be idiomatic and well-documented
- Prefer clarity over cleverness — this is a teaching resource - Prefer clarity over cleverness
- Keep dependencies minimal - Keep dependencies minimal
- All public items need doc comments that explain the *why* - All public items need doc comments that explain the *why*
- Code must compile and pass `just all` before a chapter is considered complete - Code must compile and pass `just all` before a chapter is considered complete
Generated
+96
View File
@@ -45,6 +45,12 @@ dependencies = [
"generic-array", "generic-array",
] ]
[[package]]
name = "bumpalo"
version = "3.20.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d20789868f4b01b2f2caec9f5c4e0213b41e3e5702a50157d699ae31ced2fcb"
[[package]] [[package]]
name = "cc" name = "cc"
version = "1.2.59" version = "1.2.59"
@@ -158,6 +164,18 @@ dependencies = [
"pulldown-cmark", "pulldown-cmark",
] ]
[[package]]
name = "coracle-wasm"
version = "0.1.0"
dependencies = [
"coracle-lib",
"hex",
"secp256k1",
"serde-wasm-bindgen",
"serde_json",
"wasm-bindgen",
]
[[package]] [[package]]
name = "cpufeatures" name = "cpufeatures"
version = "0.2.17" version = "0.2.17"
@@ -264,6 +282,16 @@ version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "js-sys"
version = "0.3.95"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2964e92d1d9dc3364cae4d718d93f227e3abb088e747d92e0395bfdedf1c12ca"
dependencies = [
"once_cell",
"wasm-bindgen",
]
[[package]] [[package]]
name = "libc" name = "libc"
version = "0.2.184" version = "0.2.184"
@@ -276,6 +304,12 @@ version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79" checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]] [[package]]
name = "opaque-debug" name = "opaque-debug"
version = "0.3.1" version = "0.3.1"
@@ -390,6 +424,12 @@ dependencies = [
"getrandom", "getrandom",
] ]
[[package]]
name = "rustversion"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]] [[package]]
name = "salsa20" name = "salsa20"
version = "0.10.2" version = "0.10.2"
@@ -441,6 +481,17 @@ dependencies = [
"serde_derive", "serde_derive",
] ]
[[package]]
name = "serde-wasm-bindgen"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8302e169f0eddcc139c70f139d19d6467353af16f9fce27e8c30158036a1e16b"
dependencies = [
"js-sys",
"serde",
"wasm-bindgen",
]
[[package]] [[package]]
name = "serde_core" name = "serde_core"
version = "1.0.228" version = "1.0.228"
@@ -578,6 +629,51 @@ version = "0.11.1+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
[[package]]
name = "wasm-bindgen"
version = "0.2.118"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0bf938a0bacb0469e83c1e148908bd7d5a6010354cf4fb73279b7447422e3a89"
dependencies = [
"cfg-if",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.118"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eeff24f84126c0ec2db7a449f0c2ec963c6a49efe0698c4242929da037ca28ed"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.118"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d08065faf983b2b80a79fd87d8254c409281cf7de75fc4b773019824196c904"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.118"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5fd04d9e306f1907bd13c6361b5c6bfc7b3b3c095ed3f8a9246390f8dbdee129"
dependencies = [
"unicode-ident",
]
[[package]] [[package]]
name = "zerocopy" name = "zerocopy"
version = "0.8.48" version = "0.8.48"
+1
View File
@@ -8,4 +8,5 @@ members = [
"coracle-domain", "coracle-domain",
"coracle-content", "coracle-content",
"coracle-storage", "coracle-storage",
"coracle-wasm",
] ]
+8
View File
@@ -9,6 +9,11 @@ It has the following crates:
- `coracle-content` - Text parsing and rendering utilities - `coracle-content` - Text parsing and rendering utilities
- `coracle-storage` - Storage adapters for different platforms - `coracle-storage` - Storage adapters for different platforms
Platform bindings live in separate crates to keep the core library free of platform-specific annotations:
- `coracle-wasm` - Web bindings via `wasm-bindgen`, producing JS-friendly types
- `coracle-ffi` - C-compatible FFI via UniFFI, consumed by KMP (Kotlin Multiplatform)
All code is written in a [literate programming](https://en.wikipedia.org/wiki/Literate_programming) style and compiled to both html documentation and rust source code. The goal of this repository is threefold: All code is written in a [literate programming](https://en.wikipedia.org/wiki/Literate_programming) style and compiled to both html documentation and rust source code. The goal of this repository is threefold:
- To create a complete resource for learning how to work with the nostr protocol for humans - To create a complete resource for learning how to work with the nostr protocol for humans
@@ -73,6 +78,9 @@ coracle-tangle/ # The tangle/weave tool (Rust binary)
coracle-lib/ # Core nostr types and utilities (generated src/) coracle-lib/ # Core nostr types and utilities (generated src/)
coracle-net/ # Relay networking (generated src/) coracle-net/ # Relay networking (generated src/)
coracle-signer/ # Signing abstractions (generated src/) coracle-signer/ # Signing abstractions (generated src/)
coracle-domain/ # Domain-specific nostr types (generated src/)
coracle-content/ # Content parsing (generated src/) coracle-content/ # Content parsing (generated src/)
coracle-storage/ # Storage adapters (generated src/) coracle-storage/ # Storage adapters (generated src/)
coracle-wasm/ # Web bindings (wasm-bindgen)
coracle-ffi/ # C/KMP bindings (UniFFI)
``` ```
+8 -1
View File
@@ -108,7 +108,7 @@ belongs to us rather than a foreign crate.
/// ///
/// This is the "name" half of a nostr identity. It's safe to log, share, and /// This is the "name" half of a nostr identity. It's safe to log, share, and
/// store — it identifies an author but grants no ability to speak as them. /// store — it identifies an author but grants no ability to speak as them.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct PublicKey(secp256k1::XOnlyPublicKey); pub struct PublicKey(secp256k1::XOnlyPublicKey);
impl PublicKey { impl PublicKey {
@@ -133,6 +133,13 @@ impl PublicKey {
.map_err(|_| KeyError::InvalidKey)?; .map_err(|_| KeyError::InvalidKey)?;
Ok(PublicKey(inner)) Ok(PublicKey(inner))
} }
/// Parse from raw bytes (exactly 32).
pub fn from_bytes(bytes: &[u8]) -> Result<Self, KeyError> {
let inner = secp256k1::XOnlyPublicKey::from_slice(bytes)
.map_err(|_| KeyError::InvalidKey)?;
Ok(PublicKey(inner))
}
} }
``` ```
-202
View File
@@ -1,202 +0,0 @@
# Events
Everything in nostr is an **event**. A short note, a profile update, a reaction, a relay list —
they're all events. An event is a JSON object signed by its author's cryptographic key. Once
signed, it cannot be altered without invalidating the signature, which means any relay or client
can verify that an event is authentic without trusting anyone.
## The event structure
Here's what a nostr event looks like as JSON:
```json
{
"id": "4376c65d2f232afbe9b882a35baa4f6fe8bce184396e157b1a35b3f01b3e285c",
"pubkey": "6e468422dfb74a5738702a8823b9b28168abab8655faacb6853cd0ee15deee93",
"created_at": 1673347337,
"kind": 1,
"tags": [
["e", "3da979448d9ba263864c4d6f14984c423a3838364ec255f03c7904b1ae77f206"],
["p", "bf2376e17ba4ec269d10fcc996a4746b451152be9031fa48e74553dde5526bce"]
],
"content": "Hello, nostr!",
"sig": "908a15e46fb4d8675bab026fc230a0e3542bfade63da02d542fb78b2a8513fcd..."
}
```
Let's define this in Rust. We'll start with the module declaration:
```rust {file=coracle-lib/src/lib.rs}
pub mod event;
```
And now the `Event` struct itself. Each field maps directly to the JSON above:
```rust {file=coracle-lib/src/event.rs}
use serde::{Deserialize, Serialize};
use sha2::{Sha256, Digest};
/// A nostr event — the fundamental data type of the protocol.
///
/// Every interaction on nostr is represented as a signed event. The `id` is a
/// SHA-256 hash of the canonical serialization, and the `sig` is a Schnorr
/// signature over that hash using the author's private key.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Event {
/// The event ID: a 32-byte hex-encoded SHA-256 hash of the canonical serialization.
pub id: String,
/// The author's public key: a 32-byte hex-encoded secp256k1 public key (x-only).
pub pubkey: String,
/// Unix timestamp in seconds when the event was created.
pub created_at: u64,
/// The event kind determines how the event should be interpreted.
/// Kind 0 = metadata, kind 1 = short text note, kind 3 = contacts, etc.
pub kind: u16,
/// A list of tags. Each tag is an array of strings where the first element
/// identifies the tag type. Common tags: ["e", <event-id>], ["p", <pubkey>].
pub tags: Vec<Vec<String>>,
/// The event content. Interpretation depends on the kind.
pub content: String,
/// A 64-byte hex-encoded Schnorr signature of the event ID.
pub sig: String,
}
```
## Computing the event ID
The event ID is not chosen by the author — it's *derived* from the event's content via a
deterministic process. This is what makes events tamper-proof: change any field and the ID
changes, which invalidates the signature.
The ID is the SHA-256 hash of a specific JSON serialization called the **canonical form**:
```json
[0, <pubkey>, <created_at>, <kind>, <tags>, <content>]
```
That leading `0` is a version number reserved for future use. The array is serialized as
compact JSON (no whitespace) and UTF-8 encoded before hashing.
```rust {file=coracle-lib/src/event.rs}
impl Event {
/// Serialize this event into the canonical form used for ID computation.
/// The result is: `[0, pubkey, created_at, kind, tags, content]`
pub fn serialize(&self) -> String {
serde_json::to_string(
&serde_json::json!([0, self.pubkey, self.created_at, self.kind, self.tags, self.content])
).expect("event serialization should never fail")
}
/// Compute the event ID by SHA-256 hashing the canonical serialization.
pub fn compute_id(&self) -> String {
let serialized = self.serialize();
let hash = Sha256::digest(serialized.as_bytes());
hex::encode(hash)
}
/// Check whether the event's `id` field matches the computed hash.
pub fn id_is_valid(&self) -> bool {
self.id == self.compute_id()
}
}
```
## Verifying signatures
Nostr uses **Schnorr signatures** over the secp256k1 curve — the same curve used by Bitcoin.
The signature is computed over the event ID (which is already a hash), and can be verified
using the author's public key.
This means anyone can verify an event's authenticity without contacting the author or any
trusted authority. The math guarantees it.
```rust {file=coracle-lib/src/event.rs}
impl Event {
/// Verify the event's Schnorr signature against its public key and ID.
/// Returns `true` if the signature is valid and the ID matches the content.
pub fn verify(&self) -> bool {
if !self.id_is_valid() {
return false;
}
let id_bytes = match hex::decode(&self.id) {
Ok(b) => b,
Err(_) => return false,
};
let sig_bytes = match hex::decode(&self.sig) {
Ok(b) => b,
Err(_) => return false,
};
let pubkey_bytes = match hex::decode(&self.pubkey) {
Ok(b) => b,
Err(_) => return false,
};
let msg = match secp256k1::Message::from_digest_slice(&id_bytes) {
Ok(m) => m,
Err(_) => return false,
};
let sig = match secp256k1::schnorr::Signature::from_slice(&sig_bytes) {
Ok(s) => s,
Err(_) => return false,
};
let pubkey = match secp256k1::XOnlyPublicKey::from_slice(&pubkey_bytes) {
Ok(k) => k,
Err(_) => return false,
};
sig.verify(&msg, &pubkey).is_ok()
}
}
```
## Creating events
To create an event, you provide the content fields and sign with a secret key. The process is:
1. Fill in `pubkey`, `created_at`, `kind`, `tags`, and `content`
2. Compute the `id` from the canonical serialization
3. Sign the `id` with the secret key to produce `sig`
```rust {file=coracle-lib/src/event.rs}
impl Event {
/// Create a new signed event from the given fields and secret key.
///
/// The `pubkey`, `id`, and `sig` fields are computed automatically.
pub fn new(
kind: u16,
content: &str,
tags: Vec<Vec<String>>,
created_at: u64,
secret_key: &secp256k1::SecretKey,
) -> Self {
let secp = secp256k1::Secp256k1::new();
let keypair = secp256k1::Keypair::from_secret_key(&secp, secret_key);
let (pubkey, _parity) = keypair.x_only_public_key();
let mut event = Event {
id: String::new(),
pubkey: hex::encode(pubkey.serialize()),
created_at,
kind,
tags,
content: content.to_string(),
sig: String::new(),
};
event.id = event.compute_id();
let id_bytes = hex::decode(&event.id).expect("just computed, must be valid hex");
let msg = secp256k1::Message::from_digest_slice(&id_bytes)
.expect("SHA-256 output is always 32 bytes");
let sig = secp.sign_schnorr_no_aux_rand(&msg, &keypair);
event.sig = hex::encode(sig.serialize());
event
}
}
```
With this, you can create and verify events — the atomic unit of all nostr communication. In
the next chapter, we'll look at how keys work in more detail and introduce signing abstractions
that go beyond a raw secret key.
+306
View File
@@ -0,0 +1,306 @@
# Tags
Nostr needs a way to attach structured data to things: who wrote
this, what it replies to, which topics it belongs under, which relay
to find something on, which article it updates. That's what tags are.
A tag is a list of strings. The first string names the tag; the rest
are its values. A collection of tags is a list of these. That's the
whole definition:
```text
["e", "4376c65d...", "wss://relay.example", "reply"]
["p", "6e468422..."]
["a", "30023:6e468422...:my-article-slug"]
["t", "nostr"]
```
It looks plainer than it is. Three choices in that shape matter.
**Lists of lists, not maps.** If tags were a dictionary, a key could
only appear once and the order would be lost. Neither property is one
nostr can give up. A piece of data commonly references several pubkeys
(`["p", ...]` repeated), several other pieces of data (`["e", ...]`
repeated), and several topics (`["t", ...]` repeated), and the order
these appear in sometimes carries meaning — NIP-10 reply threads, for
instance, use position as a fallback when explicit markers are missing.
Lists of lists preserve both.
**Single-letter names are indexed.** Relays index tags whose name is a
single letter (`a` through `z`, `A` through `Z`) and let clients query
them with `#e`, `#p`, `#t` filters. Multi-character names — `alt`,
`imeta`, `expiration` — are carried on the wire but not indexed. The
distinction matters at design time: if you want something to be
queryable, give it a single-letter name.
**Meaning is context-dependent.** The `e` tag appears in eight
different NIPs and means eight different things: a reply, a fork, a
merge, a transaction reference, a report target, a list member, an
approval, a mention. There is no way to interpret `["e", ...]`
without knowing the context it appears in. The [building-nostr]
philosophy puts this bluntly: "when resolving the meaning of a tag,
always first look at the specifications for the event's kind." A
library type that tries to parse tags into a taxonomy is betting on
the wrong end of that rule.
This chapter therefore introduces a type that is almost nothing. `Tag`
wraps `Vec<String>` and adds five or six convenience methods. `Tags`
wraps `Vec<Tag>` and adds query methods. That is enough for every
caller we'll meet in the rest of the book.
[building-nostr]: https://building-nostr.coracle.social
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod tags;
```
```rust {file=coracle-lib/src/tags.rs}
//! The nostr `Tag` type: a thin wrapper around `Vec<String>` with
//! accessors and a set of free functions that query slices of tags
//! by name.
use serde::{Deserialize, Serialize};
```
## The `Tag` type
A `Tag` is a `Vec<String>`. We wrap it as a tuple struct so that it
has its own type name and its own set of methods, and we mark the
serde impl `transparent` so that on the wire — and in the canonical
hash bytes — it is indistinguishable from the raw array.
```rust {file=coracle-lib/src/tags.rs}
/// A single nostr tag: the first string is the tag's name, the rest
/// are its values.
///
/// `Tag` serializes transparently as its inner `Vec<String>`, so the
/// wire format and the canonical hash bytes are unchanged from a bare
/// `Vec<Vec<String>>`. Wrapping it in a type only exists to hang
/// accessors and helpers off of, and to give call sites something to
/// read.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(transparent)]
pub struct Tag(pub Vec<String>);
```
Most of the methods on `Tag` just peek at positions in the vector. An
empty tag is technically well-formed but meaningless, so `name` and
`value` return `""` rather than panicking or returning an `Option`;
readers almost always want to compare against a known string and the
empty-string fallback makes those comparisons safe.
```rust {file=coracle-lib/src/tags.rs}
impl Tag {
/// Build a tag from a name and a list of values.
///
/// ```
/// # use coracle_lib::tags::Tag;
/// let t = Tag::new("t", ["nostr"]);
/// assert_eq!(t.name(), "t");
/// assert_eq!(t.value(), "nostr");
/// ```
pub fn new<N, V, S>(name: N, values: V) -> Self
where
N: Into<String>,
V: IntoIterator<Item = S>,
S: Into<String>,
{
let mut v = Vec::new();
v.push(name.into());
v.extend(values.into_iter().map(Into::into));
Tag(v)
}
/// The tag's name — its first entry, or `""` if the tag is empty.
pub fn name(&self) -> &str {
self.0.first().map(String::as_str).unwrap_or("")
}
/// The tag's primary value — its second entry, or `""` if absent.
pub fn value(&self) -> &str {
self.0.get(1).map(String::as_str).unwrap_or("")
}
/// Every entry after the name.
pub fn values(&self) -> &[String] {
if self.0.is_empty() { &[] } else { &self.0[1..] }
}
/// The entry at `i`, or `None` if out of bounds.
pub fn get(&self, i: usize) -> Option<&str> {
self.0.get(i).map(String::as_str)
}
/// The number of entries in the tag (name plus values).
pub fn len(&self) -> usize {
self.0.len()
}
/// Whether the tag is empty.
pub fn is_empty(&self) -> bool {
self.0.is_empty()
}
/// Borrow the underlying slice.
pub fn as_slice(&self) -> &[String] {
&self.0
}
}
```
Conversions to and from `Vec<String>` make it painless to drop into
and out of the wrapper:
```rust {file=coracle-lib/src/tags.rs}
impl From<Vec<String>> for Tag {
fn from(v: Vec<String>) -> Self {
Tag(v)
}
}
impl From<Tag> for Vec<String> {
fn from(t: Tag) -> Self {
t.0
}
}
```
## A collection of tags
Wherever tags appear, you want to query them by name: does the
collection have a `p` tag? What's its `d` value? Which topics are
present? These come up often enough that writing the filter by hand
every time is noise. They belong on a type.
`Tags` is a newtype around `Vec<Tag>`. Like `Tag` itself, the serde
impl is transparent, so the wire format is unchanged. A
`Deref<Target = [Tag]>` impl gives iteration, indexing, and `len` for
free, so the wrapper costs almost nothing at call sites.
```rust {file=coracle-lib/src/tags.rs}
/// A collection of tags. Serializes transparently as its inner
/// `Vec<Tag>`.
#[derive(Debug, Clone, PartialEq, Eq, Default, Serialize, Deserialize)]
#[serde(transparent)]
pub struct Tags(pub Vec<Tag>);
impl Tags {
/// An empty collection.
pub fn new() -> Self {
Tags(Vec::new())
}
/// Return the first tag with the given name.
pub fn find(&self, name: &str) -> Option<&Tag> {
self.0.iter().find(|t| t.name() == name)
}
/// Iterate over every tag with the given name.
pub fn find_all<'a>(&'a self, name: &'a str) -> impl Iterator<Item = &'a Tag> + 'a {
self.0.iter().filter(move |t| t.name() == name)
}
/// Return the value (second entry) of the first tag with the
/// given name, or `None` if no such tag exists.
pub fn value(&self, name: &str) -> Option<&str> {
self.find(name).map(Tag::value)
}
/// Iterate over the values of every tag with the given name.
pub fn values<'a>(&'a self, name: &'a str) -> impl Iterator<Item = &'a str> + 'a {
self.find_all(name).map(Tag::value)
}
/// Whether the collection contains at least one tag with the
/// given name.
pub fn has(&self, name: &str) -> bool {
self.find(name).is_some()
}
/// Append a tag built from a name and a list of values. Returns
/// `self` so the call can chain.
pub fn add<N, V, S>(mut self, name: N, values: V) -> Self
where
N: Into<String>,
V: IntoIterator<Item = S>,
S: Into<String>,
{
self.0.push(Tag::new(name, values));
self
}
/// Remove every tag with the given name. Returns `self` so the
/// call can chain.
pub fn remove(mut self, name: &str) -> Self {
self.0.retain(|t| t.name() != name);
self
}
/// Replace every tag with the given name by a single new tag
/// built from `name` and `values`. Equivalent to `remove(name)`
/// followed by `add(Tag::new(name, values))`.
pub fn set<N, V, S>(self, name: N, values: V) -> Self
where
N: Into<String>,
V: IntoIterator<Item = S>,
S: Into<String>,
{
let name = name.into();
self.remove(&name).add(name, values)
}
}
impl std::ops::Deref for Tags {
type Target = [Tag];
fn deref(&self) -> &[Tag] {
&self.0
}
}
impl From<Vec<Tag>> for Tags {
fn from(v: Vec<Tag>) -> Self {
Tags(v)
}
}
impl From<Tags> for Vec<Tag> {
fn from(t: Tags) -> Self {
t.0
}
}
impl FromIterator<Tag> for Tags {
fn from_iter<I: IntoIterator<Item = Tag>>(iter: I) -> Self {
Tags(iter.into_iter().collect())
}
}
```
That's the entire query API. No `TagKind`, no marker enum, no parsed
variants. Checking whether a collection mentions a particular pubkey
is `tags.values("p").any(|p| p == hex)` — the code reads the way the
question sounds.
Mutation is just as flat. `add` appends a tag; `remove` drops every
tag with a given name; `set` replaces every tag of a given name with
a single fresh one, which is the common case when a replaceable kind
is rewriting one of its own singleton fields. All three consume and
return `self`, so assembling or rewriting a collection reads as one
expression:
```rust
let tags = Tags::new()
.add("t", ["nostr"]) // [["t", "nostr"]]
.add("t", ["rust"]) // [["t", "nostr"], ["t", "rust"]]
.set("t", ["nothing"]) // [["t", "nothing"]]
.remove("t"); // []
```
## What's next
Tags are the structured-data building block that every other type in
the library will carry. The next chapter introduces the `Event` type —
the signed, content-addressable JSON object at the heart of the nostr
protocol — which uses `Tags` as one of its seven fields.
+612
View File
@@ -0,0 +1,612 @@
# Events
Almost everything else in this book is built out of events. Profiles, follow
lists, direct messages, zap receipts, emoji reactions, long-form articles —
they are all the same data type with different values in a handful of fields.
If you understand events, you understand most of what a nostr library does.
An event is a JSON object with seven fields. What makes it interesting is
that the `id` field is a hash of the other fields and the `sig` field is a
Schnorr signature over that id. Anyone can verify, offline, that a given
event was produced by the holder of a particular secret key and has not been
altered since. There is no database row to update, no server to consult. The
event either hashes to its claimed id and verifies under its claimed pubkey,
or it doesn't — and that property, referential transparency in the functional
sense, is why nostr can work at all.
This chapter builds the Rust types that represent events and the two
primitives that make them go: computing the id and verifying the signature.
Construction is layered across six structs so that each stage of an event's
life is a distinct type; you can't sign something that hasn't been hashed,
and you can't hash something that doesn't know its author.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod events;
```
```rust {file=coracle-lib/src/events.rs}
//! The nostr `Event` type, built up through a layered hierarchy of structs
//! that each add a single field: [`EventContent`], [`EventTemplate`],
//! [`StampedEvent`], [`OwnedEvent`], [`HashedEvent`], and [`Event`].
use serde::de::{self, MapAccess, Visitor};
use serde::ser::SerializeStruct;
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use sha2::{Digest, Sha256};
use std::fmt;
use crate::keys::PublicKey;
use crate::tags::Tags;
```
## Errors
```rust {file=coracle-lib/src/events.rs}
/// Errors that can occur when parsing or verifying an event.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum EventError {
/// The recomputed id did not match the event's `id` field.
InvalidId,
/// The Schnorr signature did not verify against the event's pubkey and id.
InvalidSignature,
}
impl fmt::Display for EventError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
EventError::InvalidId => write!(f, "event id does not match its contents"),
EventError::InvalidSignature => write!(f, "event signature failed to verify"),
}
}
}
impl std::error::Error for EventError {}
```
## The wire format
A signed event on the wire looks like this:
```json
{
"id": "4376c65d2f232afbe9b882a35baa4f6fe8667c4e684749af565f981833ed6a65",
"pubkey": "6e468422dfb74a5738702a8823b9b28168abab8655faacb6853cd0ee15deee93",
"created_at": 1673347337,
"kind": 1,
"tags": [["t", "nostr"]],
"content": "hello nostr",
"sig": "908a15e46fb4d8675bab026fc230a0e3542bfade63da02d542fb78b2a8513fcd0092619a2c8c1221e581946e0191f2af505dfdf8657a414dbca329186f009262"
}
```
`id`, `pubkey`, and `sig` are hex strings of fixed length. `created_at` is a
Unix timestamp in seconds. `kind` is a 16-bit integer. `tags` is a list of
lists of strings. `content` is a string whose meaning depends on the kind.
## Building up an event
Rather than one `Event` struct that may or may not have its id and signature
filled in, we define six structs, each of which adds exactly one field to the
previous. A value's type tells you which steps it has been through, and a
`.foo()` method takes you from one stage to the next.
### `EventContent`
The starting point is the payload the caller wants to publish: a
human-readable `content` string plus the structured `tags` that reference
other events, pubkeys, topics, and whatever else. Tags and content travel
together because tags are part of what the hash commits to — they are not
metadata in the "could be added later" sense.
`EventContent` is a builder. `new` hands back an empty payload and
chainable `content` and `tags` setters fill it in. Neither field is
required at construction time — a zero-length note with no tags is a
perfectly legal kind-1 event — and the builder shape means callers
don't have to pass `""` or `vec![]` placeholders when they only care
about one of the two.
```rust {file=coracle-lib/src/events.rs}
/// The content of an event: the human-readable body plus its tags.
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct EventContent {
pub content: String,
pub tags: Tags,
}
impl EventContent {
/// An empty payload: no content, no tags.
pub fn new() -> Self {
Self::default()
}
/// Set the human-readable content. Returns `self` so the call can
/// chain.
pub fn content(mut self, content: impl Into<String>) -> Self {
self.content = content.into();
self
}
/// Set the tags. Returns `self` so the call can chain.
pub fn tags(mut self, tags: impl Into<Tags>) -> Self {
self.tags = tags.into();
self
}
}
```
### `EventTemplate`
A template adds the numeric `kind` that decides how the content should be
interpreted. A kind-1 template is a short text note; a kind-6 template is a
repost; a kind-1984 template is a report. This chapter stays agnostic about
which kinds mean what — that's the next layer of the stack.
```rust {file=coracle-lib/src/events.rs}
/// Event content plus its kind.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct EventTemplate {
pub content: String,
pub tags: Tags,
pub kind: u16,
}
impl EventContent {
/// Tag the content with a kind, producing a publishable template.
pub fn kind(self, kind: u16) -> EventTemplate {
EventTemplate { content: self.content, tags: self.tags, kind }
}
}
```
### `StampedEvent`
Stamping adds the `created_at` timestamp. The library stays out of clock
policy — the timestamp is whatever the caller says it is — but in practice
"whatever the caller says" is almost always "right now," so we give that
common case a name. The `now` helper lives in a `util` module, the home for
the small, stateless helpers that don't belong to any one type.
```rust {file=coracle-lib/src/lib.rs}
pub mod util;
```
```rust {file=coracle-lib/src/util.rs}
//! Small stateless helpers shared across the library.
use std::time::{SystemTime, UNIX_EPOCH};
/// The current time as a Unix timestamp in seconds.
pub fn now() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
```
Stamping a template with the current time then reads as
`template.stamp(now())`. `stamp` itself takes a plain `u64`, leaving the
choice of clock to the caller.
```rust {file=coracle-lib/src/events.rs}
/// A template with a `created_at` timestamp attached.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct StampedEvent {
pub content: String,
pub kind: u16,
pub tags: Tags,
pub created_at: u64,
}
impl EventTemplate {
pub fn stamp(self, created_at: u64) -> StampedEvent {
StampedEvent {
content: self.content,
kind: self.kind,
tags: self.tags,
created_at,
}
}
}
```
### `OwnedEvent`
Ownership means attaching an author. This is where the `PublicKey` from
chapter 2 shows up: the event is about to be something that some specific key
claims responsibility for.
```rust {file=coracle-lib/src/events.rs}
/// A stamped event with its author's public key attached.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct OwnedEvent {
pub content: String,
pub kind: u16,
pub tags: Tags,
pub created_at: u64,
pub pubkey: PublicKey,
}
impl StampedEvent {
pub fn own(self, pubkey: PublicKey) -> OwnedEvent {
OwnedEvent {
content: self.content,
kind: self.kind,
tags: self.tags,
created_at: self.created_at,
pubkey,
}
}
}
```
### `HashedEvent`
Hashing produces the event id — the 32-byte SHA-256 that NIP-01 defines as
the hash of a specific canonical serialization. The canonical form is a JSON
*array*, not an object:
```text
[0, pubkey, created_at, kind, tags, content]
```
The leading `0` is a version marker reserved by the protocol. The order is
fixed. Every nostr implementation on the network has to produce byte-
identical output from the same fields, or ids wouldn't match.
```rust {file=coracle-lib/src/events.rs}
fn canonical(
pubkey: &PublicKey,
created_at: u64,
kind: u16,
tags: &Tags,
content: &str,
) -> String {
serde_json::json!([
0,
pubkey.to_hex(),
created_at,
kind,
tags,
content,
])
.to_string()
}
/// An owned event with its computed id.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct HashedEvent {
pub content: String,
pub kind: u16,
pub tags: Tags,
pub created_at: u64,
pub pubkey: PublicKey,
pub id: [u8; 32],
}
impl OwnedEvent {
pub fn hash(self) -> HashedEvent {
let id: [u8; 32] = Sha256::digest(
canonical(&self.pubkey, self.created_at, self.kind, &self.tags, &self.content)
.as_bytes(),
)
.into();
HashedEvent {
content: self.content,
kind: self.kind,
tags: self.tags,
created_at: self.created_at,
pubkey: self.pubkey,
id,
}
}
}
```
### `Event`
The final step attaches the signature and yields the type that goes on the
wire.
```rust {file=coracle-lib/src/events.rs}
/// A fully-formed nostr event: hashed and signed.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Event {
pub content: String,
pub kind: u16,
pub tags: Tags,
pub created_at: u64,
pub pubkey: PublicKey,
pub id: [u8; 32],
pub sig: [u8; 64],
}
impl HashedEvent {
pub fn sign(self, sig: [u8; 64]) -> Event {
Event {
content: self.content,
kind: self.kind,
tags: self.tags,
created_at: self.created_at,
pubkey: self.pubkey,
id: self.id,
sig,
}
}
}
```
The two-step split between `hash` and `sign` is deliberate. `hash` is pure
and knows nothing about keys; `sign` takes an already-computed signature as
input rather than doing the signing itself. That keeps the signing primitive
out of the event module, which is where we need it next.
## Signing a 32-byte message
We need exactly one new capability to drive the pipeline above end-to-end: a
way for a `SecretKey` to produce a Schnorr signature over 32 arbitrary bytes.
Not "sign an event" — just "sign these bytes." A future chapter will wrap
this in a `Signer` trait that knows about events; we want only one place in
the library where the cryptographic operation actually happens.
```rust {file=coracle-lib/src/keys.rs}
impl SecretKey {
/// Schnorr-sign a 32-byte message under this secret key. The digest
/// length is enforced at the type level; callers have to decide what
/// 32 bytes they're signing. Deterministic under BIP-340.
pub fn sign(&self, message: &[u8; 32]) -> [u8; 64] {
let keypair = secp256k1::Keypair::from_secret_key(SECP256K1, &self.0);
let msg = secp256k1::Message::from_digest(*message);
SECP256K1.sign_schnorr_no_aux_rand(&msg, &keypair).serialize()
}
}
```
With that in place, the full pipeline from "hello nostr" to a signed event is
one expression:
```rust
let hashed = EventContent::new()
.content("hello nostr")
.kind(1)
.stamp(1_700_000_000)
.own(secret.public_key())
.hash();
let event = hashed.clone().sign(secret.sign(&hashed.id));
```
The pipeline is illustrative, not tangled — the library does not ship an
`Event::sign_with` or `Signer` convenience yet. The later chapter on signers
wraps these inner two lines in a `Signer::sign_event` default so that remote
signers (NIP-07, NIP-46) plug into the same shape.
## Verification
Verification reverses the pipeline. Given an `Event`, recompute the canonical
form from its current fields, compare the resulting hash against the stored
`id`, and check the Schnorr signature.
```rust {file=coracle-lib/src/events.rs}
impl Event {
/// Recompute the event id and compare it to the stored `id`.
pub fn verify_id(&self) -> bool {
let expected: [u8; 32] = Sha256::digest(
canonical(&self.pubkey, self.created_at, self.kind, &self.tags, &self.content)
.as_bytes(),
)
.into();
expected == self.id
}
/// Verify both the id and the Schnorr signature.
pub fn verify(&self) -> Result<(), EventError> {
if !self.verify_id() {
return Err(EventError::InvalidId);
}
let msg = secp256k1::Message::from_digest(self.id);
let sig = secp256k1::schnorr::Signature::from_slice(&self.sig)
.map_err(|_| EventError::InvalidSignature)?;
let xonly = secp256k1::XOnlyPublicKey::from_slice(&self.pubkey.as_bytes())
.map_err(|_| EventError::InvalidSignature)?;
sig.verify(&msg, &xonly).map_err(|_| EventError::InvalidSignature)
}
}
```
The canonical-form helper is the single source of truth for "what bytes go
into the hash." Both `OwnedEvent::hash` and `Event::verify_id` call it, so
there is no way for the signing and verification paths to disagree about the
serialization.
## JSON
On the wire an event is the JSON object from the start of the chapter. The
binary fields — `id`, `pubkey`, and `sig` — are hex strings there, and
fixed-size byte arrays in memory. We hand-write `Serialize` and `Deserialize`
to bridge the two without pulling in `serde_with`.
```rust {file=coracle-lib/src/events.rs}
impl Serialize for Event {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let mut state = serializer.serialize_struct("Event", 7)?;
state.serialize_field("id", &hex::encode(self.id))?;
state.serialize_field("pubkey", &self.pubkey.to_hex())?;
state.serialize_field("created_at", &self.created_at)?;
state.serialize_field("kind", &self.kind)?;
state.serialize_field("tags", &self.tags)?;
state.serialize_field("content", &self.content)?;
state.serialize_field("sig", &hex::encode(self.sig))?;
state.end()
}
}
```
The deserializer walks the fields into `Option`s, validates hex shape at the
end, and silently ignores unknown keys — relays and clients sometimes attach
their own, and rejecting them would make this parser less permissive than
the network.
```rust {file=coracle-lib/src/events.rs}
impl<'de> Deserialize<'de> for Event {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
deserializer.deserialize_map(EventVisitor)
}
}
struct EventVisitor;
impl<'de> Visitor<'de> for EventVisitor {
type Value = Event;
fn expecting(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("a nostr event object")
}
fn visit_map<M: MapAccess<'de>>(self, mut map: M) -> Result<Event, M::Error> {
let mut id: Option<String> = None;
let mut pubkey: Option<String> = None;
let mut created_at: Option<u64> = None;
let mut kind: Option<u16> = None;
let mut tags: Option<Tags> = None;
let mut content: Option<String> = None;
let mut sig: Option<String> = None;
while let Some(key) = map.next_key::<String>()? {
match key.as_str() {
"id" => id = Some(map.next_value()?),
"pubkey" => pubkey = Some(map.next_value()?),
"created_at" => created_at = Some(map.next_value()?),
"kind" => kind = Some(map.next_value()?),
"tags" => tags = Some(map.next_value()?),
"content" => content = Some(map.next_value()?),
"sig" => sig = Some(map.next_value()?),
_ => {
let _: serde::de::IgnoredAny = map.next_value()?;
}
}
}
let id = decode_32(&id.ok_or_else(|| de::Error::missing_field("id"))?)
.map_err(|_| de::Error::custom("invalid hex in field `id`"))?;
let sig = decode_64(&sig.ok_or_else(|| de::Error::missing_field("sig"))?)
.map_err(|_| de::Error::custom("invalid hex in field `sig`"))?;
let pubkey = PublicKey::from_hex(
&pubkey.ok_or_else(|| de::Error::missing_field("pubkey"))?,
)
.map_err(|_| de::Error::custom("invalid hex in field `pubkey`"))?;
Ok(Event {
id,
pubkey,
created_at: created_at.ok_or_else(|| de::Error::missing_field("created_at"))?,
kind: kind.ok_or_else(|| de::Error::missing_field("kind"))?,
tags: tags.ok_or_else(|| de::Error::missing_field("tags"))?,
content: content.ok_or_else(|| de::Error::missing_field("content"))?,
sig,
})
}
}
fn decode_32(s: &str) -> Result<[u8; 32], ()> {
let bytes = hex::decode(s).map_err(|_| ())?;
bytes.try_into().map_err(|_| ())
}
fn decode_64(s: &str) -> Result<[u8; 64], ()> {
let bytes = hex::decode(s).map_err(|_| ())?;
let arr: [u8; 64] = bytes.try_into().map_err(|_| ())?;
Ok(arr)
}
```
Deserialization does not verify the signature. Parsing is cheap; `verify()`
is not, and there are many places where you want to read an event off disk
without paying for a signature check.
## A shared surface across event stages
There is a lot of overlap between various event structs. A few minimal accessor
traits capture read access to various fields so that we can define extensions
to events without duplication.
```rust {file=coracle-lib/src/events.rs}
pub trait HasTags {
fn tags(&self) -> &Tags;
}
pub trait HasId {
fn id(&self) -> &[u8; 32];
}
impl HasTags for HashedEvent {
fn tags(&self) -> &Tags {
&self.tags
}
}
impl HasTags for Event {
fn tags(&self) -> &Tags {
&self.tags
}
}
impl HasId for HashedEvent {
fn id(&self) -> &[u8; 32] {
&self.id
}
}
impl HasId for Event {
fn id(&self) -> &[u8; 32] {
&self.id
}
}
```
Subsequent chapters will add *extension traits* bounded on these accessors,
supply the method bodies as defaults, and close with a blanket impl over every
type that satisfies the bound. The logic is written once and reaches both event
types — and any event type added later — without another line per struct.
The expiration chapter's version reads like this:
```rust
pub trait EventExtensionExpiration: HasTags {
fn is_expired(&self) -> bool {
is_expired(self.tags())
}
// ...the rest of the expiration queries
}
impl<T: HasTags> EventExtensionExpiration for T {}
```
A trait's methods are only callable where the trait is in scope, so the
library gathers these extension traits in a `prelude`. One glob import brings
the whole behavior-query surface along:
```rust
use coracle_lib::prelude::*;
```
```rust {file=coracle-lib/src/lib.rs}
pub mod prelude;
```
```rust {file=coracle-lib/src/prelude.rs}
//! One-stop import for the extension traits that add behavior-tag queries to
//! events. With `coracle_lib::prelude::*` in scope, methods like
//! `is_expired`, `is_protected`, and `get_pow` become callable on
//! `HashedEvent` and `Event`.
pub use crate::events::{HasId, HasTags};
```
Each later chapter that defines an extension trait adds it to this prelude.
## What's next
The next chapter introduces kinds — the integer that decides how content
and tags on a given event should be read — and with it the beginning of
a taxonomy we have so far resisted building.
+274
View File
@@ -0,0 +1,274 @@
# Kinds
Every event carries a `kind` field — a 16-bit integer that says what the
event means. Kind 0 is a profile. Kind 1 is a short text note. Kind 5 is
a deletion request. Kind 1984 is a content report. The number is the only
thing that distinguishes a follow list from a zap receipt; the rest of the
event structure is identical.
Using integers instead of human-readable names is deliberate. In a protocol
built on signed, immutable data, names can never change once chosen. Numbers
carry no inherent meaning, which lets every implementer assign their own
terminology based on their understanding of a kind's purpose. The protocol
says "kind 0"; whether your library calls that `Metadata`, `Profile`, or
`UserInfo` is your business.
This chapter does not catalog individual kinds — a future series of chapters
will do that. Instead it builds the infrastructure that those chapters will use:
a trait for attaching behavior to a kind number, free functions for
classifying raw kind values, and a concrete example to prove the pattern
works.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod kinds;
```
```rust {file=coracle-lib/src/kinds.rs}
//! Event kind classification and the [`Kind`] trait for attaching
//! domain-specific behavior to kind numbers.
use std::fmt;
use crate::events::{Event, EventContent, EventTemplate};
```
## Classifying a kind number
The nostr protocol divides the 16-bit kind space into four ranges, each
with different storage and replacement semantics:
| Range | Name | Behavior |
|-------|------|----------|
| 09999 | Regular | Stored by relays, no replacement logic |
| 1000019999 | Replaceable | Relay keeps only the latest event per author and kind |
| 2000029999 | Ephemeral | Relays broadcast but do not store |
| 3000039999 | Addressable | Like replaceable, but partitioned by a `d` tag |
Two exceptions break the pattern: kinds 0 and 3 sit below 10000 but are
replaceable. A user's profile (kind 0) and follow list (kind 3) predate
the range system and were grandfathered in.
These four functions encode the rules so that the rest of the library doesn't
have to remember them:
```rust {file=coracle-lib/src/kinds.rs}
/// A regular event: stored by relays with no replacement logic.
pub fn is_regular(kind: u16) -> bool {
kind != 0 && kind != 3 && kind < 10_000
}
/// A replaceable event: relays keep only the latest per (author, kind).
/// Kinds 0 and 3 are replaceable despite falling below 10000.
pub fn is_replaceable(kind: u16) -> bool {
kind == 0 || kind == 3 || (10_000 <= kind && kind < 20_000)
}
/// An ephemeral event: relays broadcast it but do not store it.
pub fn is_ephemeral(kind: u16) -> bool {
20_000 <= kind && kind < 30_000
}
/// An addressable event: like replaceable, but the relay partitions by
/// the event's `d` tag in addition to author and kind.
pub fn is_addressable(kind: u16) -> bool {
30_000 <= kind && kind < 40_000
}
```
These are free functions rather than methods on a type because plenty of
code needs to classify a raw `u16` without having a `Kind` implementor
in hand. Relay logic, filter construction, and storage layers all work
with kind numbers directly.
## Errors
Parsing an event into a kind-specific type can fail for two reasons: the
event's kind doesn't match what the parser expects, or the content and
tags don't conform to the kind's format.
```rust {file=coracle-lib/src/kinds.rs}
/// Errors that can occur when parsing an event into a kind-specific
/// domain type.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum KindError {
/// The event's kind field does not match the expected kind number.
WrongKind {
expected: u16,
got: u16,
},
/// The event's content or tags could not be parsed for this kind.
InvalidContent(String),
}
impl fmt::Display for KindError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
KindError::WrongKind { expected, got } => {
write!(f, "expected kind {expected}, got {got}")
}
KindError::InvalidContent(msg) => {
write!(f, "invalid content for kind: {msg}")
}
}
}
}
impl std::error::Error for KindError {}
```
## The `Kind` trait
A kind is more than a number — it's a contract about what an event's
content and tags mean and how to work with them. The `Kind` trait captures
that contract. Implement it for a domain type and you get:
- A kind number constant that ties the type to a specific integer, plus
a `kind()` method to access it from an instance.
- A `from_event` entry point that checks the kind and delegates to your
parser.
- A `to_template` method that serializes your type back into an event.
The set of kinds is open-ended — every NIP can introduce new ones — so
a trait is the right abstraction. Each module that handles a specific kind
implements the trait for its own type without touching a central enum or
registry.
```rust {file=coracle-lib/src/kinds.rs}
/// A domain type that corresponds to a specific event kind.
///
/// Implementors provide a kind number, a parser that reads an event's
/// content and tags into the domain type, and a serializer that
/// converts it back into an event template. The trait provides a
/// `from_event` entry point that checks the kind number before parsing.
pub trait Kind: Sized {
/// The kind number this type handles.
const KIND: u16;
/// Parse an event's content and tags into this domain type.
///
/// This is called by [`from_event`](Kind::from_event) after the kind
/// number check passes, so implementations can assume the kind is
/// correct.
fn parse(event: &Event) -> Result<Self, KindError>;
/// Convert this domain type into an event's content and tags.
fn to_content(&self) -> EventContent;
/// The kind number for this instance.
fn kind(&self) -> u16 {
Self::KIND
}
/// Convert this domain type into an event template.
///
/// The default implementation calls [`to_content`](Kind::to_content)
/// and attaches the kind number. Override this only if you need
/// non-standard behavior.
fn to_template(&self) -> EventTemplate {
self.to_content().kind(Self::KIND)
}
/// Parse an event into this domain type, checking the kind number
/// first.
fn from_event(event: &Event) -> Result<Self, KindError> {
if event.kind != Self::KIND {
return Err(KindError::WrongKind {
expected: Self::KIND,
got: event.kind,
});
}
Self::parse(event)
}
}
```
The trait has one associated constant and two required methods.
Implementors provide `parse` (event to domain type) and `to_content`
(domain type to event content). The trait provides `kind()` for
instance access, `to_template()` to attach the kind number, and
`from_event()` to check the kind before parsing.
Classification lives in the free functions from earlier in the chapter.
Given any `Kind` implementor, `is_replaceable(deletion.kind())` tells
you what you need to know.
The `to_content` / `to_template` split keeps implementors from having to
repeat the kind number. `to_content` returns the content and tags;
`to_template` attaches the kind. Since the kind is already known from
the constant, there's no reason to ask every implementor to specify it
again.
## A concrete example
Kind 5 is a deletion request. Its content is an optional human-readable
reason, and its tags reference the events or addresses being deleted. It
makes a good first implementation of the `Kind` trait: simple enough to
fit in a few lines, real enough to exercise every part of the interface.
```rust
use coracle_lib::kinds::{Kind, KindError};
use coracle_lib::events::{Event, EventContent};
use coracle_lib::tags::Tags;
/// A deletion request (kind 5). The content is an optional reason
/// string, and the tags reference the events or addresses to delete.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Deletion {
/// An optional human-readable reason for the deletion.
pub reason: String,
/// The tags from the original event, which reference the events
/// or addresses being deleted.
pub tags: Tags,
}
impl Kind for Deletion {
const KIND: u16 = 5;
fn parse(event: &Event) -> Result<Self, KindError> {
Ok(Deletion {
reason: event.content.clone(),
tags: event.tags.clone(),
})
}
fn to_content(&self) -> EventContent {
EventContent::new()
.content(self.reason.clone())
.tags(self.tags.clone())
}
}
```
With this implementation, parsing a deletion event is:
```rust
let deletion = Deletion::from_event(&event)?;
```
And creating an event template from a `Deletion` value is:
```rust
let template = deletion.to_template();
assert_eq!(template.kind, 5);
```
Classification uses the free functions with the instance's kind number:
```rust
use coracle_lib::kinds;
assert!(kinds::is_regular(deletion.kind()));
assert!(!kinds::is_replaceable(deletion.kind()));
```
Later chapters will follow this same pattern for profiles, follow lists,
reactions, zaps, and everything else. The shape is always the same:
define a struct, implement `Kind`, and the trait provides the rest.
## What's next
The next chapter introduces addresses — the stable `kind:pubkey:identifier`
references that let you point at a replaceable or addressable event without
tying yourself to a particular version's id.
+408
View File
@@ -0,0 +1,408 @@
# Addresses
An event's id is a hash of its contents. Change the content, sign again, and
you get a different id. That's fine for regular events — they're immutable,
so the id is a permanent name. But replaceable and addressable events are
designed to be overwritten. A user's profile (kind 0) updates every time they
change their display name. An article (kind 30023) updates every time the
author edits a paragraph. The old id is gone; a new one takes its place.
You need a way to point at the *slot* rather than the *version*. That's what
an address is: the triple `kind:pubkey:identifier` that stays the same no
matter how many times the event is replaced. For a replaceable event (kind
0, 3, or 1000019999), the identifier is empty — the relay keeps one event
per author and kind. For an addressable event (kind 3000039999), the
identifier is the `d` tag's value, which partitions the kind further so
that a single author can maintain many independent slots under the same kind.
The `a` tag carries this triple as its value, and the `naddr` bech32 encoding
wraps it for sharing. This chapter builds the `Address` type that handles
both.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod addresses;
```
```rust {file=coracle-lib/src/addresses.rs}
//! Event addresses: stable references to replaceable and addressable events.
//!
//! An [`Address`] is the triple `(kind, pubkey, identifier)` that names the
//! slot a mutable event occupies, independent of any particular version.
use std::fmt;
use std::str::FromStr;
use bech32::{Bech32, Hrp};
use crate::events::Event;
use crate::keys::{KeyError, PublicKey};
use crate::kinds;
```
## Errors
Address operations can fail when parsing a string or decoding bech32. A
single error type covers all the ways.
```rust {file=coracle-lib/src/addresses.rs}
/// Errors that can occur when parsing or encoding an address.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AddressError {
/// The string could not be parsed as `kind:pubkey:identifier`.
InvalidFormat,
/// The bech32 string was malformed.
InvalidBech32,
/// The bech32 prefix was not `naddr`.
WrongPrefix { found: String },
/// The public key bytes were not a valid secp256k1 point.
InvalidKey,
/// A required TLV field was missing.
FieldMissing(&'static str),
/// The TLV data was malformed.
TlvError,
}
impl fmt::Display for AddressError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
AddressError::InvalidFormat => write!(f, "invalid address format"),
AddressError::InvalidBech32 => write!(f, "invalid bech32 encoding"),
AddressError::WrongPrefix { found } => {
write!(f, "wrong bech32 prefix: expected naddr, found {found}")
}
AddressError::InvalidKey => write!(f, "invalid public key"),
AddressError::FieldMissing(name) => {
write!(f, "missing required field: {name}")
}
AddressError::TlvError => write!(f, "malformed TLV data"),
}
}
}
impl std::error::Error for AddressError {}
impl From<KeyError> for AddressError {
fn from(_: KeyError) -> Self {
AddressError::InvalidKey
}
}
```
## The `Address` struct
An address has three fields that identify the slot — `kind`, `pubkey`, and
`identifier` — plus a `hints` field that carries relay URLs where the event
might be found. The hints are transport metadata: they travel inside `naddr`
bech32 strings and `a` tags but are not part of the logical identity. Two
addresses that differ only in their hints point at the same event.
```rust {file=coracle-lib/src/addresses.rs}
/// A stable reference to a replaceable or addressable event.
///
/// The triple `(kind, pubkey, identifier)` names the slot a mutable event
/// occupies. Two events with the same address are versions of the same
/// logical object — the relay keeps whichever has the later `created_at`.
///
/// `hints` carries relay URLs where the event might be found. These are
/// transport metadata used when encoding to `naddr` or `a` tags, but are
/// not part of the address identity — `PartialEq` and `Hash` ignore them.
#[derive(Debug, Clone)]
pub struct Address {
/// The event kind.
pub kind: u16,
/// The author's public key.
pub pubkey: PublicKey,
/// The `d` tag value. Empty for plain replaceable events.
pub identifier: String,
/// Relay URLs where this event might be found.
pub hints: Vec<String>,
}
impl PartialEq for Address {
fn eq(&self, other: &Self) -> bool {
self.kind == other.kind
&& self.pubkey == other.pubkey
&& self.identifier == other.identifier
}
}
impl Eq for Address {}
impl std::hash::Hash for Address {
fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
self.kind.hash(state);
self.pubkey.hash(state);
self.identifier.hash(state);
}
}
impl Address {
/// Create a new address from its components.
pub fn new(kind: u16, pubkey: PublicKey, identifier: impl Into<String>) -> Self {
Address {
kind,
pubkey,
identifier: identifier.into(),
hints: Vec::new(),
}
}
/// Attach relay hints to this address.
pub fn hints<I, S>(mut self, relays: I) -> Self
where
I: IntoIterator<Item = S>,
S: Into<String>,
{
self.hints = relays.into_iter().map(Into::into).collect();
self
}
}
```
`PartialEq` and `Hash` are hand-implemented to exclude `hints`. Two
addresses that name the same slot are equal regardless of which relays
they suggest — the hints are advice, not identity.
### Building from an event
The most common way to get an address is from an event you already have.
For addressable events, the identifier comes from the `d` tag (defaulting
to the empty string if absent). For plain replaceable events, the
identifier is always empty — even if a `d` tag happens to be present, it
has no meaning for replacement semantics, so we ignore it. If the kind is
neither replaceable nor addressable, there is no address — the event is
immutable and identified by its id alone.
```rust {file=coracle-lib/src/addresses.rs}
impl Address {
/// Extract the address from an event, if the event's kind is
/// replaceable or addressable.
///
/// Returns `None` for regular and ephemeral events. The returned
/// address has no relay hints — call `.hints(...)` to attach them.
pub fn from_event(event: &Event) -> Option<Self> {
let identifier = if kinds::is_addressable(event.kind) {
event.tags.value("d").unwrap_or("").to_string()
} else if kinds::is_replaceable(event.kind) {
String::new()
} else {
return None;
};
Some(Address {
kind: event.kind,
pubkey: event.pubkey,
identifier,
hints: Vec::new(),
})
}
}
```
## String format
The canonical string form of an address is `kind:pubkey:identifier` — the
same value that goes inside an `a` tag. `Display` writes it, `FromStr`
parses it.
Parsing splits on `:` and takes the first segment as the kind, the second
as the pubkey hex, and everything after the second colon as the identifier.
Joining the remainder handles identifiers that themselves contain colons,
which is unusual but legal.
```rust {file=coracle-lib/src/addresses.rs}
impl fmt::Display for Address {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}:{}:{}", self.kind, self.pubkey, self.identifier)
}
}
impl FromStr for Address {
type Err = AddressError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
// Try naddr bech32 first.
if s.starts_with("naddr1") {
return Self::from_naddr(s);
}
let mut parts = s.splitn(3, ':');
let kind: u16 = parts
.next()
.and_then(|s| s.parse().ok())
.ok_or(AddressError::InvalidFormat)?;
let pubkey = parts
.next()
.ok_or(AddressError::InvalidFormat)
.and_then(|s| PublicKey::from_hex(s).map_err(|_| AddressError::InvalidFormat))?;
let identifier = parts.next().unwrap_or("").to_string();
Ok(Address {
kind,
pubkey,
identifier,
hints: Vec::new(),
})
}
}
```
`FromStr` also accepts `naddr1…` strings, so callers can parse either
format with a single `.parse()` call — the same convenience we used for
public keys accepting both hex and `npub`.
## NIP-19 `naddr` encoding
The simple bech32 encodings we built in the keys chapter — `npub` and `nsec`
— wrap a fixed 32-byte payload. An address carries variable-length data: the
identifier can be any length, and there may be zero or more relay hints. NIP-19
handles this with a **TLV** (type-length-value) scheme inside the bech32
payload.
Each TLV entry is:
| Byte | Meaning |
|------|---------|
| 0 | Type tag: what this entry represents |
| 1 | Length: how many bytes follow |
| 2..2+len | Value |
Four type tags are defined:
| Type | Name | Value |
|------|------|-------|
| 0 | special | The identifier (`d` tag), UTF-8 bytes |
| 1 | relay | A relay URL, UTF-8 bytes |
| 2 | author | The pubkey, 32 raw bytes |
| 3 | kind | The kind number, 4-byte big-endian u32 |
An `naddr` payload concatenates one entry for the identifier, one for
the author, one for the kind, and zero or more for relay hints. The
order of these entries is identifier first by convention, but decoders
should not rely on ordering.
```rust {file=coracle-lib/src/addresses.rs}
const NADDR_HRP: Hrp = Hrp::parse_unchecked("naddr");
const TLV_SPECIAL: u8 = 0;
const TLV_RELAY: u8 = 1;
const TLV_AUTHOR: u8 = 2;
const TLV_KIND: u8 = 3;
```
### Encoding
```rust {file=coracle-lib/src/addresses.rs}
impl Address {
/// Encode as an `naddr1…` bech32 string per NIP-19.
///
/// Any relay hints on this address are embedded in the TLV payload.
pub fn to_naddr(&self) -> String {
let id_bytes = self.identifier.as_bytes();
let relay_len: usize = self.hints.iter().map(|r| 2 + r.len()).sum();
let capacity = (2 + id_bytes.len()) + (2 + 32) + (2 + 4) + relay_len;
let mut buf = Vec::with_capacity(capacity);
// Identifier (TLV type 0)
buf.push(TLV_SPECIAL);
buf.push(id_bytes.len() as u8);
buf.extend_from_slice(id_bytes);
// Relays (TLV type 1, one entry per relay)
for relay in &self.hints {
buf.push(TLV_RELAY);
buf.push(relay.len() as u8);
buf.extend_from_slice(relay.as_bytes());
}
// Author (TLV type 2)
buf.push(TLV_AUTHOR);
buf.push(32);
buf.extend_from_slice(&self.pubkey.as_bytes());
// Kind (TLV type 3, 4-byte big-endian u32)
buf.push(TLV_KIND);
buf.push(4);
buf.extend_from_slice(&(self.kind as u32).to_be_bytes());
bech32::encode::<Bech32>(NADDR_HRP, &buf)
.expect("naddr encoding cannot fail for well-formed TLV")
}
/// Decode an `naddr1…` bech32 string into an address.
///
/// Any relay hints embedded in the payload are stored in the
/// returned address's `hints` field.
pub fn from_naddr(s: &str) -> Result<Self, AddressError> {
let (hrp, data) = bech32::decode(s).map_err(|_| AddressError::InvalidBech32)?;
if hrp != NADDR_HRP {
return Err(AddressError::WrongPrefix {
found: hrp.to_string(),
});
}
let mut identifier: Option<String> = None;
let mut pubkey: Option<PublicKey> = None;
let mut kind: Option<u16> = None;
let mut hints: Vec<String> = Vec::new();
let mut pos = 0;
while pos < data.len() {
let t = *data.get(pos).ok_or(AddressError::TlvError)?;
let l = *data.get(pos + 1).ok_or(AddressError::TlvError)? as usize;
let value = data
.get(pos + 2..pos + 2 + l)
.ok_or(AddressError::TlvError)?;
pos += 2 + l;
match t {
TLV_SPECIAL => {
if identifier.is_none() {
identifier = Some(
String::from_utf8(value.to_vec())
.map_err(|_| AddressError::TlvError)?,
);
}
}
TLV_RELAY => {
if let Ok(url) = String::from_utf8(value.to_vec()) {
hints.push(url);
}
}
TLV_AUTHOR => {
if pubkey.is_none() {
pubkey = Some(PublicKey::from_bytes(value)?);
}
}
TLV_KIND => {
if kind.is_none() {
let bytes: [u8; 4] =
value.try_into().map_err(|_| AddressError::TlvError)?;
kind = Some(u32::from_be_bytes(bytes) as u16);
}
}
_ => {} // Unknown TLV types are ignored for forward compatibility.
}
}
Ok(Address {
kind: kind.ok_or(AddressError::FieldMissing("kind"))?,
pubkey: pubkey.ok_or(AddressError::FieldMissing("pubkey"))?,
identifier: identifier.ok_or(AddressError::FieldMissing("identifier"))?,
hints,
})
}
}
```
The decoder takes first-occurrence-wins for each field type, which matches
what every other implementation does. Unknown TLV types are silently
skipped — future NIPs may add new ones, and rejecting them would break
forward compatibility.
## What's next
We now have a way to name mutable events that survives content changes.
The next chapter introduces proof of work — the optional CPU cost a client
can burn into an event id to signal effort and reduce spam.
+322
View File
@@ -0,0 +1,322 @@
# Proof of Work
Every event id is a SHA-256 hash. Most of the time that hash is
whatever it happens to be — nobody cares about the specific bit
pattern. Proof of work changes that. NIP-13 defines a convention where
a client burns CPU time searching for a nonce that makes the event id
start with a target number of leading zero bits. The result is an event
that carries a verifiable proof of computational effort baked into its
identity.
The purpose is spam resistance. A relay can require that incoming events
have, say, 20 leading zero bits. Generating one costs the publisher
roughly 2^20 hash attempts — a second or two on modern hardware — but
spamming a thousand events costs a thousand times that. It is not a
substitute for moderation, but it raises the floor.
The protocol mechanism is a single tag: `["nonce", "<counter>",
"<target>"]`. The first value is the nonce that was found; the second
is the difficulty the miner was aiming for. Both are needed: the nonce
changes the event id (because tags are part of the hash input), and the
target records intent so that a verifier can distinguish a miner who
committed to 20 bits of work from one who committed to 4 and got lucky.
This chapter builds three things: a function to count leading zero bits,
a function to read the validated proof-of-work difficulty from an event,
and a pair of mining functions that search for a valid nonce and return
a `HashedEvent` with the proof embedded. The mining API is batch-based:
callers control the loop, which keeps the library free of
platform-specific threading or async machinery.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod pow;
```
```rust {file=coracle-lib/src/pow.rs}
//! NIP-13 proof of work: difficulty validation and mining.
//!
//! [`mine_pow`] blocks until a valid nonce is found. [`mine_pow_batch`]
//! tries a bounded range of nonces and returns `None` if the batch is
//! exhausted, giving the caller control over cancellation and
//! parallelization.
use sha2::{Digest, Sha256};
use crate::events::{HasId, HasTags, HashedEvent, OwnedEvent};
use crate::tags::{Tag, Tags};
```
## Counting leading zero bits
Everything else in this module builds on one primitive: counting how many
leading bits of a byte slice are zero. A hash with 20 leading zero bits
starts with at least five zero hex characters (20 / 4 = 5). Difficulty
is measured in bits, not hex characters, because bits give finer
granularity — you can require 21 bits of work, not just 20 or 24.
The algorithm walks through bytes. Each fully-zero byte contributes
eight bits. The first non-zero byte contributes its own leading zeros
(via the `leading_zeros` intrinsic, which compiles to a single CPU
instruction on most architectures), and then we stop — no byte after
that can contribute leading zeros.
```rust {file=coracle-lib/src/pow.rs}
/// Count the number of leading zero bits in a byte slice.
///
/// Returns 0 for an empty slice. For a 32-byte SHA-256 hash, the
/// maximum return value is 256.
pub fn get_leading_zero_bits(hash: &[u8]) -> u8 {
let mut count: u8 = 0;
for &byte in hash {
if byte == 0 {
count += 8;
} else {
count += byte.leading_zeros() as u8;
break;
}
}
count
}
```
## Validation
Validating proof of work on an existing event means answering the
question: how much work did this event *commit to*? That's the minimum
of two values — the actual leading zero bits in the hash and the
difficulty claimed in the nonce tag. The hash proves work was done; the
tag proves intent. A miner who targeted difficulty 4 and happened to
land on 20 zero bits only committed to 4 bits of work.
The core logic takes just an id and tags so it can be shared across
event types.
```rust {file=coracle-lib/src/pow.rs}
/// Return the validated proof-of-work difficulty for an event id and tags.
///
/// The result is the minimum of the actual leading zero bits in `id`
/// and the difficulty target claimed in the `nonce` tag. Returns 0 if
/// no nonce tag is present.
pub fn get_pow(id: &[u8; 32], tags: &Tags) -> u8 {
let leading = get_leading_zero_bits(id);
let claimed: u8 = tags
.find("nonce")
.and_then(|t| t.get(2))
.and_then(|s| s.parse().ok())
.unwrap_or(0);
leading.min(claimed)
}
```
The nonce tag's third entry (index 2) holds the difficulty target as a
decimal string. If the tag is absent or the value doesn't parse, the
claimed target is zero, so `get_pow` returns zero — no verifiable
commitment was made.
### Methods on event types
Proof of work is the first query that needs the event id as well as its
tags, so its extension trait is bounded on both `HasId` and `HasTags` from
the events chapter. The method body delegates to the free function through
those accessors, and the blanket impl puts `get_pow` on every type that
satisfies the bound — both `HashedEvent` and `Event` do. Callers write
`event.get_pow() >= 20` regardless of whether the event has been signed yet,
as long as the trait is in scope (via `coracle_lib::prelude::*`).
```rust {file=coracle-lib/src/pow.rs}
/// Proof-of-work queries on any event that has an id and tags.
pub trait EventExtensionProofOfWork: HasId + HasTags {
/// Return the validated proof-of-work difficulty of this event.
///
/// The result is the minimum of the actual leading zero bits in the
/// event id and the difficulty claimed in the nonce tag.
fn get_pow(&self) -> u8 {
get_pow(self.id(), self.tags())
}
}
impl<T: HasId + HasTags> EventExtensionProofOfWork for T {}
```
```rust {file=coracle-lib/src/prelude.rs}
pub use crate::pow::EventExtensionProofOfWork;
```
## Mining
Mining is a brute-force search. For each candidate nonce, the miner
builds a nonce tag, constructs the canonical serialization that the
events chapter defined, hashes it, and checks the leading zero bits. If
the hash meets the target, the search is over; if not, it tries the
next nonce.
The canonical form is the same JSON array from the events chapter:
`[0, pubkey, created_at, kind, tags, content]`. The nonce tag is
appended to the event's existing tags before serialization, so the
nonce value feeds into the hash. Changing the nonce changes the hash —
that's the whole mechanism.
Two free functions expose this search at different levels of control,
and `OwnedEvent` gets methods that delegate to them.
### `mine_pow` — the simple interface
For callers who just want a result and are willing to block until it
arrives:
```rust {file=coracle-lib/src/pow.rs}
/// Mine proof of work for an event, blocking until a valid nonce is found.
///
/// Appends a `["nonce", "<counter>", "<difficulty>"]` tag to the event's
/// tags and searches for a nonce that produces an event id with at least
/// `difficulty` leading zero bits. Returns the resulting `HashedEvent`.
///
/// This function does not return until a solution is found. For
/// cancellation or parallelization, use [`mine_pow_batch`] instead.
pub fn mine_pow(event: &OwnedEvent, difficulty: u8) -> HashedEvent {
let mut start: u64 = 0;
loop {
let batch_size: u64 = 1_000_000;
if let Some(result) = mine_pow_batch(event, difficulty, start, batch_size) {
return result;
}
start += batch_size;
}
}
```
### `mine_pow_batch` — the batch interface
For callers who need control over when to stop or how to distribute
work across threads or web workers:
```rust {file=coracle-lib/src/pow.rs}
/// Try to mine proof of work over a bounded range of nonces.
///
/// Searches nonces from `start` to `start + count - 1`. Returns
/// `Some(HashedEvent)` if a nonce producing at least `difficulty`
/// leading zero bits is found, or `None` if the entire range is
/// exhausted without a match.
///
/// # Parallelization
///
/// To distribute work across N workers, give each worker a
/// non-overlapping range: worker *i* calls
/// `mine_pow_batch(event, difficulty, i * chunk, chunk)`.
/// The first worker to return `Some` wins; the rest can be cancelled
/// by simply not issuing further batches.
pub fn mine_pow_batch(
event: &OwnedEvent,
difficulty: u8,
start: u64,
count: u64,
) -> Option<HashedEvent> {
let difficulty_str = difficulty.to_string();
let mut tags = event.tags.clone();
tags.0.push(Tag::new("nonce", ["0", &difficulty_str]));
let nonce_idx = tags.0.len() - 1;
let end = start.saturating_add(count);
for nonce in start..end {
tags.0[nonce_idx].0[1] = nonce.to_string();
let canonical = serde_json::json!([
0,
event.pubkey.to_hex(),
event.created_at,
event.kind,
&tags,
&event.content,
])
.to_string();
let id: [u8; 32] = Sha256::digest(canonical.as_bytes()).into();
if get_leading_zero_bits(&id) >= difficulty {
return Some(HashedEvent {
content: event.content.clone(),
kind: event.kind,
tags,
created_at: event.created_at,
pubkey: event.pubkey,
id,
});
}
}
None
}
```
The batch function clones the event's tags once, appends a placeholder
nonce tag, then mutates the nonce value in place on each iteration.
The canonical JSON is rebuilt every iteration — the nonce string changes
each time, so the serialization must too — but the tag vector itself is
reused rather than reallocated.
The returned `HashedEvent` includes the winning nonce tag in its tags.
From there it slots into the normal event pipeline: call `.sign()` to
produce a full `Event` ready for the wire.
### Methods on `OwnedEvent`
For callers who prefer the method style, `OwnedEvent` delegates to the
free functions:
```rust {file=coracle-lib/src/pow.rs}
impl OwnedEvent {
/// Mine proof of work, blocking until a valid nonce is found.
///
/// See [`mine_pow`] for details.
pub fn mine_pow(&self, difficulty: u8) -> HashedEvent {
mine_pow(self, difficulty)
}
/// Try to mine proof of work over a bounded range of nonces.
///
/// See [`mine_pow_batch`] for details.
pub fn mine_pow_batch(
&self,
difficulty: u8,
start: u64,
count: u64,
) -> Option<HashedEvent> {
mine_pow_batch(self, difficulty, start, count)
}
}
```
## Usage patterns
The split between `mine_pow` and `mine_pow_batch` keeps the library
code simple while supporting a range of caller strategies:
**Blocking single-threaded** — the common case. `event.mine_pow(20)`
blocks until it finds a valid nonce and returns the `HashedEvent`.
**Batched with cancellation** — a UI that wants to let the user abort.
The caller loops over `event.mine_pow_batch(20, cursor, 100_000)`,
incrementing `cursor` by 100,000 each round. Between rounds it checks
whether the user pressed cancel.
**Parallel native threads** — each of N threads gets a non-overlapping
chunk of the nonce space. The first thread to return `Some` wins; the
rest are abandoned.
**WASM web workers** — the main thread posts a batch range to a worker
via `postMessage`. The worker calls `mine_pow_batch`, posts the result
back. To cancel, the main thread simply stops posting new batches. No
shared memory, no atomics, no platform-specific API in the library.
**Validation** — a relay checking incoming events writes
`event.get_pow() >= 20`. The method works on both `HashedEvent` and
`Event`.
## What's next
The next chapter introduces filters — the query language clients use to
ask relays for events matching a set of criteria.
+191
View File
@@ -0,0 +1,191 @@
# Expiring Events
Not every event is meant to last. A chat message, a transient status, a
time-limited invitation — these are events whose value decays to zero
at a known point in the future. NIP-40 gives them a way to say so: an
`expiration` tag carrying a Unix timestamp in seconds. After that
timestamp, the event is considered no longer valid.
The tag is a **behavior tag** — orthogonal to the event's kind. A kind
1 note with an expiration is still a note; the expiration is a request
that relays stop serving it and clients stop displaying it once the
time is up. The distinction matters. Nostr clients and relays already
agreed, per kind, on what the payload *means*. The behavior tag layer
lets authors add a separate instruction about what the network should
*do* with the event, without reopening the kind's semantics.
Expiration is a cooperative hint, not a cryptographic guarantee.
Anyone who has a copy of the event keeps that copy; the signature
remains valid forever. What the tag buys is a shared convention:
well-behaved relays will refuse to serve the event after the
timestamp, well-behaved clients will hide it, and storage layers that
care about disk space can sweep it. The library's job at this level is
only to parse the tag and answer "is this event expired?" — deciding
what to *do* about that answer belongs to code further up the stack.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod expiration;
```
```rust {file=coracle-lib/src/expiration.rs}
//! NIP-40 expiring events: reading the `expiration` tag and checking
//! whether an event has expired.
//!
//! The free functions in this module operate on [`Tags`] so they can be
//! used wherever a tag slice is in hand. The [`EventExtensionExpiration`]
//! trait layers method-call sugar on top for any event type, reachable
//! through the crate prelude. The tag itself is built with
//! [`Tag::expiration`] from the tags module.
use crate::events::HasTags;
use crate::tags::Tags;
use crate::util::now;
```
## Reading the tag
The shape on the wire is `["expiration", "<unix_seconds>"]`. Parsing it
means finding the first tag with that name and interpreting its value
as a `u64`. Two ways to read this can go wrong — the tag may be absent,
or its value may not parse as a number — and both collapse to the same
answer: `None`. A malformed tag is indistinguishable from no tag at
all; neither tells us when the event should be considered expired, so
neither should produce a different downstream behavior.
```rust {file=coracle-lib/src/expiration.rs}
/// Return the expiration timestamp from a tag set, if present.
///
/// Reads the first `expiration` tag and parses its value as a Unix
/// timestamp in seconds. Returns `None` if the tag is missing or its
/// value does not parse as a `u64` — treating "malformed" the same as
/// "absent" avoids forcing every caller to distinguish a case they
/// cannot usefully act on.
pub fn get_expiration(tags: &Tags) -> Option<u64> {
tags.value("expiration").and_then(|v| v.parse().ok())
}
```
## A testable predicate
A predicate that reads the system clock is awkward to test. The
standard fix — take the clock as a parameter — costs nothing here
because the caller usually has a `now` value on hand anyway (a relay
checking a batch of incoming events picks a timestamp once and reuses
it, a UI rendering a timeline pins "now" at render time). The
two-function split keeps both audiences happy: callers who already
have a timestamp pass it in, and callers who just want to check
against wall-clock time call the shorter form that fills it in for
them.
The comparison is strict: `expiration < now`. An event whose
expiration equals the current time is still valid; it becomes expired
the moment `now` advances past it. "Expires at T" reads naturally as
"invalid after T," and that reading matches what every reference
implementation does.
```rust {file=coracle-lib/src/expiration.rs}
/// Test whether the tags declare an expiration that has already
/// passed at the given `now` timestamp.
///
/// Returns `false` if no expiration tag is present. The comparison is
/// strict: an event with expiration equal to `now` is *not* yet
/// expired.
pub fn is_expired_at(tags: &Tags, now: u64) -> bool {
matches!(get_expiration(tags), Some(t) if t < now)
}
```
## Against the system clock
The wall-clock form is a one-liner over `is_expired_at`, filling in the
timestamp from `crate::util::now`.
```rust {file=coracle-lib/src/expiration.rs}
/// Test whether the tags declare an expiration that has already
/// passed, relative to the system clock.
///
/// For batch checks or testable callers, prefer [`is_expired_at`] and
/// pass an explicit timestamp.
pub fn is_expired(tags: &Tags) -> bool {
is_expired_at(tags, now())
}
```
## Building the tag
We can add a small convenience method for building this tag:
```rust {file=coracle-lib/src/tags.rs}
impl Tag {
/// Build a NIP-40 `expiration` tag with the given Unix timestamp in seconds.
pub fn expiration(timestamp: u64) -> Tag {
Tag::new("expiration", [timestamp.to_string()])
}
}
```
## Methods on event types
Rather than repeat these queries on each event struct, expiration exposes
them through an extension trait bounded on [`HasTags`]. The method bodies
live once, as defaults, and a blanket impl makes them available on every
type that can hand back its tags — `HashedEvent`, `Event`, and anything
added later. With `coracle_lib::prelude::*` in scope, `event.is_expired()`
reads the same whether the event is hashed or signed.
```rust {file=coracle-lib/src/expiration.rs}
/// Expiration queries on any event that carries tags.
pub trait EventExtensionExpiration: HasTags {
/// Return the expiration timestamp of this event, if any.
fn get_expiration(&self) -> Option<u64> {
get_expiration(self.tags())
}
/// Whether this event has expired relative to the given timestamp.
fn is_expired_at(&self, now: u64) -> bool {
is_expired_at(self.tags(), now)
}
/// Whether this event has expired relative to the system clock.
fn is_expired(&self) -> bool {
is_expired(self.tags())
}
}
impl<T: HasTags> EventExtensionExpiration for T {}
```
```rust {file=coracle-lib/src/prelude.rs}
pub use crate::expiration::EventExtensionExpiration;
```
## Usage patterns
**Building an expiring event.** Attach the tag at template
construction:
```rust
let event = EventContent::new()
.content("this link is good for an hour")
.tags(Tags::new().add_tag(Tag::expiration(now + 3600)))
.kind(1)
.stamp(now)
.own(pubkey);
```
**Filtering out expired events.** Relays and clients both might need to
remove expired events from a collection. Using `is_expired_at` avoids the
need to re-generate the current timestamp repeatedly:
```rust
let now = util::now();
incoming.retain(|e| !e.is_expired_at(now));
```
## What's next
Expiration is one kind of behavior tag. The next chapter introduces
another — NIP-70's `protected` tag, which tells relays not to accept
an event from anyone but its author.
+149
View File
@@ -0,0 +1,149 @@
# Protected Events
The previous chapter introduced behavior tags through expiration: a tag
that requests something of the network without touching what the event
itself means. NIP-70 adds another, aimed at a different concern. Where
expiration asks relays to stop serving an event after a time,
NIP-70's `protected` tag asks them to be careful about who they accept
it from in the first place. Its shape on the wire is a single, valueless
tag:
```json
["-"]
```
That is the whole thing — a tag whose name is a lone dash and which
carries no value. Its mere presence is the signal. An event marked this
way should be accepted by a relay only when it is published *directly by
its author*: the connection delivering it must have authenticated over
NIP-42, and the authenticated key must match the event's `pubkey`.
## What the marker asks for
A relay that honors NIP-70 runs a small decision when a protected event
arrives. If the connection has not authenticated, the relay issues a
NIP-42 `AUTH` challenge and rejects the event for now, inviting the
client to prove who it is and try again. If the connection *has*
authenticated but the authenticated key is not the event's author, the
relay rejects the event outright — someone other than the author is
trying to push it. Only an authenticated author gets through.
As with expiration, the marker is a cooperative request, not a guarantee.
A relay is free to ignore it; a relay that has already stored the event
keeps it; the signature stays valid regardless. The tag asks well-behaved
relays to gate acceptance on authorship.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod protected;
```
```rust {file=coracle-lib/src/protected.rs}
//! NIP-70 protected events: the valueless `["-"]` marker that asks
//! relays to accept an event only from its authenticated author.
//!
//! This module is the stateless half of NIP-70 — building the tag and
//! detecting it. The relay-side enforcement it implies (a NIP-42 `AUTH`
//! challenge followed by a pubkey comparison) needs a live, authenticated
//! connection and is handled in the relay layer.
use crate::events::HasTags;
use crate::tags::Tags;
```
## Detecting the marker
Because the tag has no value, detection collapses to a single question:
is a tag named `"-"` present? There is nothing to parse, so there is no
`Option` and no malformed-versus-absent distinction to draw — the answer
is a plain `bool`. `Tags::has` already answers exactly this.
```rust {file=coracle-lib/src/protected.rs}
/// Whether the tag set carries the NIP-70 `["-"]` protected marker.
///
/// A protected event asks relays to accept it only from its
/// authenticated author. This function reports the marker's presence;
/// acting on it is the relay's job.
pub fn is_protected(tags: &Tags) -> bool {
tags.has("-")
}
```
## Building the tag
The constructor produces a tag with the name `"-"` and no values. We
build it through `Tag::new` like every other tag, passing an empty value
iterator so the result is the one-element `["-"]`.
```rust {file=coracle-lib/src/tags.rs}
impl Tag {
/// Build a NIP-70 `["-"]` protected tag.
///
/// The tag has a name (`"-"`) and no values; its presence asks
/// relays to accept the event only from its authenticated author.
pub fn protected() -> Tag {
Tag::new("-", std::iter::empty::<String>())
}
}
```
## Methods on event types
A caller holding an event wants to ask whether it is protected without
reaching for its tags directly. Following the pattern from expiration, the
query is an extension trait bounded on [`HasTags`], so one default method
serves every event type. With `coracle_lib::prelude::*` in scope,
`event.is_protected()` works on both `HashedEvent` and `Event`.
```rust {file=coracle-lib/src/protected.rs}
/// Protected-event queries on any event that carries tags.
pub trait EventExtensionProtected: HasTags {
/// Whether this event carries the NIP-70 protected marker.
fn is_protected(&self) -> bool {
is_protected(self.tags())
}
}
impl<T: HasTags> EventExtensionProtected for T {}
```
```rust {file=coracle-lib/src/prelude.rs}
pub use crate::protected::EventExtensionProtected;
```
## Usage patterns
**Marking an event protected.** Attach the tag at template construction,
just like any other:
```rust
let event = EventContent::new()
.content("members only")
.tags(vec![Tag::protected()])
.kind(1)
.stamp(now)
.own(pubkey);
```
**Checking on receipt.** A relay or client deciding how to handle an
incoming event asks the event directly:
```rust
if event.is_protected() {
// only accept from an authenticated author
}
```
## What's next
Marking an event protected is half of NIP-70; honoring the mark is the
other half, and it lives where connections and authentication do. The
[Relay Authentication](26-relay-authentication.md) chapter implements the
`AUTH` exchange and the author check that turn this tag into a rule.
One consequence is worth flagging now: a protected event should not be
re-broadcast by being embedded inside someone else's repost, since that
would route it past the very acceptance check it asked for. Reposts are
their own topic; the protected marker is simply something that logic will
need to consult.
+911
View File
@@ -0,0 +1,911 @@
# Filters
A filter is a predicate over events. Given an event, a filter answers one
question: does this event match? The protocol uses filters inside `REQ`
messages to tell relays what to send, but the data structure itself is
equally useful for querying a local database, routing events to the right
handler, or deciding which events to display — anywhere you need to select
a subset of events from a larger set.
A single filter expresses a conjunction: every field that is present must
match. An array of filters expresses a disjunction: the event must match
at least one. Between the two, you can describe any positive selection
over the event fields that nostr exposes — ids, authors, kinds, tags,
and time ranges. There is no negation; the protocol rejected it early on
to keep relay implementations simple.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod filters;
```
```rust {file=coracle-lib/src/filters.rs}
//! Event filters: the [`Filter`] type for matching events by id, author,
//! kind, tags, and time range, plus utilities for hashing, grouping, and
//! estimating result cardinality.
use std::collections::{BTreeMap, BTreeSet};
use std::fmt;
use serde::de::{self, MapAccess, Visitor};
use serde::ser::SerializeMap;
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use crate::addresses::Address;
use crate::events::Event;
use crate::keys::PublicKey;
```
## The `Filter` struct
Each field corresponds to one axis of selection. The set fields — `ids`,
`authors`, `kinds` — use `Option<BTreeSet<T>>`. The `Option` layer
carries meaning: `None` says "I have no constraint on this field" and
matches everything, while `Some(empty set)` says "the value must be a
member of this empty set" and matches nothing. The distinction matters
for composition — merging two filters into one relies on being able to
tell "don't care" from "impossible."
`BTreeSet` rather than `HashSet` gives two things: O(log n) membership
checks regardless of the key type, and deterministic iteration order for
serialization and hashing.
The `tags` field is a `BTreeMap` from a tag key to a set of values. The
key is the tag name, prefixed either by `#` (any value matches, defined
in NIP-01) or `&` (all values match, defined in NIP-91).
```rust {file=coracle-lib/src/filters.rs}
/// A predicate over nostr events.
///
/// Every present field must match for the filter to match (AND semantics).
/// An array of filters matches if any single filter matches (OR semantics).
///
/// `None` on a set field means "no constraint" — it matches any value.
/// `Some(empty set)` means "must be a member of the empty set" — it
/// matches nothing. This distinction is important for filter composition.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Filter {
/// Event IDs to match.
pub ids: Option<BTreeSet<[u8; 32]>>,
/// Author public keys to match.
pub authors: Option<BTreeSet<PublicKey>>,
/// Event kinds to match.
pub kinds: Option<BTreeSet<u16>>,
/// Tag filters, keyed by prefixed tag name.
pub tags: BTreeMap<String, BTreeSet<String>>,
/// Lower bound on `created_at` (inclusive).
pub since: Option<u64>,
/// Upper bound on `created_at` (inclusive).
pub until: Option<u64>,
/// Maximum number of events a consumer should return. This is not a
/// matching criterion — [`matches`](Filter::matches) ignores it.
pub limit: Option<usize>,
/// NIP-50 full-text search query. The string is opaque at this point:
/// its structure and local relevance scoring are the subject of the next
/// chapter. `None` means no search constraint.
pub search: Option<String>,
}
```
The `limit` field is part of the NIP-01 filter object, so it belongs in
the struct. But it is a result-count constraint for consumers (relays,
storage engines), not a predicate over individual events. The `matches`
method ignores it entirely.
The `search` field comes from NIP-50. At this point it is just a string
carried alongside the other fields; the [Search](12-search.md) chapter
gives it meaning — a query model and a local relevance score.
## Matching
Matching walks each present field and returns `false` as soon as one
fails. Scalar checks — ids, kinds, authors — come first because they
are a simple set-membership test and reject most non-matching events
immediately, before the more involved tag iteration.
Tag matching checks every entry in the filter's `tags` map, and the key's
leading character determines the semantics.
```rust {file=coracle-lib/src/filters.rs}
impl Filter {
/// Test whether an event satisfies this filter.
///
/// All present fields must match (AND semantics). The `limit` field
/// is ignored — it is a hint for result-set sizing, not a predicate.
pub fn matches(&self, event: &Event) -> bool {
if let Some(ids) = &self.ids {
if !ids.contains(&event.id) {
return false;
}
}
if let Some(kinds) = &self.kinds {
if !kinds.contains(&event.kind) {
return false;
}
}
if let Some(authors) = &self.authors {
if !authors.contains(&event.pubkey) {
return false;
}
}
for (key, values) in &self.tags {
let name = &key[1..];
let has_match = match key.chars().next() {
Some('#') => event
.tags
.find_all(name)
.any(|tag| values.contains(tag.value())),
Some('&') => values.iter().all(|value| {
event
.tags
.find_all(name)
.any(|tag| tag.value() == value.as_str())
}),
_ => false,
};
if !has_match {
return false;
}
}
if let Some(since) = self.since {
if event.created_at < since {
return false;
}
}
if let Some(until) = self.until {
if event.created_at > until {
return false;
}
}
true
}
}
```
A convenience free function handles the common case of testing an event
against an array of filters — the OR-across-filters semantics that
NIP-01 defines for `REQ` subscriptions.
```rust {file=coracle-lib/src/filters.rs}
/// Test whether an event matches any filter in the slice.
///
/// Returns `true` if at least one filter matches. An empty slice matches
/// nothing.
pub fn matches_any(filters: &[Filter], event: &Event) -> bool {
filters.iter().any(|f| f.matches(event))
}
```
## Construction
An empty filter matches everything — no constraints means no rejections.
The `add_*` methods insert values into the constraint sets, consuming and
returning `self` so calls can be chained. Matching `remove_*` methods
take values back out, and `clear_*` methods reset a field to `None` —
removing the constraint entirely.
Tag filters carry a one-character wire prefix — `#` for NIP-01, `&`
for NIP-91. Rather than ask callers to spell that prefix into the key
string, where a typo (`"t"` instead of `"#t"`) would silently produce a
constraint that never matches, the tag methods take a [`TagMatch`] and a
bare name.
```rust {file=coracle-lib/src/filters.rs}
/// How the values of a tag constraint combine when matching events.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TagMatch {
/// NIP-01 (`#`): the event matches if it carries the tag with *any* of
/// the listed values.
Any,
/// NIP-91 (`&`): the event must carry the tag for *every* listed value.
All,
}
impl TagMatch {
/// Build the prefixed map key for a bare tag name. This is the only
/// place the wire prefix is produced, so a key can never be anything
/// other than `#`- or `&`-prefixed.
fn key(self, name: &str) -> String {
let prefix = match self {
TagMatch::Any => '#',
TagMatch::All => '&',
};
format!("{prefix}{name}")
}
}
impl Filter {
/// Create an empty filter that matches every event.
pub fn new() -> Self {
Filter {
ids: None,
authors: None,
kinds: None,
tags: BTreeMap::new(),
since: None,
until: None,
limit: None,
search: None,
}
}
/// Add an event id to the constraint set.
pub fn add_id(mut self, id: [u8; 32]) -> Self {
self.ids.get_or_insert_with(BTreeSet::new).insert(id);
self
}
/// Add multiple event ids to the constraint set.
pub fn add_ids(mut self, ids: impl IntoIterator<Item = [u8; 32]>) -> Self {
self.ids.get_or_insert_with(BTreeSet::new).extend(ids);
self
}
/// Remove an event id from the constraint set. If the set becomes
/// empty, it remains as `Some(empty)` — use `clear_ids` to remove
/// the constraint entirely.
pub fn remove_id(mut self, id: &[u8; 32]) -> Self {
if let Some(ids) = &mut self.ids {
ids.remove(id);
}
self
}
/// Remove the ids constraint, matching any event id.
pub fn clear_ids(mut self) -> Self {
self.ids = None;
self
}
/// Add an author to the constraint set.
pub fn add_author(mut self, author: PublicKey) -> Self {
self.authors
.get_or_insert_with(BTreeSet::new)
.insert(author);
self
}
/// Add multiple authors to the constraint set.
pub fn add_authors(mut self, authors: impl IntoIterator<Item = PublicKey>) -> Self {
self.authors
.get_or_insert_with(BTreeSet::new)
.extend(authors);
self
}
/// Remove an author from the constraint set.
pub fn remove_author(mut self, author: &PublicKey) -> Self {
if let Some(authors) = &mut self.authors {
authors.remove(author);
}
self
}
/// Remove the authors constraint, matching any author.
pub fn clear_authors(mut self) -> Self {
self.authors = None;
self
}
/// Add a kind to the constraint set.
pub fn add_kind(mut self, kind: u16) -> Self {
self.kinds.get_or_insert_with(BTreeSet::new).insert(kind);
self
}
/// Add multiple kinds to the constraint set.
pub fn add_kinds(mut self, kinds: impl IntoIterator<Item = u16>) -> Self {
self.kinds.get_or_insert_with(BTreeSet::new).extend(kinds);
self
}
/// Remove a kind from the constraint set.
pub fn remove_kind(mut self, kind: &u16) -> Self {
if let Some(kinds) = &mut self.kinds {
kinds.remove(kind);
}
self
}
/// Remove the kinds constraint, matching any kind.
pub fn clear_kinds(mut self) -> Self {
self.kinds = None;
self
}
/// Add a value to a tag filter.
pub fn add_tag(mut self, mode: TagMatch, name: &str, value: impl Into<String>) -> Self {
self.tags
.entry(mode.key(name))
.or_default()
.insert(value.into());
self
}
/// Add multiple values to a tag filter.
pub fn add_tags(
mut self,
mode: TagMatch,
name: &str,
values: impl IntoIterator<Item = impl Into<String>>,
) -> Self {
self.tags
.entry(mode.key(name))
.or_default()
.extend(values.into_iter().map(Into::into));
self
}
/// Remove a value from a tag filter.
pub fn remove_tag(mut self, mode: TagMatch, name: &str, value: &str) -> Self {
let key = mode.key(name);
if let Some(values) = self.tags.get_mut(&key) {
values.remove(value);
if values.is_empty() {
self.tags.remove(&key);
}
}
self
}
/// Remove an entire tag filter by its mode and name.
pub fn clear_tag(mut self, mode: TagMatch, name: &str) -> Self {
self.tags.remove(&mode.key(name));
self
}
/// Remove all tag filters.
pub fn clear_tags(mut self) -> Self {
self.tags.clear();
self
}
/// Set the lower bound on `created_at` (inclusive).
pub fn add_since(mut self, since: u64) -> Self {
self.since = Some(since);
self
}
/// Remove the lower bound on `created_at`.
pub fn clear_since(mut self) -> Self {
self.since = None;
self
}
/// Set the upper bound on `created_at` (inclusive).
pub fn add_until(mut self, until: u64) -> Self {
self.until = Some(until);
self
}
/// Remove the upper bound on `created_at`.
pub fn clear_until(mut self) -> Self {
self.until = None;
self
}
/// Set the result-count limit.
pub fn add_limit(mut self, limit: usize) -> Self {
self.limit = Some(limit);
self
}
/// Remove the result-count limit.
pub fn clear_limit(mut self) -> Self {
self.limit = None;
self
}
}
```
### Address convenience
Filtering for an addressable event by its address is common enough — and
error-prone enough when done by hand — to warrant a dedicated method.
An address carries a kind, an author, and an identifier; the method
translates these into the corresponding filter fields.
```rust {file=coracle-lib/src/filters.rs}
impl Filter {
/// Add constraints that match events at the given address.
///
/// Sets the kind, author, and `#d` tag filter from the address's
/// components. If the identifier is empty (plain replaceable events),
/// the `#d` tag filter is still set — the event must have a `d` tag
/// with an empty value.
pub fn add_address(self, addr: &Address) -> Self {
self.add_kind(addr.kind)
.add_author(addr.pubkey)
.add_tag(TagMatch::Any, "d", &addr.identifier)
}
}
impl Default for Filter {
fn default() -> Self {
Filter::new()
}
}
```
A filter built with `.add_address()` looks like this:
```rust
use coracle_lib::filters::Filter;
use coracle_lib::addresses::Address;
let addr: Address = "30023:ab12...cd34:my-article".parse().unwrap();
let filter = Filter::new().add_address(&addr);
// Equivalent to:
// Filter::new().add_kind(30023).add_author(pubkey).add_tag(TagMatch::Any, "d", "my-article")
```
## Serialization
The NIP-01 wire format for a filter is a flat JSON object where tag
filters appear as keys prefixed with `#` or `&`:
```json
{
"kinds": [1],
"authors": ["ab12...cd34"],
"#t": ["nostr", "rust"],
"since": 1700000000,
"limit": 10
}
```
The `tags` map already holds prefixed keys, so flattening is direct: each
key becomes a top-level field verbatim, and reconstituting it on the way
back is just as simple. Still, interleaving tag keys with the fixed
fields rules out `#[derive(Serialize, Deserialize)]` — we need a hand-
written implementation.
```rust {file=coracle-lib/src/filters.rs}
impl Serialize for Filter {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
// Count present fields for the map size hint.
let mut count = self.tags.len();
if self.ids.is_some() {
count += 1;
}
if self.authors.is_some() {
count += 1;
}
if self.kinds.is_some() {
count += 1;
}
if self.since.is_some() {
count += 1;
}
if self.until.is_some() {
count += 1;
}
if self.limit.is_some() {
count += 1;
}
if self.search.is_some() {
count += 1;
}
let mut map = serializer.serialize_map(Some(count))?;
if let Some(ids) = &self.ids {
let hex_ids: Vec<String> = ids.iter().map(hex::encode).collect();
map.serialize_entry("ids", &hex_ids)?;
}
if let Some(authors) = &self.authors {
let hex_authors: Vec<String> = authors.iter().map(|pk| pk.to_hex()).collect();
map.serialize_entry("authors", &hex_authors)?;
}
if let Some(kinds) = &self.kinds {
let kinds_vec: Vec<u16> = kinds.iter().copied().collect();
map.serialize_entry("kinds", &kinds_vec)?;
}
for (key, values) in &self.tags {
let vals: Vec<&str> = values.iter().map(String::as_str).collect();
map.serialize_entry(key, &vals)?;
}
if let Some(since) = self.since {
map.serialize_entry("since", &since)?;
}
if let Some(until) = self.until {
map.serialize_entry("until", &until)?;
}
if let Some(limit) = self.limit {
map.serialize_entry("limit", &limit)?;
}
if let Some(search) = &self.search {
map.serialize_entry("search", search)?;
}
map.end()
}
}
```
Deserialization collects known keys into their fields and routes any key
starting with `#` or `&` into the `tags` map, prefix and all. Unknown
keys are silently ignored for forward compatibility.
```rust {file=coracle-lib/src/filters.rs}
impl<'de> Deserialize<'de> for Filter {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
deserializer.deserialize_map(FilterVisitor)
}
}
struct FilterVisitor;
impl<'de> Visitor<'de> for FilterVisitor {
type Value = Filter;
fn expecting(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("a nostr filter object")
}
fn visit_map<M: MapAccess<'de>>(self, mut map: M) -> Result<Filter, M::Error> {
let mut ids: Option<BTreeSet<[u8; 32]>> = None;
let mut authors: Option<BTreeSet<PublicKey>> = None;
let mut kinds: Option<BTreeSet<u16>> = None;
let mut tags: BTreeMap<String, BTreeSet<String>> = BTreeMap::new();
let mut since: Option<u64> = None;
let mut until: Option<u64> = None;
let mut limit: Option<usize> = None;
let mut search: Option<String> = None;
while let Some(key) = map.next_key::<String>()? {
match key.as_str() {
"ids" => {
let hex_ids: Vec<String> = map.next_value()?;
let mut set = BTreeSet::new();
for h in hex_ids {
let bytes = hex::decode(&h)
.map_err(|_| de::Error::custom("invalid hex in ids"))?;
let arr: [u8; 32] = bytes
.try_into()
.map_err(|_| de::Error::custom("id must be 32 bytes"))?;
set.insert(arr);
}
ids = Some(set);
}
"authors" => {
let hex_authors: Vec<String> = map.next_value()?;
let mut set = BTreeSet::new();
for h in hex_authors {
let pk = PublicKey::from_hex(&h)
.map_err(|_| de::Error::custom("invalid pubkey in authors"))?;
set.insert(pk);
}
authors = Some(set);
}
"kinds" => {
let kind_vec: Vec<u16> = map.next_value()?;
kinds = Some(kind_vec.into_iter().collect());
}
"since" => since = Some(map.next_value()?),
"until" => until = Some(map.next_value()?),
"limit" => limit = Some(map.next_value()?),
"search" => search = Some(map.next_value()?),
other if other.starts_with('#') || other.starts_with('&') => {
let values: Vec<String> = map.next_value()?;
tags.insert(other.to_string(), values.into_iter().collect());
}
_ => {
let _: de::IgnoredAny = map.next_value()?;
}
}
}
Ok(Filter {
ids,
authors,
kinds,
tags,
since,
until,
limit,
search,
})
}
}
```
A round-trip through JSON preserves all fields:
```rust
use coracle_lib::filters::{Filter, TagMatch};
let filter = Filter::new()
.add_kind(1)
.add_tag(TagMatch::Any, "t", "nostr")
.add_tag(TagMatch::All, "t", "rust")
.add_since(1_700_000_000)
.add_limit(10);
let json = serde_json::to_string(&filter).unwrap();
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(filter, parsed);
```
## Identity and grouping
Two methods support deduplication and merging at higher layers —
subscription managers, relay pools, and storage engines that need to
detect redundant or combinable filters.
`id` produces a deterministic hash of the entire filter. Two filters
with the same id are structurally identical. Since `Filter` derives
`Hash`, we feed it through a standard `Hasher` — no need to round-trip
through JSON.
```rust {file=coracle-lib/src/filters.rs}
use std::hash::{Hash, Hasher};
use std::collections::hash_map::DefaultHasher;
impl Filter {
/// Compute a deterministic identifier for this filter.
///
/// Two filters with the same id are structurally identical. The id
/// is derived from the `Hash` implementation, which covers every
/// field including `limit`.
pub fn id(&self) -> u64 {
let mut hasher = DefaultHasher::new();
self.hash(&mut hasher);
hasher.finish()
}
}
```
`group` determines which filters can be merged by unioning their
set fields. Two filters can only be merged if they have the same
structural shape (which set fields are present, which tag names appear)
*and* the same scalar constraints (time windows). A filter with a
`limit` can never be merged — combining two limited queries into one
would change the result semantics — so each limited filter gets a
unique group key.
```rust {file=coracle-lib/src/filters.rs}
static GROUP_COUNTER: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
impl Filter {
/// Compute a group key that determines merge compatibility.
///
/// Filters in the same group can be merged by unioning their set
/// fields (`ids`, `authors`, `kinds`, tag values). The group key
/// captures:
///
/// - Which set fields are present and which tag keys appear
/// (structural shape)
/// - For `&` (AND) tag keys, the exact values too — they cannot be
/// unioned, so different ones cannot be combined
/// - The exact `since` and `until` values (different time windows
/// cannot be combined)
/// - The exact `search` query (different searches cannot be combined)
///
/// A filter with a `limit` always gets a unique group key, because
/// merging limited filters would change result-count semantics.
pub fn group(&self) -> u64 {
let mut hasher = DefaultHasher::new();
self.ids.is_some().hash(&mut hasher);
self.authors.is_some().hash(&mut hasher);
self.kinds.is_some().hash(&mut hasher);
for (key, values) in &self.tags {
key.hash(&mut hasher);
if key.starts_with('&') {
for value in values {
value.hash(&mut hasher);
}
}
}
self.since.hash(&mut hasher);
self.until.hash(&mut hasher);
self.search.hash(&mut hasher);
if self.limit.is_some() {
// Each limited filter gets a unique group — merging two
// limited queries into one would change which events are
// returned.
GROUP_COUNTER
.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
.hash(&mut hasher);
}
hasher.finish()
}
}
```
Including tag names in the group key means that a filter on `#e` tags
and a filter on `#p` tags land in different groups — as they should,
since merging them by union would change the semantics. Likewise, two
filters with different `since` or `until` values land in different
groups, because a union of their sets under one time window would either
over-fetch or under-fetch relative to what was requested. The `search`
query is treated the same way: two filters with different searches can
never be merged, so each distinct search forms its own group. AND tag
keys go a step further: because `&t: ["meme"]` and `&t: ["cat"]` can
never be unioned without demanding both at once, a `&` key folds its
values into the group identity too, so each distinct AND constraint
lands in its own group.
## Union and intersection
Two operations combine filters in different ways.
`union_filters` takes a list of filters and merges those that share
the same group — same structural shape, same time window, no limit.
Within each group it unions the set fields: ids, authors, kinds, and
tag values. The result is a shorter list that matches the same events
as the original but with fewer individual filters to evaluate.
```rust {file=coracle-lib/src/filters.rs}
/// Merge compatible filters by unioning their set fields.
///
/// Filters with the same [`group`](Filter::group) are combined into a
/// single filter whose set fields are the union of the originals. The
/// result matches the same events as the input but with fewer filters.
pub fn union_filters(filters: &[Filter]) -> Vec<Filter> {
let mut groups: BTreeMap<u64, Filter> = BTreeMap::new();
for filter in filters {
let key = filter.group();
groups
.entry(key)
.and_modify(|existing| {
merge_sets(&mut existing.ids, &filter.ids);
merge_sets(&mut existing.authors, &filter.authors);
merge_sets(&mut existing.kinds, &filter.kinds);
for (key, values) in &filter.tags {
existing
.tags
.entry(key.clone())
.or_default()
.extend(values.iter().cloned());
}
})
.or_insert_with(|| filter.clone());
}
groups.into_values().collect()
}
fn merge_sets<T: Ord + Clone>(
target: &mut Option<BTreeSet<T>>,
source: &Option<BTreeSet<T>>,
) {
match (target.as_mut(), source) {
(Some(t), Some(s)) => t.extend(s.iter().cloned()),
_ => {}
}
}
```
`intersect_filters` takes multiple groups of filters — each group
representing one independent query — and produces the set of filters
that satisfies all groups simultaneously. It does this by computing
the cartesian product across groups, combining each pair by unioning
their set fields and tightening their time windows: the latest `since`,
the earliest `until`. Finally it passes the result through
`union_filters` to collapse any redundancy.
```rust {file=coracle-lib/src/filters.rs}
/// Combine independent filter groups into filters that satisfy all of
/// them.
///
/// Each inner `Vec<Filter>` represents one group of alternatives (OR).
/// The result matches events that satisfy at least one filter from
/// *every* group (AND across groups, OR within each group).
///
/// Set fields are unioned. Time windows are tightened: the latest
/// `since` and earliest `until` win. If both filters have a `limit`,
/// the larger one is kept. Two filters carrying *different* searches
/// cannot be combined into one, so the pair is kept separate instead.
/// The result is simplified with [`union_filters`].
pub fn intersect_filters(groups: &[Vec<Filter>]) -> Vec<Filter> {
let Some(first) = groups.first() else {
return vec![];
};
let mut result: Vec<Filter> = first.clone();
for filters in &groups[1..] {
let mut combined = Vec::with_capacity(result.len() * filters.len());
for f1 in &result {
for f2 in filters {
match combine_pair(f1, f2) {
Some(f) => combined.push(f),
// Two different searches can't be combined into one
// filter; keep both so neither query is lost.
None => {
combined.push(f1.clone());
combined.push(f2.clone());
}
}
}
}
result = combined;
}
union_filters(&result)
}
fn combine_pair(a: &Filter, b: &Filter) -> Option<Filter> {
// Two different searches cannot be expressed as a single search, so
// there is no filter that satisfies both. Returning `None` tells the
// caller to keep the pair separate rather than fabricate one.
if let (Some(s1), Some(s2)) = (&a.search, &b.search) {
if s1 != s2 {
return None;
}
}
let mut f = Filter::new();
f.ids = union_option_sets(&a.ids, &b.ids);
f.authors = union_option_sets(&a.authors, &b.authors);
f.kinds = union_option_sets(&a.kinds, &b.kinds);
for (key, values) in a.tags.iter().chain(b.tags.iter()) {
f.tags
.entry(key.clone())
.or_default()
.extend(values.iter().cloned());
}
f.since = match (a.since, b.since) {
(Some(a), Some(b)) => Some(a.max(b)),
(s, None) | (None, s) => s,
};
f.until = match (a.until, b.until) {
(Some(a), Some(b)) => Some(a.min(b)),
(u, None) | (None, u) => u,
};
f.limit = match (a.limit, b.limit) {
(Some(a), Some(b)) => Some(a.max(b)),
(l, None) | (None, l) => l,
};
// At most one search is present here (equal searches collapse to one).
f.search = a.search.clone().or_else(|| b.search.clone());
Some(f)
}
fn union_option_sets<T: Ord + Clone>(
a: &Option<BTreeSet<T>>,
b: &Option<BTreeSet<T>>,
) -> Option<BTreeSet<T>> {
match (a, b) {
(Some(a), Some(b)) => {
let mut merged = a.clone();
merged.extend(b.iter().cloned());
Some(merged)
}
(Some(s), None) | (None, Some(s)) => Some(s.clone()),
(None, None) => None,
}
}
```
## What's next
The `search` field rides along through serialization, grouping, and the
set algebra here, but it has no meaning yet — `matches` doesn't look at
it, and the string is uninterpreted. The next chapter takes up NIP-50
full-text search: a typed query model that separates terms from
extensions, a local relevance score, and relevance-ordered results.
+214
View File
@@ -0,0 +1,214 @@
# Search
NIP-50 adds one field to the filter from the previous chapter: a `search`
string. A relay that advertises the capability reads the string as a
human-readable query — `best nostr apps` — matches it against event content,
and returns results ordered by relevance rather than by `created_at`, with
`limit` applied after ranking.
Search is opt-in and implementation-defined. Relays decide whether they index events
at all, what matches, and how ranking works. The query may also carry
`key:value` extensions — `domain:`, `language:`, `sentiment:`, `nsfw:`,
`include:spam` — and a relay honors only the ones it understands, ignoring the
rest. There is no global index and no guarantee of completeness: a client
queries the relays it believes support search and accepts a partial view.
Search may be implemented relay-side, or it may be performed on a client in some
situations. This chapter provides utilities for parsing search terms along with
a very basic model for implementing search that is decoupled from filter matching
itself and entirely opt-in.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod search;
```
```rust {file=coracle-lib/src/search.rs}
//! NIP-50 full-text search queries.
//!
//! A [`SearchQuery`] holds the terms of a search string and computes a
//! best-effort relevance score against event content — for the case where
//! search runs on the client, over events already in hand, rather than on a
//! relay.
use std::fmt;
```
## The query model
A `SearchQuery` is just the query's terms: the words split out of the search
string. NIP-50 also defines `key:value` extensions, but their meaning is
relay-defined, and the local scorer has no way to evaluate `sentiment:negative`
or `domain:example.com` without data it doesn't have. Rather than model
extensions we can't honor, we treat every token as a term. A relay that
understands an extension still sees it verbatim in the query string; the local
scorer simply matches it as text like any other word.
```rust {file=coracle-lib/src/search.rs}
/// A parsed NIP-50 search query: the terms of the query string.
///
/// NIP-50 `key:value` extensions are not modeled separately — their semantics
/// are relay-defined and cannot be evaluated locally, so each is kept as an
/// ordinary term.
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct SearchQuery {
/// The query's terms, in order.
pub terms: Vec<String>,
}
```
### Parsing
Parsing splits the query on whitespace. Every token becomes a term, including
anything that looks like an extension. There is nothing to reject, so parsing is
total — it never errors.
```rust {file=coracle-lib/src/search.rs}
impl SearchQuery {
/// Create an empty query.
pub fn new() -> Self {
SearchQuery::default()
}
/// Parse a raw query string by splitting it on whitespace. Every token,
/// extension-like or not, becomes a term. Parsing never fails.
pub fn parse(input: &str) -> Self {
SearchQuery {
terms: input.split_whitespace().map(str::to_string).collect(),
}
}
/// True when the query has no terms.
pub fn is_empty(&self) -> bool {
self.terms.is_empty()
}
}
```
Rendering joins the terms back into a query string. It is the inverse of
parsing: feeding the output of one into the other gives an equal query, modulo
runs of whitespace collapsing to single spaces.
```rust {file=coracle-lib/src/search.rs}
impl fmt::Display for SearchQuery {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.terms.join(" "))
}
}
```
## Scoring
NIP-50 returns results in descending order of relevance, so a boolean "matches
or not" is the wrong shape for a local implementation. The scorer instead
returns a number in `0.0..=1.0`, which can drive both inclusion (anything above
zero is a hit) and ordering.
The score has two parts. The base is the fraction of the query's terms that
appear in the content, compared case-insensitively — three terms, two present,
gives `2/3`. On top of that, repeated occurrences add a small, diminishing
bonus, so that among events matching the same set of terms the ones that mention
them more often rank higher. The bonus is bounded below `1/total`, which means
it can reorder events *within* a fraction but can never push a partial match up
to a full one: a missing term always costs more than any number of repetitions
can recover. An empty query — no terms — scores `1.0`, since there is no text to
constrain.
```rust {file=coracle-lib/src/search.rs}
impl SearchQuery {
/// Score `content` against this query's terms, in `0.0..=1.0`.
///
/// The base score is the fraction of the query's terms found in the content
/// (case-insensitive substring). Repeated occurrences add a diminishing
/// bonus, strictly less than one term's worth, so a partial match never
/// reaches `1.0`. An empty query scores `1.0`: there is no text to match.
pub fn score(&self, content: &str) -> f64 {
let total = self.terms.len();
if total == 0 {
return 1.0;
}
let haystack = content.to_lowercase();
let mut matched = 0usize;
let mut extra = 0usize;
for term in &self.terms {
let needle = term.to_lowercase();
if needle.is_empty() {
// An empty term imposes no constraint; treat it as present.
matched += 1;
continue;
}
let count = haystack.matches(needle.as_str()).count();
if count > 0 {
matched += 1;
extra += count - 1;
}
}
let base = matched as f64 / total as f64;
let bonus = (1.0 - 1.0 / (1.0 + extra as f64)) / total as f64;
(base + bonus).min(1.0)
}
}
```
Lowercasing uses `to_lowercase`, which folds case across Unicode rather than
only ASCII. That allocates, but nostr content is multilingual, and correctness
on non-Latin text is worth more than avoiding a copy in a best-effort matcher.
## Connecting queries to filters
The previous chapter gave `Filter` a `search` field but no way to set it. The
setters follow the established `add_*` / `clear_*` vocabulary.
```rust {file=coracle-lib/src/filters.rs}
use crate::search::SearchQuery;
impl Filter {
/// Set the NIP-50 search query.
pub fn add_search(mut self, search: impl Into<String>) -> Self {
self.search = Some(search.into());
self
}
/// Remove the search query, leaving no search constraint.
pub fn clear_search(mut self) -> Self {
self.search = None;
self
}
}
```
Scoring an event against a filter is then a matter of parsing the field and
delegating to `SearchQuery::score`. With no search set the method returns `1.0`,
so an unsearched filter never penalizes an event. This is purely the search
dimension — it is independent of the structural `matches` check from the
previous chapter, and the two are meant to be composed by the caller, not folded
together. A consumer that wants search-ranked results filters with `matches`,
scores with `search_score`, and sorts as it sees fit.
```rust {file=coracle-lib/src/filters.rs}
impl Filter {
/// Best-effort local relevance score for `event`, in `0.0..=1.0`.
///
/// Parses the `search` field and scores it against the event's content,
/// returning `1.0` when there is no search. This considers *only* the
/// `search` field; it is independent of [`matches`](Filter::matches).
pub fn search_score(&self, event: &Event) -> f64 {
match &self.search {
Some(query) => SearchQuery::parse(query).score(&event.content),
None => 1.0,
}
}
}
```
## What's next
Search depends on routing the query to a relay that actually supports it.
Discovering which relays advertise NIP-50, and choosing among them, is a
networking and relay-metadata concern — the subject of the Domain and Networking
sections, where relay selection is built on top of the filter types assembled
here.
+43 -41
View File
@@ -6,60 +6,62 @@
- [Keys](02-keys.md) - [Keys](02-keys.md)
- [Encryption](03-encryption.md) - [Encryption](03-encryption.md)
- [Events](04-events.md) - [Tags](04-tags.md)
- [Signing](05-signing.md) - [Events](05-events.md)
- [Tags](06-tags.md) - [Kinds](06-kinds.md)
- [Kinds](07-kinds.md) - [Addresses](07-addresses.md)
- [Kind Ranges](08-kind-ranges.md) - [Proof of Work](08-proof-of-work.md)
- [Addresses](09-addresses.md) - [Expiring Events](09-expiring-events.md)
- [Proof of Work](10-proof-of-work.md) - [Protected Events](10-protected-events.md)
- [Filters](11-filters.md) - [Filters](11-filters.md)
- [Search](12-search.md)
## Domain ## Domain
- [Relay Selections](12-relay-selections.md) - [Relay Selections](13-relay-selections.md)
- [Relay Metadata](13-relay-metadata.md) - [Relay Metadata](14-relay-metadata.md)
- [Relay Membership](14-relay-membership.md) - [Relay Membership](15-relay-membership.md)
- [Profiles](15-profiles.md) - [Profiles](16-profiles.md)
- [Follows](16-follows.md) - [Follows](17-follows.md)
- [Microblogging](17-microblogging.md) - [Microblogging](18-microblogging.md)
- [Reactions](18-reactions.md) - [Reactions](19-reactions.md)
- [Reports](19-reports.md) - [Reports](20-reports.md)
- [Emojis](20-emojis.md) - [Emojis](21-emojis.md)
- [Zaps](21-zaps.md) - [Zaps](22-zaps.md)
- [Rooms](22-rooms.md) - [Rooms](23-rooms.md)
- [Open Timestamp Attestations](24-open-timestamp-attestations.md)
## Networking ## Networking
- [Relay Connections](23-relay-connections.md) - [Relay Connections](25-relay-connections.md)
- [Relay Authentication](24-relay-authentication.md) - [Relay Authentication](26-relay-authentication.md)
- [Relay Policies](25-relay-policies.md) - [Relay Policies](27-relay-policies.md)
- [Server Authentication](26-server-authentication.md) - [Server Authentication](28-server-authentication.md)
- [Relay Management API](27-relay-management-api.md) - [Relay Management API](29-relay-management-api.md)
- [Blossom Media Storage](28-blossom-media-storage.md) - [Blossom Media Storage](30-blossom-media-storage.md)
## Signers ## Signers
- [Signer Interface](29-signer-interface.md) - [Signer Interface](31-signer-interface.md)
- [Secret Signers](30-secret-signers.md) - [Secret Signers](32-secret-signers.md)
- [Remote Signers](31-remote-signers.md) - [Remote Signers](33-remote-signers.md)
- [Android Signers](32-android-signers.md) - [Android Signers](34-android-signers.md)
- [Browser Signers](33-browser-signers.md) - [Browser Signers](35-browser-signers.md)
## Content ## Content
- [Entities](34-entities.md) - [Entities](36-entities.md)
- [Relays](35-relays.md) - [Relays](37-relays.md)
- [Rooms](36-rooms.md) - [Rooms](38-rooms.md)
- [Links](37-links.md) - [Links](39-links.md)
- [Lightning](38-lightning.md) - [Lightning](40-lightning.md)
- [Cashu](39-cashu.md) - [Cashu](41-cashu.md)
- [Emojis](40-emojis.md) - [Emojis](42-emojis.md)
- [Topics](41-topics.md) - [Topics](43-topics.md)
- [Code](42-code.md) - [Code](44-code.md)
## Storage ## Storage
- [Event Repository](43-event-repository.md) - [Event Repository](45-event-repository.md)
- [In Memory Backend](44-in-memory-backend.md) - [In Memory Backend](46-in-memory-backend.md)
- [Sqlite Backend](45-sqlite-backend.md) - [Sqlite Backend](47-sqlite-backend.md)
+62
View File
@@ -0,0 +1,62 @@
# Plan: Addresses
## Overview
This chapter introduces the `Address` struct — a stable reference to a replaceable or
addressable event. Where an event ID is a hash of a specific version, an address is the
triple `(kind, pubkey, identifier)` that names the *slot* a mutable event occupies.
## Prerequisites
Reader has already covered: keys (PublicKey, bech32), tags (Tags, d-tag access via
`tags.value("d")`), events (Event struct), kinds (kind ranges, is_addressable,
is_replaceable).
## Structure
### 1. Motivation (prose only)
- Event IDs are hashes of content — they change when the content changes
- For replaceable/addressable events, you need a stable reference
- An address is `kind:pubkey:identifier` — it names the slot, not the version
### 2. The module
- Register `pub mod addresses;` in lib.rs
- Import dependencies: keys::PublicKey, tags::Tag, events::Event, kinds, bech32, std::fmt, std::str::FromStr
### 3. Error type
- `AddressError` enum: InvalidFormat, InvalidBech32, WrongPrefix, InvalidKey, FieldMissing(&'static str), TlvError
### 4. The `Address` struct
- Fields: `kind: u16`, `pubkey: PublicKey`, `identifier: String`
- Constructor: `Address::new(kind, pubkey, identifier)`
- `from_event(event) -> Option<Self>` — returns None for non-replaceable/non-addressable events. Uses `tags.value("d").unwrap_or("").to_string()` for identifier.
- `to_tag(relay_hint: Option<&str>) -> Tag` — produces an `a` tag
### 5. Display / FromStr (kind:pubkey:identifier format)
- `Display`: `write!(f, "{}:{}:{}", self.kind, self.pubkey, self.identifier)`
- `FromStr`: split on `:`, take first two segments, join remainder as identifier (handles colons in d-tags)
### 6. NIP-19 naddr encoding
- Introduce TLV format: type(1 byte) + length(1 byte) + value(N bytes)
- Constants: SPECIAL=0, RELAY=1, AUTHOR=2, KIND=3
- `to_naddr() -> String` — encode as bech32 with "naddr" prefix
- `from_naddr(s: &str) -> Result<(Self, Vec<String>), AddressError>` — returns address + relay hints
- TLV decode: while loop consuming bytes
- SPECIAL=identifier (UTF-8), RELAY=URL string, AUTHOR=32-byte pubkey, KIND=u32 big-endian
### 7. What's next
- Brief pointer to proof of work or filters chapter
## Code placement
- All code in `coracle-lib/src/addresses.rs`
- Module registered in `coracle-lib/src/lib.rs`
## Tests needed
- `Address::new` basic construction
- `from_event` for addressable event (returns Some)
- `from_event` for regular event (returns None)
- `from_event` for replaceable event (returns Some, empty identifier)
- Display round-trip through FromStr
- to_naddr / from_naddr round-trip
- FromStr with identifier containing colons
- to_tag produces correct `a` tag format
+234
View File
@@ -0,0 +1,234 @@
# Plan: Events
## Topic Summary
Introduce the core nostr `Event` type: the seven wire fields, JSON
(de)serialization, the NIP-01 canonical form used for ID computation, the
unsigned-vs-signed lifecycle distinction, and minimal structural validation. Add
a signing primitive on `SecretKey` that signs an arbitrary byte message (not an
event), and show — in prose, not in tangled code — how to combine `UnsignedEvent::id()`
and `SecretKey::sign()` to produce a complete signed `Event`. A richer
event-signing API and the `Signer` trait are deferred to chapter 05.
## Chapter Outline
1. **Opening framing.** Events as the entire nostr substrate: signed,
content-addressable, permissionless facts. Referential transparency. Why a
regular event's id is `sha256` of its canonical form. Connects back to
chapter 02 ("your public key is your name") and forward ("everything in this
book is built out of events").
2. **The module.** Register `events` in `lib.rs`. Imports.
3. **Errors.** `EventError` enum covering JSON failures, id mismatch, signature
failure, and structural problems.
4. **The wire structure.** Walk through the seven fields with a minimal JSON
example. Show the raw shape the network agrees on.
5. **The `Event` struct.** Define a plain struct with public fields, derive
`Clone`, `Debug`, `PartialEq`, `Eq`. Field types:
- `id: [u8; 32]`
- `pubkey: PublicKey` (from chapter 02)
- `created_at: u64`
- `kind: u16`
- `tags: Vec<Vec<String>>`
- `content: String`
- `sig: [u8; 64]`
Add `serde` impls (custom for hex-encoded binary fields, derived otherwise).
6. **Canonical form and the id.** Show `[0, pubkey, created_at, kind, tags, content]`.
Produce it with `serde_json::json!`. Hash with `sha2::Sha256`. Return 32 bytes.
Introduce `UnsignedEvent` struct (same fields minus `id` and `sig`) and put
`fn id(&self) -> [u8; 32]` on it.
7. **Signing primitive on `SecretKey`.** Add `SecretKey::sign(&self, msg: &[u8]) -> [u8; 64]`.
This belongs in `keys.rs` but is introduced *here* because this is where we
first need it. It wraps `secp256k1`'s Schnorr signing over a 32-byte digest,
using the shared `SECP256K1` context. Show a worked example — in prose, NOT
tangled — of `secret.sign(&unsigned.id())` producing a `[u8; 64]` that can be
combined with the `UnsignedEvent` fields to form an `Event`. This keeps
orchestration out of the core so chapter 05 can build a proper `Signer`
abstraction on top.
8. **Verification.** Two methods on `Event`:
- `fn verify_id(&self) -> bool` — recomputes the id from the other fields and
compares.
- `fn verify(&self) -> Result<(), EventError>` — verifies id and Schnorr
signature. Uses `secp256k1::Message::from_digest` + `verify_schnorr` with
the stored pubkey.
Note: no `PublicKey::verify` method yet — the verification goes directly
through `secp256k1` inside `Event::verify`. A future signer chapter can pull
that out.
9. **JSON round-tripping.** `Event` implements `Serialize`/`Deserialize`.
Because we store id/sig as bytes, we implement serde by hand via an
intermediate struct with hex-string fields. Highlight: incoming JSON is
validated structurally by serde but the id/sig are NOT cryptographically
validated on deserialization — that's what `verify()` is for.
10. **A worked example.** Build an unsigned text-note event, compute its id,
sign it, assemble the `Event`, serialize to JSON, parse it back, verify.
This block is illustrative — marked without a `{file=...}` annotation so
it is NOT tangled. It exists only in the narrative to show how the pieces
fit together.
11. **What's next.** Pointer forward to chapter 05 (Signing), which will wrap
`SecretKey::sign` in a proper `Signer` abstraction that knows how to sign
events directly.
## API Design
New in `coracle-lib/src/events.rs`:
```rust
pub enum EventError {
Json(String),
IdMismatch,
InvalidSignature,
InvalidHexField(&'static str),
}
pub struct Event {
pub id: [u8; 32],
pub pubkey: PublicKey,
pub created_at: u64,
pub kind: u16,
pub tags: Vec<Vec<String>>,
pub content: String,
pub sig: [u8; 64],
}
impl Event {
pub fn verify_id(&self) -> bool;
pub fn verify(&self) -> Result<(), EventError>;
}
impl Serialize for Event { ... }
impl<'de> Deserialize<'de> for Event { ... }
pub struct UnsignedEvent {
pub pubkey: PublicKey,
pub created_at: u64,
pub kind: u16,
pub tags: Vec<Vec<String>>,
pub content: String,
}
impl UnsignedEvent {
pub fn id(&self) -> [u8; 32];
}
```
New in `coracle-lib/src/keys.rs` (addition):
```rust
impl SecretKey {
pub fn sign(&self, message: &[u8; 32]) -> [u8; 64];
}
```
Taking `&[u8; 32]` rather than `&[u8]` encodes Schnorr-over-a-digest at the
type level and matches `secp256k1::Message::from_digest`. The chapter notes that
this is "sign an arbitrary 32-byte message," not "sign an event."
## Code Organization
- `coracle-lib/src/events.rs` — new file, all event-related types.
- `coracle-lib/src/lib.rs` — register `pub mod events;`.
- `coracle-lib/src/keys.rs` — append a small block adding `SecretKey::sign`.
- `coracle-lib/tests/events.rs` — hand-written integration tests (not tangled):
round-trip a known event, verify a pre-signed fixture, reject tampered id/sig.
## Dependencies
All already in `coracle-lib/Cargo.toml`:
- `serde`, `serde_json` — JSON and canonical form
- `sha2` — event id hash
- `secp256k1` — Schnorr signing and verification
- `hex` — id/sig hex encoding in serde impls
No new dependencies.
## Narrative Notes
- **Open with referential transparency.** The single most important concept in
this chapter is that the event id is a function of the content. Lead with
that, not with field enumeration.
- **Canonical form deserves its own subsection.** Readers coming from
conventional APIs will expect "just JSON"; it's worth stopping to explain why
nostr uses a positional array, not an object, to compute the hash.
- **Two separate structs, not `Option<id>`.** Explain the choice: making the
lifecycle visible in types is worth the slightly larger API surface in a
teaching library. Reference rust-nostr's similar approach.
- **Why `sign` is on `SecretKey`, not `Event`.** Keep signing as a low-level
primitive over bytes. Chapter 05 will introduce a `Signer` trait that
abstracts over local/remote/browser signers; only that trait will have an
`sign_event` method. Keeping `Event` signer-agnostic now avoids having to
retrofit it later.
- **Show the worked example early enough that readers can see the pieces click.**
Put it right after verification so they see create → hash → sign → verify in
one flow.
- **Structural validation is shallow.** Point out explicitly that `verify()`
does the real work and structural validation is about rejecting garbage, not
about enforcing semantics. Tie back to the "validation philosophy" from
chapter 01's framing and from the building-nostr philosophy.
- **Custom serde for binary fields.** Walk through why we hand-write
`Serialize`/`Deserialize`: the wire format expects id/sig as lowercase hex
strings, but we want to hold them as `[u8; 32]`/`[u8; 64]` internally. An
intermediate struct with hex `String` fields + `From` conversions is the
cleanest approach.
## Design Decisions
1. **Plain struct with public fields.** Matches rust-nostr. No getters, no
setters, no builder — the teaching goal is clarity about what an event *is*.
A builder can come later if it earns its place.
2. **Separate `UnsignedEvent`.** Option 1 from the research. Clearer lifecycle
in types; avoids `Option` juggling.
3. **`[u8; 32]` / `[u8; 64]` for id/sig, not newtypes.** Keep the type surface
small for this chapter. A later chapter can promote them to `EventId` etc. if
it's worth it. The serde impl hides the raw array at the JSON boundary.
4. **Custom serde impls via an intermediate hex-string struct.** Mirrors
rust-nostr's `EventIntermediate` approach but simpler: one owned struct for
deserialization, hand-written `Serialize` for output. Avoids pulling in
`serde_with` or writing `serialize_with` attributes.
5. **`SecretKey::sign` takes `&[u8; 32]`, not `&[u8]`.** Schnorr signs a 32-byte
digest; the fixed-size array makes that contract visible and avoids runtime
errors. Matches `secp256k1::Message::from_digest`.
6. **No `PublicKey::verify` yet.** One primitive at a time. Verification lives
inside `Event::verify` for now, which hides the secp256k1 details.
Chapter 05 can pull it out when the `Signer` trait emerges.
7. **`u64` for `created_at`.** Unix seconds fit comfortably. A signed int would
suggest pre-epoch events are meaningful; they aren't in nostr.
8. **`u16` for `kind`.** nostr kinds fit in 16 bits and the canonical form
serializes them as bare integers. No enum yet — that's chapter 07.
9. **Tags as `Vec<Vec<String>>`.** Matches every reference. Richer tag types
wait for chapter 06.
10. **Worked example is NOT tangled.** Per user's explicit instruction: show
event signing as a prose example using the `sign` primitive, but do not
generate a `sign_event` or similar helper in the library.
## Open Questions
- **Should `Event::verify` validate structural invariants (pubkey length, hex
shape of tags, etc.) in addition to id and signature?** Leaning no — those are
caught at deserialization by serde. `verify()` should be *cryptographic*
verification only. Will decide in writing.
- **Error granularity for JSON failures.** Keep a single `Json(String)` variant,
or split into `InvalidJson` / `MissingField` / `WrongType`? Leaning on a
single variant to match the style of the keys chapter's `KeyError`.
- **Should `UnsignedEvent` have a convenience constructor for kind-1 text notes
with `created_at = now()`?** Leaning no — this chapter should show the raw
fields; builders and helpers can come later without breaking anything.
+137
View File
@@ -0,0 +1,137 @@
# Plan: Expiring Events
## Topic Summary
NIP-40 defines an optional `expiration` tag:
`["expiration", "<unix_timestamp_seconds>"]`. It marks an event as no
longer valid after the given time. Relays SHOULD stop serving expired
events and MAY drop them; clients SHOULD hide them. Expiration is a
cooperative hint, not a cryptographic guarantee.
This chapter adds tag-layer support: a parser, a predicate, a
constructor, and the usual method facades on `HashedEvent` and `Event`.
Enforcement (storage sweeps, UI hiding) is out of scope.
## Chapter Outline
1. **Framing** — expiration as a behavior tag; why it lives on `Tags`
rather than on kind; cooperative nature of the feature.
2. **The module declaration**`pub mod expiration;` in `lib.rs` plus
the module header.
3. **Reading the tag**`get_expiration(&Tags) -> Option<u64>`.
Explain missing-vs-malformed both collapsing to `None`.
4. **Testable predicate**`is_expired_at(&Tags, now: u64) -> bool`.
Explain the strict-`<` comparison.
5. **System-clock predicate**`is_expired(&Tags) -> bool`, plus the
private `now_seconds()` helper.
6. **Constructor**`Tag::expiration(ts: u64) -> Tag`. Explain why we
bother: a chapter's worth of call sites read better with a named
constructor than with `Tag::new("expiration", [ts.to_string()])`.
7. **Methods on event types**`get_expiration`, `is_expired`,
`is_expired_at` on both `HashedEvent` and `Event`.
8. **Usage patterns** — the expected call sites (a relay filter, a UI
check, a builder attaching a tag), very briefly.
9. **What's next** — bridge to protected events (chapter 10).
## API Design
### Module `coracle-lib/src/expiration.rs`
```rust
pub fn get_expiration(tags: &Tags) -> Option<u64>;
pub fn is_expired_at(tags: &Tags, now: u64) -> bool;
pub fn is_expired(tags: &Tags) -> bool;
impl HashedEvent {
pub fn get_expiration(&self) -> Option<u64>;
pub fn is_expired_at(&self, now: u64) -> bool;
pub fn is_expired(&self) -> bool;
}
impl Event {
pub fn get_expiration(&self) -> Option<u64>;
pub fn is_expired_at(&self, now: u64) -> bool;
pub fn is_expired(&self) -> bool;
}
```
### Tag constructor in `coracle-lib/src/tags.rs`
```rust
impl Tag {
pub fn expiration(timestamp: u64) -> Tag;
}
```
Rationale for placement: the constructor lives with `Tag` (not in
`expiration.rs`) because it belongs to the tag-construction surface,
which callers already import when they build tags. This matches how the
nonce tag is handled — built inline at its mining site — but gives
`expiration` a nicer call site because building this tag is common.
## Code Organization
- **`coracle-lib/src/expiration.rs`** — new module for the free
functions and the method impls.
- **`coracle-lib/src/lib.rs`** — add `pub mod expiration;`.
- **`coracle-lib/src/tags.rs`** — add the `Tag::expiration` constructor
inside an existing `impl Tag` block, tangled from the expiration
chapter (code blocks targeting the same file concatenate; the new
block adds another `impl Tag` block appended at the end, which is
valid Rust).
- **`coracle-lib/tests/expiration.rs`** — integration test (hand-
written, not tangled).
## Dependencies
None. `std::time::{SystemTime, UNIX_EPOCH}` is already in std. No new
crate dependencies, no new workspace entries.
## Narrative Notes
- Lead with the behavior-tag framing. A reader who has just finished
proof of work will recognize the pattern: a tag that *requests*
downstream cooperation. Expiration is the cleaner example.
- Make the "hint, not guarantee" point explicitly. The chapter is not
asserting that expired events are gone — it's giving callers the
primitive they need to act on the hint.
- Strict-`<` deserves a sentence. An event whose expiration equals
`now` is still valid; it becomes expired the moment `now` advances.
- The `get_expiration` / `is_expired` split mirrors the pattern from
`get_pow` in the previous chapter: a free function that takes tags,
plus method sugar on the event types. Call this out briefly — the
consistency is the point.
- Don't over-design `is_expired`. The `SystemTime::now()` fallback to
`0` on a clock error means an unreachable case maps to "never
expired," which is the safer default than "always expired".
- Keep the constructor section short. It's one line of code; the
narrative justifies adding it at all.
## Design Decisions
- **No `Timestamp` newtype.** `created_at` is `u64` everywhere; this
stays consistent.
- **No clock injection.** The testable `is_expired_at(now)` is enough;
a `Clock` trait would be ceremony in a library this size.
- **Strict `<` not `<=`.** Matches rust-nostr, welshman, applesauce.
"Expires at T" reads as "invalid after T".
- **Free functions take `&Tags` only.** The predicate doesn't need the
event id or pubkey. Keeping it at the `Tags` level makes it reusable
for any caller holding a tag slice (e.g., a relay streaming raw JSON
can check without reconstructing an `Event`).
- **No background sweeper, no storage integration.** That belongs in
a later storage chapter, if at all. This chapter is the parse layer.
- **Tag constructor in `tags.rs`.** Discoverable next to
`Tag::new`; avoids an import from `expiration` just to build a tag.
- **No `set_expiration(ts)` helper on event builders.** Callers can
write `.tags(tags.add_tag(Tag::expiration(ts)))` or the equivalent;
the builder surface is already chainable enough.
## Open Questions
- Whether to document the interaction with NIP-09 (deletion requests)
in this chapter. Keep it out — deletion is its own chapter's problem,
and entangling them here dilutes the focus.
- Whether to add a convenience `is_expired_now` on `Tags` directly.
Skip it; the free function is already named clearly, and adding
methods on `Tags` blurs the "tags are a generic container" line.
+207
View File
@@ -0,0 +1,207 @@
# Plan: Filters
## Topic Summary
Filters are the NIP-01 data structure for matching events. They form an elegant primitive for
matching events independent of the client/relay context — not just for REQ messages, but as a
general-purpose event matching and querying abstraction. The chapter covers the filter structure,
matching semantics (AND within a filter, OR across filters), tag filters, timestamp constraints,
limits, construction, hashing, grouping, and cardinality estimation.
## Chapter Outline
1. **Introduction** — Filters as a general-purpose event matching primitive. Not tied to relays;
they're a predicate you can evaluate against any event. Analogy to database WHERE clauses.
2. **The Filter Struct** — Walk through the fields:
- `ids: Option<BTreeSet<[u8; 32]>>` — match event IDs
- `authors: Option<BTreeSet<PublicKey>>` — match event authors
- `kinds: Option<BTreeSet<u16>>` — match event kinds
- `tags: BTreeMap<String, BTreeSet<String>>` — match tag values by tag name
- `since: Option<u64>` — lower bound on `created_at` (inclusive)
- `until: Option<u64>` — upper bound on `created_at` (inclusive)
- `limit: Option<usize>` — result count constraint (not a matching criterion)
Explain `Option` semantics: `None` = no constraint, `Some(empty set)` = matches nothing.
Note that `limit` is metadata for consumers, not part of matching logic.
3. **Matching** — Implement `matches(&self, event: &Event) -> bool`:
- AND semantics: all present fields must match
- Early exit on scalar checks (ids, kinds, authors) before tag matching
- Tag matching: for each tag filter, event must have at least one tag with that name
whose value is in the filter's set (OR within a tag filter, AND across tag filters)
- Timestamp: `since <= created_at <= until`
- `limit` is ignored
- Implement `matches_any(filters: &[Filter], event: &Event) -> bool` as a free function
for OR-across-filters semantics
4. **Construction** — Builder pattern with fluent API:
- `Filter::new()` — empty filter (matches everything)
- `.id(id)` / `.ids(iter)` — add event IDs
- `.author(pk)` / `.authors(iter)` — add authors
- `.kind(k)` / `.kinds(iter)` — add kinds
- `.tag(name, value)` / `.tags(name, iter)` — add arbitrary tag filters
- `.since(ts)` / `.until(ts)` — set timestamp bounds
- `.limit(n)` — set result limit
- `.address(addr)` — convenience: sets kind, author, and `#d` tag from an Address
5. **Serialization** — Custom serde implementation:
- Standard fields serialize normally, skip `None` fields
- `tags` BTreeMap flattened: key `"foo"` becomes JSON key `"#foo"` with array value
- Handle `limit: 0` vs omitted limit (Some(0) serializes as `"limit": 0`)
- Deserialization: any key starting with `#` collected into `tags` map
- Show round-trip example
6. **Identity and Grouping** — Utilities for deduplication and merging:
- `filter_id(filter) -> String` — deterministic hash of filter contents for dedup
- `filter_group(filter) -> String` — hash of structural fields only (ids, kinds, authors,
tag keys) excluding values and temporal fields. Two filters in the same group can be
merged by unioning their value sets.
7. **Cardinality**`cardinality(&self) -> Option<usize>`:
- Returns `Some(n)` when the maximum number of matching events can be determined
- `ids` present → `ids.len()`
- All kinds are replaceable + `authors` present → `authors.len() * kinds.len()`
- All kinds are addressable + `authors` present + `#d` present →
`authors.len() * kinds.len() * d_values.len()`
- Otherwise → `None` (unbounded)
- If explicit `limit` is set, return `min(limit, computed)` when computed is Some,
or `Some(limit)` when computed is None
- Empty set in any field → `Some(0)`
8. **Recap** — Summarize filter as a composable primitive. Tease usage in relay connections
chapter.
## API Design
```rust
// --- Filter struct ---
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Filter {
pub ids: Option<BTreeSet<[u8; 32]>>,
pub authors: Option<BTreeSet<PublicKey>>,
pub kinds: Option<BTreeSet<u16>>,
pub since: Option<u64>,
pub until: Option<u64>,
pub limit: Option<usize>,
// Flattened in serde as #key -> [values]
pub tags: BTreeMap<String, BTreeSet<String>>,
}
// --- Construction (builder, consuming self) ---
impl Filter {
pub fn new() -> Self
pub fn id(self, id: [u8; 32]) -> Self
pub fn ids(self, ids: impl IntoIterator<Item = [u8; 32]>) -> Self
pub fn author(self, author: PublicKey) -> Self
pub fn authors(self, authors: impl IntoIterator<Item = PublicKey>) -> Self
pub fn kind(self, kind: u16) -> Self
pub fn kinds(self, kinds: impl IntoIterator<Item = u16>) -> Self
pub fn tag(self, name: impl Into<String>, value: impl Into<String>) -> Self
pub fn tags(self, name: impl Into<String>, values: impl IntoIterator<Item = impl Into<String>>) -> Self
pub fn since(self, since: u64) -> Self
pub fn until(self, until: u64) -> Self
pub fn limit(self, limit: usize) -> Self
pub fn address(self, addr: &Address) -> Self
}
// --- Matching ---
impl Filter {
pub fn matches(&self, event: &Event) -> bool
pub fn cardinality(&self) -> Option<usize>
}
pub fn matches_any(filters: &[Filter], event: &Event) -> bool
// --- Identity and grouping ---
pub fn filter_id(filter: &Filter) -> String
pub fn filter_group(filter: &Filter) -> String
```
## Code Organization
All code in `coracle-lib/src/filters.rs`. Single file, single module. Add `pub mod filters;`
to `coracle-lib/src/lib.rs`.
## Dependencies
- `serde` / `serde_json` — already used in the events chapter for serialization
- `std::collections::BTreeSet` / `BTreeMap` — stdlib, no external crate
- `sha2` — already used in events chapter for hashing; reuse for filter_id
No new external dependencies needed.
## Narrative Notes
- Open by framing filters as a standalone primitive. They're a predicate, not a protocol
message. The fact that relays use them in REQ is one application, but they're equally
useful for client-side filtering, local storage queries, and event routing decisions.
- The `Option` semantics deserve careful explanation. Show the difference:
`None` = "I don't care about this field" vs `Some(empty)` = "this field must match
one of these zero values (i.e., nothing matches)". This is the key insight that makes
filters composable.
- When explaining matching, walk through a concrete example: construct a filter, show an
event, trace through the matching logic field by field.
- For tag filters, emphasize that tag keys are arbitrary strings — not restricted to
single letters. The single-letter convention is a relay indexing optimization, not a
protocol constraint.
- `limit` gets a brief note: it's not part of matching. It tells a consumer (relay, storage
engine) how many results to return. Include it in the struct because it's part of the
NIP-01 filter object, but `matches()` ignores it.
- For serialization, the interesting part is the tag flattening. Show the JSON representation
and explain how `tags: {"e": {"abc"}, "p": {"def"}}` becomes `{"#e": ["abc"], "#p": ["def"]}`.
- `filter_id` and `filter_group` are utility functions, not methods, because they serve
infrastructure concerns (dedup, subscription management) rather than core filter semantics.
- `cardinality` leverages kind classification from the kinds chapter. Connect the dots:
replaceable events have at most one per author per kind, addressable events have at most
one per author per kind per identifier.
## Design Decisions
1. **`Option<BTreeSet<T>>` for set fields** — Preserves the None-vs-empty distinction that
NIP-01 requires. BTreeSet gives O(log n) membership checks and deterministic iteration
order for serialization/hashing. (Research: rust-nostr uses this approach.)
2. **Arbitrary string tag keys** — Not restricted to single letters. The protocol allows any
tag name; single-letter indexing is a relay optimization. Consumers can enforce restrictions.
3. **Minimal builder API**`.id()`, `.author()`, `.kind()`, `.tag()`, `.address()` plus
plural variants. No convenience methods for every common tag (#e, #p, #t, etc.) — the
generic `.tag("e", value)` is clear enough. Keeps the chapter focused.
4. **`limit` in struct but not in matching** — NIP-01 defines it as part of the filter object,
so it belongs in the struct. But it's a result constraint, not a predicate, so `matches()`
ignores it. (Research: NDK, nostr-tools, all implementations agree on this.)
5. **Free functions for identity/grouping**`filter_id` and `filter_group` are not methods
because they serve infrastructure concerns. Keeps the Filter impl block focused on
construction and matching.
6. **`cardinality` returns `Option<usize>`** — `None` means unbounded. Leverages kind
classification (replaceable, addressable) to compute tight upper bounds when possible.
(Research: nostr-tools' `getFilterLimit`, nostrlib's `GetTheoreticalLimit`.)
7. **Custom serde for tag flattening** — Tags serialize as `#name` keys at the top level of
the JSON object, matching the NIP-01 wire format. This requires custom Serialize/Deserialize
implementations rather than derive macros.
8. **`.address()` convenience** — Translates an Address into the correct combination of kind,
author, and #d tag filter. This is the one domain-aware convenience method because
address-based filtering is extremely common and error-prone to construct manually.
## Open Questions
- Should `filter_group` include tag *names* (keys) in the group hash, or only the set of
field names that are present? Including tag names means `{#e: [...]}` and `{#p: [...]}`
are in different groups (correct for merging). Leaning toward including tag names.
+175
View File
@@ -0,0 +1,175 @@
# Plan: Kinds
## Topic Summary
The kinds chapter explains what event kinds are — 16-bit integers that determine how an
event's content and tags should be interpreted — without going into detail on any particular
kind. It introduces a `Kind` trait that domain chapters will implement to attach behavior
(validation, parsing, creation) to specific kind numbers. Classification methods like
`is_replaceable()` get default implementations derived from the kind number.
## Chapter Outline
1. **Introduction** — What kinds are and why they're integers not names. Draw on
building-nostr philosophy: numbers are meaningless, which allows subjective naming while
keeping technical meaning intact. Mention that this chapter builds infrastructure; the
domain chapters fill it in.
2. **The module** — Add `pub mod kinds;` to lib.rs, set up the file.
3. **The `Kind` trait** — The core abstraction. One required method: `fn kind_number() -> u16`.
This is an associated function (not `&self`) since kind number is a property of the type,
not an instance. Default methods for classification derive from it.
4. **Classification methods** — Default implementations for `is_regular()`,
`is_replaceable()`, `is_ephemeral()`, `is_addressable()`. Explain the ranges and the
special cases (kinds 0 and 3 are replaceable despite being < 10000). Mention the
building-nostr critique that coupling storage behavior to kind ranges is a design flaw,
but we implement the protocol as-is.
5. **Parsing events** — The `from_event` associated function on the trait. Takes an `&Event`,
checks `event.kind == Self::kind_number()`, then delegates to a required `parse` method.
Returns `Result<Self, KindError>`. This gives domain types a uniform entry point:
`Profile::from_event(&event)`.
6. **Creating events** — A `to_template` method that converts a domain struct back into an
`EventTemplate`. Required, no default — each kind defines its own serialization.
7. **A concrete example** — A trivial `Deletion` kind (kind 5) as a proof-of-concept
implementation, showing how domain chapters will use the trait. Simple enough to not
steal thunder from the domain chapters but real enough to prove the pattern works.
8. **What's next** — Tease kind ranges chapter (deeper dive into the range system and its
implications for relay storage and query planning).
## API Design
```rust
/// Errors when parsing an event into a domain type.
pub enum KindError {
/// The event's kind field doesn't match the expected kind number.
WrongKind { expected: u16, got: u16 },
/// The event's content or tags couldn't be parsed for this kind.
InvalidContent(String),
}
/// The core trait for attaching behavior to a kind number.
pub trait Kind: Sized {
/// The kind number this type handles.
fn kind_number() -> u16;
/// Parse an event's content and tags into this domain type.
/// Called by `from_event` after the kind number check passes.
fn parse(event: &Event) -> Result<Self, KindError>;
/// Convert this domain type back into an event template.
fn to_template(&self) -> EventTemplate;
/// Parse an event into this type, checking the kind number first.
fn from_event(event: &Event) -> Result<Self, KindError> {
if event.kind != Self::kind_number() {
return Err(KindError::WrongKind {
expected: Self::kind_number(),
got: event.kind,
});
}
Self::parse(event)
}
// --- Classification defaults ---
fn is_regular() -> bool {
let k = Self::kind_number();
k != 0 && k != 3 && k < 10_000
}
fn is_replaceable() -> bool {
let k = Self::kind_number();
k == 0 || k == 3 || (10_000 <= k && k < 20_000)
}
fn is_ephemeral() -> bool {
let k = Self::kind_number();
20_000 <= k && k < 30_000
}
fn is_addressable() -> bool {
let k = Self::kind_number();
30_000 <= k && k < 40_000
}
}
```
Also provide free functions for the same classification on raw `u16` values, for code that
doesn't have a `Kind` impl handy (e.g. relay logic, filters):
```rust
pub fn is_regular(kind: u16) -> bool { ... }
pub fn is_replaceable(kind: u16) -> bool { ... }
pub fn is_ephemeral(kind: u16) -> bool { ... }
pub fn is_addressable(kind: u16) -> bool { ... }
```
## Code Organization
- All code in `coracle-lib/src/kinds.rs`
- Module declared in `coracle-lib/src/lib.rs`
- Depends on `events` module (uses `Event`, `EventTemplate`, `EventContent`)
- No dependency on any domain crate
## Dependencies
No new external crates needed. Only uses std and types from earlier chapters.
## Narrative Notes
- Lead with the philosophy of why kinds are numbers. The building-nostr quote about
meaningless integers allowing subjective naming is compelling.
- Explain the trait design choice: why a trait instead of an enum. The set of kinds is
open-ended and grows with every NIP. An enum would require updating a central file for
every new kind. A trait lets domain chapters add kinds independently.
- The classification defaults are a nice teaching moment: "implement one method, get four
for free" demonstrates Rust's default method pattern.
- The `from_event`/`parse` split mirrors applesauce's factory/cast pattern but in Rust
idiom: `from_event` is the public API that does the kind check, `parse` is the
kind-specific logic implementors provide.
- Keep the Deletion example minimal — just enough to show the trait in action.
- Note the building-nostr critique of kind ranges coupling content type with storage
behavior, but don't dwell on it. We implement the protocol.
## Design Decisions
1. **Trait, not enum** — The set of kinds is open and grows with each NIP. An enum would
centralize all kind definitions. A trait lets each domain module add its own kinds
independently. This follows the distributed pattern seen in applesauce (no centralized
registry) and is idiomatic Rust.
2. **`kind_number()` as associated function, not const** — An associated const
(`const KIND: u16`) would also work but associated functions compose better with
default methods. Either would be fine; the chapter can use whichever reads better.
Actually, a const may be cleaner since it can't vary per call. Decide during writing.
3. **Events keep `u16`** — Per user preference. The `Kind` trait lives alongside events,
not inside them. Events are protocol-level; kinds are application-level.
4. **Free functions mirror trait methods** — Not all code has a `Kind` impl. Relay logic,
filter building, and storage layers need classification from a raw `u16`. The free
functions are the source of truth; the trait defaults call them.
5. **`from_event` with kind check as default** — Implementors only write `parse`, which
can assume the kind is correct. The trait provides the boilerplate.
6. **Deletion as example** — Kind 5 is simple (content is a reason string, tags are
references to deleted events) and universally understood. It won't conflict with
domain chapter content since deletion is a protocol-level concept.
## Open Questions
1. **Associated const vs function** — Should it be `const KIND: u16 = 5;` or
`fn kind_number() -> u16 { 5 }`? Const is more idiomatic for a fixed value.
Try const during writing and see if defaults can reference it cleanly.
2. **Should `to_template` return `EventTemplate` or `EventContent`?** — EventTemplate
includes the kind, which the trait already knows. Returning EventContent and having
a default `to_template` that adds the kind could reduce boilerplate. Explore during
writing.
+125
View File
@@ -0,0 +1,125 @@
# Plan: Proof of Work
## Topic Summary
Full implementation of NIP-13 proof-of-work: validation and generation. The mining function
receives an `OwnedEvent` reference with a target difficulty and hashes until the difficulty
is achieved, returning a `HashedEvent`. The API is batch-based and pure — no threading, no
async, no platform-specific code. Callers control the loop and can parallelize or cancel by
managing batch ranges externally.
## Chapter Outline
1. **Introduction** — What PoW means in nostr: leading zero bits on the SHA-256 event id.
Why it exists (spam resistance, relay filtering, signal of effort). Reference NIP-13.
2. **Module setup** — Add `pub mod pow;` to `coracle-lib/src/lib.rs`. Module imports.
3. **Counting leading zero bits**`get_leading_zero_bits(hash: &[u8]) -> u8`. Pure utility
that iterates bytes: full zero byte = +8 bits, partial byte uses `leading_zeros()`. This
is the foundation everything else builds on.
4. **The nonce tag** — NIP-13 specifies a `["nonce", "<counter>", "<difficulty>"]` tag. Explain
that the nonce is part of the event content that gets hashed — changing it changes the id.
Show how to add/read the tag using the existing `Tags` type.
5. **Validation**`check_pow(event: &HashedEvent, difficulty: u8) -> bool`. Checks that the
event id has at least `difficulty` leading zero bits AND that the nonce tag's claimed
difficulty is at least `difficulty`. Both checks are needed: the id check proves the work
was done, the tag check proves the miner intended at least this difficulty.
6. **Mining**`mine_pow(event: &OwnedEvent, difficulty: u8, start: Option<u64>, count: Option<u64>) -> Option<HashedEvent>`.
Clones the event's tags, appends/updates a nonce tag, hashes, checks difficulty. Iterates
nonces from `start` (default 0) for up to `count` attempts (default unbounded). Returns
`Some(HashedEvent)` on success, `None` if the batch is exhausted.
7. **Usage patterns** — Brief prose (not tangled code) showing:
- Simple: `mine_pow(&event, 20, None, None)` — blocks until done
- Batched: loop calling with `Some(cursor)` and `Some(100_000)`
- Parallel: N threads/workers with non-overlapping ranges
- WASM: worker runs batches, main thread stops sending to cancel
8. **What's next** — Transition to filters chapter.
## API Design
```rust
/// Count the leading zero bits in a byte slice.
pub fn get_leading_zero_bits(hash: &[u8]) -> u8
/// Check that an event meets a minimum proof-of-work difficulty.
/// Returns true if the event id has at least `difficulty` leading zero bits
/// AND the nonce tag claims at least `difficulty`.
pub fn check_pow(event: &HashedEvent, difficulty: u8) -> bool
/// Mine proof-of-work for an event.
///
/// Tries nonces starting from `start` (default 0) for up to `count` attempts
/// (default unbounded). Returns `Some(HashedEvent)` with the nonce tag included
/// if a valid nonce is found, `None` if the batch is exhausted.
pub fn mine_pow(
event: &OwnedEvent,
difficulty: u8,
start: Option<u64>,
count: Option<u64>,
) -> Option<HashedEvent>
```
## Code Organization
- **Crate**: `coracle-lib`
- **Module**: `coracle-lib/src/pow.rs`
- **Exports**: `get_leading_zero_bits`, `check_pow`, `mine_pow`
- **Dependencies on existing code**: `OwnedEvent`, `HashedEvent` from `events`, `Tags` from `tags`
- Add `pub mod pow;` to `coracle-lib/src/lib.rs`
## Dependencies
No new external crates. SHA-256 (`sha2`) and hex encoding are already available from the
events chapter. Leading zero bit counting uses only `u8::leading_zeros()` from std.
## Narrative Notes
- **Why both id check and tag check**: An event with 25 leading zeros but a nonce tag claiming
difficulty 10 only committed to 10 bits of work — the extra zeros are luck. Relays that
require difficulty 20 should reject it. Explain this clearly.
- **Why batch-based**: The chapter should explain the design choice. A blocking `loop until
found` function can't be cancelled and can't be parallelized. By exposing start/count, the
library stays pure and platform-agnostic while giving callers full control. Show how this
maps to threads (native) and workers (WASM).
- **Nonce tag placement**: The tag must be part of the serialized event before hashing. The
mining loop adds the tag, hashes, checks, and either returns or tries the next nonce. Work
on a clone of the input's tags — don't mutate the original.
- **Difficulty semantics**: Difficulty is measured in leading zero *bits*, not bytes or hex
characters. Difficulty 20 means the first 20 bits of the 256-bit hash are zero. This allows
fine-grained difficulty that isn't locked to 4-bit (hex) or 8-bit (byte) boundaries.
## Design Decisions
1. **Batch-based API over blocking loop**: Enables cancellation and parallelization without
platform-specific code. Callers who want simple blocking behavior pass `None` for both
start and count. Research showed rust-nostr uses a tight blocking loop with optional
multi-threading via feature flags; welshman uses Web Workers. Our approach avoids both
by pushing the concurrency concern to the caller.
2. **No prefix generation**: `get_prefixes_for_difficulty()` (seen in rust-nostr and nostr-tools)
converts bit difficulty to hex prefixes for relay querying. Deferred — it's orthogonal to
mining/validation and can be added later if needed.
3. **Nonce as u64**: rust-nostr uses u128, but u64 gives 1.8×10¹⁹ attempts — more than enough
for any practical difficulty. Keeps the tag string shorter and matches JS number limits
for WASM interop.
4. **Clone, don't mutate**: `mine_pow` takes `&OwnedEvent` and clones internally. The caller
keeps their original event unchanged. This matches the functional style of the event
pipeline (each step returns a new type).
5. **check_pow validates both id and tag**: Following NIP-13 semantics. The id check proves
the work; the tag check proves intent. Both are needed for correct validation.
## Open Questions
None — scope is well-defined and all design decisions are resolved.
+160
View File
@@ -0,0 +1,160 @@
# Plan: Protected Events
## Topic Summary
NIP-70 defines an optional, valueless `["-"]` tag — the "protected" marker.
Its presence asks relays to accept the event only when published directly by
its author: the publishing connection must be NIP-42-authenticated and the
authenticated pubkey must equal the event's `pubkey`. Relays receiving a
protected event over an unauthenticated connection should issue an `AUTH`
challenge and reject it for now; relays receiving one from an authenticated but
non-matching connection should reject it outright.
Like NIP-40 expiration, this is a **behavior tag** — orthogonal to kind, a
cooperative request rather than a guarantee. This chapter adds tag-layer support
only: a constructor, a boolean predicate, and method facades. Relay-side
enforcement is deferred to the Relay Authentication chapter (ch 26), which has
the NIP-42 machinery. The tag carries no value, so — unlike expiration — there
is nothing to parse and no `get_*` function.
## Chapter Outline
1. **Framing** — protected as a behavior tag, sibling to expiration; the
`["-"]` tag has no value; the cooperative, relay-enforced (not cryptographic)
nature of the request. Pick up the thread chapter 9 left dangling.
2. **What the relay rule is** — state the NIP-42 auth-then-compare rule in prose
so the reader understands what the tag *asks for*, and explain why none of
that logic lives here (the tag layer is stateless; enforcement needs a
connection and an authenticated pubkey — ch 26).
3. **The module declaration**`pub mod protected;` in `lib.rs` plus the module
header doc comment.
4. **The predicate**`is_protected(&Tags) -> bool`. A thin wrapper over
`Tags::has("-")`; justify it on consistency and discoverability (a named
NIP-70 function), the same justification the expiration chapter used for its
constructor. No value to parse means no `Option`, just `bool`.
5. **Building the tag**`Tag::protected() -> Tag` returning `["-"]`. One line;
the narrative explains the unusual single-dash, valueless shape.
6. **Methods on event types**`is_protected(&self)` on both `HashedEvent` and
`Event`, delegating to the free function.
7. **Usage patterns** — building a protected event with the template builder;
checking on receipt. Brief.
8. **What's next** — bridge forward: enforcement lands with relay
authentication (ch 26); note the NIP-18 repost interaction (a protected event
shouldn't be embedded in a repost) as a forward reference, not implemented
here.
## API Design
### Module `coracle-lib/src/protected.rs`
```rust
use crate::events::{Event, HashedEvent};
use crate::tags::Tags;
/// True if the tag set carries the NIP-70 `["-"]` protected marker.
pub fn is_protected(tags: &Tags) -> bool;
impl HashedEvent {
pub fn is_protected(&self) -> bool;
}
impl Event {
pub fn is_protected(&self) -> bool;
}
```
`is_protected(tags)` is `tags.has("-")`.
### Tag constructor in `coracle-lib/src/tags.rs`
```rust
impl Tag {
/// Build a NIP-70 `["-"]` protected tag.
pub fn protected() -> Tag;
}
```
Implemented as `Tag::new("-", std::iter::empty::<String>())` — a tag with the
name `"-"` and no values, serializing as `["-"]`.
## Code Organization
- **`coracle-lib/src/protected.rs`** — new module: the free function and the two
method impls. Mirrors `expiration.rs`.
- **`coracle-lib/src/lib.rs`** — add `pub mod protected;` (after `expiration`,
matching SUMMARY order: ch 10 follows ch 9).
- **`coracle-lib/src/tags.rs`** — append a `Tag::protected` constructor inside a
new `impl Tag` block (code blocks targeting the same file concatenate; another
`impl Tag` block at the end is valid Rust, exactly as `Tag::expiration` does).
- **`coracle-lib/tests/protected.rs`** — hand-written integration test, not
tangled. Covers the constructor shape, the predicate (present / absent), the
method facades on both event types, and tag survival through hash + sign.
## Dependencies
None. Pure std and existing crate types. No new `Cargo.toml` entries, no new
workspace members.
## Narrative Notes
- **Lead by connecting to expiration.** Chapter 9 ends with an explicit promise:
"The next chapter introduces another — NIP-70's `protected` tag, which tells
relays not to accept an event from anyone but its author." Open by fulfilling
that promise. The reader already has the behavior-tag mental model.
- **The valueless tag is the one novel thing.** Spend a sentence on why `["-"]`
has no value and why that collapses the whole API to a boolean — there is no
malformed-vs-absent distinction to make, no parse to fail.
- **Be honest about enforcement.** This is the "hint, not guarantee" point from
expiration, sharpened: the tag asks relays to require NIP-42 auth from the
author. Without a cooperating relay, the tag does nothing. State the relay
rule precisely (auth challenge, then pubkey comparison) so the reader knows
what they're requesting, then say plainly that implementing it is ch 26's job.
- **Justify the free function.** It is `tags.has("-")` and nothing more. Defend
it the way ch 9 defended `Tag::expiration`: a named, discoverable NIP-70 entry
point reads better at call sites and at the doc level than a bare string
literal, and it keeps the protected-events surface symmetrical with the rest
of the library (a free function on `&Tags` plus method sugar on the events).
- **Keep it short.** This is the shortest behavior-tag chapter — no parsing, no
timestamp comparison, no clock. Resist padding. The interest is conceptual
(what the tag asks of the network), not mechanical.
- **Forward references, not detours.** Mention enforcement (ch 26) and the
repost interaction once each; do not explain them.
## Design Decisions
- **Boolean, not `Option`.** The tag has no value; presence is the entire
signal. `is_protected -> bool` is correct and matches every reference
(`isProtectedEvent`, `IsProtected`, `Event::is_protected`).
- **No `get_protected`.** Nothing to return but a bool, which `is_protected`
already gives. A `get_*` would imply a value that does not exist.
- **Free function on `&Tags` plus method facades.** Matches the
`get_expiration`/`get_pow` pattern exactly. The free function is reusable by
any caller holding a tag slice (e.g., a relay inspecting raw tags before
reconstructing an `Event`); the methods are the ergonomic surface.
- **Constructor in `tags.rs`, not `protected.rs`.** Lives with `Tag::new` and
`Tag::expiration`; discoverable where callers build tags; no import from
`protected` just to construct one. Identical placement reasoning to ch 9.
- **Enforcement deferred to ch 26.** The acceptance rule needs an authenticated
connection and a pubkey to compare against — state, not a stateless tag query.
Putting a `should_accept`-style helper here (the option the user declined)
would strand half the logic away from where the connection lives. The chapter
describes the rule and points forward.
- **No `set_protected(bool)` builder helper.** ndk/applesauce expose a
toggle, but our builder surface is already chainable via
`.tags(Tags::new().add(...))` / `Tag::protected()`. Adding a toggle would be
ceremony, consistent with ch 9 declining `set_expiration`.
- **Constructor uses an empty value iterator.** `Tag::new("-", iter::empty())`
produces `["-"]`. Verified against `Tag::new`'s signature (name plus an
`IntoIterator` of values); an empty iterator yields a one-element tag.
## Open Questions
- Whether to show the relay-enforcement pseudocode inline. Decision: no — a prose
statement of the rule is enough; code belongs in ch 26 where the connection and
auth state exist. Showing non-tangled relay code here would invite the reader
to think enforcement is part of this layer.
- Whether to add a `Tags`-level convenience beyond the free function. Skip, per
ch 9 reasoning: keep `Tags` a generic container; the named function lives in
`protected.rs`.
- Whether to document the NIP-18 repost interaction in depth. Keep it to a single
forward-reference sentence; reposts are their own topic.
+189
View File
@@ -0,0 +1,189 @@
# Plan: Search
## Topic Summary
NIP-50 adds an optional full-text `search` field to the subscription filter from
chapter 11. A relay that supports the capability interprets the query against
event content (and, for some kinds, other fields), returning results ordered by
relevance rather than `created_at`, with `limit` applied after ranking. The
query may carry `key:value` extensions — `domain:`, `language:`, `sentiment:`,
`nsfw:`, `include:spam` — which relays may support or ignore.
This chapter extends `Filter` with a `search` field, threads it through
serialization / grouping / set algebra, introduces a typed `SearchQuery` that
splits free-text terms from `key:value` extensions, and implements a best-effort
local relevance **score in [0, 1]** used to both include and rank events —
mirroring the NIP's "descending order by quality of result, limit last."
## Chapter Outline
1. **Intro / framing** — Search as a relay-defined, optional capability; content
discovery is client-initiated routing, not a global index; results are
partial and ranked by the relay. The local matcher is an honest best-effort
fallback, not a reimplementation of relay search.
2. **The `search` field** — Add `search: Option<String>` to `Filter`; builder
methods `add_search` / `clear_search`; note it joins the derived `Hash` (so
`id()` covers it for free).
3. **Serialization** — Emit/parse a plain `"search"` key in the hand-written
serde impl, present only when `Some`.
4. **The `SearchQuery` model** — A new `search` module: terms + ordered
`key:value` extensions, `parse`, `Display`, builders, and the `Filter` bridge.
5. **Scoring & matching**`search_score` (fraction-of-terms + diminishing
frequency bonus, capped at 1.0); `matches` includes an event when score > 0;
`rank_search_results` sorts by score then `created_at` and applies `limit`.
6. **Grouping and set algebra**`search` enters `group()` (distinct searches
never merge); `union_filters` carries it through unchanged; `intersect_filters`
keeps a conflicting-search pair separate instead of fabricating a combined query.
7. **What's next** — Brief pointer to the Domain section (relay selection,
discovering NIP-50-capable relays via relay metadata, is a later concern).
## API Design
### `coracle-lib/src/filters.rs` (extends existing `Filter`)
```rust
pub struct Filter {
// ... existing fields ...
/// NIP-50 full-text search query. Relay-interpreted; see `SearchQuery`.
pub search: Option<String>,
}
impl Filter {
pub fn add_search(self, search: impl Into<String>) -> Self; // sets Some
pub fn clear_search(self) -> Self; // sets None
/// Bridge to the typed model.
pub fn add_search_query(self, query: &SearchQuery) -> Self; // = add_search(query.to_string())
pub fn search_query(&self) -> Option<SearchQuery>; // parse the field back
/// Best-effort local relevance score in [0.0, 1.0].
/// Returns 1.0 when there is no search, or a search with no free-text
/// terms (only extensions, which are unenforceable locally).
pub fn search_score(&self, event: &Event) -> f64;
}
/// Filter `events` to those matching `filter`, sort by relevance
/// (search_score desc, then created_at desc), and apply `filter.limit`.
pub fn rank_search_results<'a>(filter: &Filter, events: &'a [Event]) -> Vec<&'a Event>;
```
`matches` gains a final check: `if self.search_score(event) == 0.0 { return false }`.
Because `search_score` returns 1.0 when there is no search (or no terms), this
only rejects when a search *with terms* matched none of them — i.e. "any term
present ⇒ included."
### `coracle-lib/src/search.rs` (new module)
```rust
/// A parsed NIP-50 search query: free-text terms plus `key:value` extensions.
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct SearchQuery {
pub terms: Vec<String>,
pub extensions: Vec<(String, String)>, // ordered; repeats allowed
}
impl SearchQuery {
pub fn new() -> Self;
/// Total parse: split on whitespace; a token is an extension iff it is
/// `key:value` with key in [A-Za-z0-9_-]+, non-empty value not starting
/// with '/'. Everything else is a term. Never fails.
pub fn parse(input: &str) -> Self;
pub fn add_term(self, term: impl Into<String>) -> Self;
pub fn add_extension(self, key: impl Into<String>, value: impl Into<String>) -> Self;
pub fn is_empty(&self) -> bool;
}
impl fmt::Display for SearchQuery { /* terms first, then "key:value" exts, space-joined */ }
```
`Filter::matches` / `search_score` tokenize via `SearchQuery::parse`, using only
`terms` (extensions are ignored by the local matcher).
### Scoring formula (`search_score`)
For the parsed query's distinct `terms` (case-insensitive), against
`event.content` lowercased:
- `total` = number of distinct terms; if 0 → return 1.0.
- For each term, `count` = non-overlapping occurrences in content.
- `matched` = terms with `count ≥ 1`; `extra` = (Σ count) matched (repeats
beyond the first hit of each matched term).
- `base = matched / total` (fraction of terms present, in [0, 1]).
- `bonus = (1 1/(1 + extra)) / total` (diminishing, strictly `< 1/total`, so a
partial match never reaches the next term's bucket).
- `score = (base + bonus).min(1.0)`.
Properties (asserted in tests): in [0, 1]; all terms once ⇒ 1.0; missing a term
`< 1.0`; more occurrences ⇒ ≥ score (monotonic, never exceeds 1.0); no terms
matched ⇒ exactly 0.0.
## Code Organization
- **`coracle-lib/src/filters.rs`** — add the `search` field, builders, the
serde changes, `search_score`, the `matches` check, `rank_search_results`,
and the `group()` / `intersect_filters` updates. `use crate::search::SearchQuery;`.
- **`coracle-lib/src/search.rs`** — the `SearchQuery` type. New `pub mod search;`
in `lib.rs`, placed before `filters` (filters depends on it).
- **`coracle-lib/src/prelude.rs`** — add `pub use crate::search::SearchQuery;`
(the prelude already re-exports commonly used items).
- **`coracle-lib/tests/search.rs`** — hand-written integration tests (not tangled).
## Dependencies
None new. Parsing and matching use `std` only. No FTS engine — out of scope and
against the minimal-dependency rule.
## Narrative Notes
- Open with the philosophy: search is opt-in and relay-defined; no global index;
results partial and relay-ranked. Frame the local scorer as a fallback for
in-memory/offline querying, and warn (per rust-nostr's SDK) that re-filtering a
relay's returned results client-side can wrongly drop legitimate hits — relays
rank with richer, extension-aware logic.
- Explain *why* extensions are parsed but **ignored locally**: `sentiment:`,
`domain:`, etc. require data the client doesn't have, so honoring them locally
is impossible; we keep them in the typed model for *building/inspecting*
queries, not for local evaluation.
- Justify the score model concretely: NIP-50 mandates relevance ordering, so a
boolean match is the wrong shape — a [0,1] score lets us both include
(score > 0) and rank. Walk through the fraction + diminishing-bonus formula
with a small worked example.
- For grouping: reuse the chapter-11 reasoning — two filters with different
searches can't be unioned without changing semantics, so `search` joins the
group key. Show that `union_filters` then keeps them separate automatically.
- For `intersect_filters`: explain the one structural change — `combine_pair`
returns `Option<Filter>`; a pair whose two searches differ returns `None`, and
the caller emits both filters separately rather than concatenating queries.
## Design Decisions
1. **Typed `SearchQuery`, lean/generic.** Terms + a generic ordered list of
`key:value` extensions, with `add_term`/`add_extension`. No per-extension
helpers or typed enums — keeps the surface small and forward-compatible with
relay-specific extensions. (Every reference treats search as opaque; the typed
model is our value-add.)
2. **Local relevance score in [0, 1]**, fraction-of-terms + diminishing frequency
bonus, capped at 1.0. Chosen over a boolean to model NIP-50's relevance
ordering. Extensions excluded from scoring.
3. **`matches` includes on score > 0** ("any term present"); ranking via
`rank_search_results` handles relevance + `limit`-after-sort.
4. **`search` participates in `group()`**, so `union_filters` never merges
distinct searches.
5. **`intersect_filters` keeps a conflicting-search pair separate** (combine
returns `Option`, `None` ⇒ emit both) rather than concatenating, per the
user's choice.
6. **Builder naming `add_search`/`clear_search`** to match the existing
`add_since`/`clear_since` vocabulary (not rust-nostr's `search`/`remove_search`).
7. **Unicode-aware lowercasing** (`to_lowercase`) for the local matcher rather
than ASCII-only, given multilingual nostr content; note the allocation
trade-off. Substring counting via `str::matches`.
8. **Extension parse heuristic** documented: a colon-bearing token like a URL may
be read as an extension; applications needing exact control build
`SearchQuery` field-by-field instead of parsing.
## Open Questions
- Exact wording of the frequency-bonus explanation — keep the formula in prose
light; lean on a worked example. (Resolved during writing.)
- Whether `rank_search_results` belongs as a free function (consistent with
`matches_any`/`union_filters`) — yes, free function.
+243
View File
@@ -0,0 +1,243 @@
# Plan: Tags
## Topic Summary
Introduce a proper `Tag` type to replace the `Vec<Vec<String>>` used in the
events chapter. Tags carry the structured data half of every event:
references to pubkeys, events, addresses, topics, relay hints, and anything
else machine-readable. This chapter defines a thin wrapper around
`Vec<String>`, a small set of accessors, free helper functions that query
slices of tags by name, and an `Address` type for parsing and constructing
the `kind:pubkey:identifier` form used by `a` tags. It then retrofits the
existing `Event` pipeline to hold `Vec<Tag>` instead of `Vec<Vec<String>>`.
The chapter stays neutral on tag semantics — no `TagKind` enum, no marker
parsing. The philosophy (from building-nostr) is clear: the meaning of a
tag depends on the event's kind, and a generic type that tries to know
better is almost always wrong.
## Chapter Outline
1. **Opening framing.** Tags are the structured half of events. Lists of
lists of strings, not maps, because ordering matters and keys repeat.
Single-letter tags are indexed by relays and filterable via `#e`, `#p`,
etc. Multi-character tags live in the same array but aren't indexed.
Point out that tag semantics depend on event kind — this type stays
neutral.
2. **The module.** Register `tags` in `lib.rs`, imports, `use` of
`PublicKey`.
3. **The `Tag` type.** `pub struct Tag(pub Vec<String>)`, tuple struct with
transparent serde. `Tag::new(name, values)` constructor.
`impl From<Vec<String>> for Tag` and back. `impl Deref<Target = [String]>`
for ergonomic slice access.
4. **Accessors.**
- `fn name(&self) -> &str` — first entry, or empty string if empty
- `fn value(&self) -> &str` — second entry, or empty string
- `fn values(&self) -> &[String]` — everything after the name
- `fn get(&self, i: usize) -> Option<&str>`
- `fn len(&self) -> usize`, `fn is_empty(&self) -> bool`
5. **Slice helpers.** Free functions that take `&[Tag]`:
- `pub fn find<'a>(tags: &'a [Tag], name: &str) -> Option<&'a Tag>`
- `pub fn find_all<'a>(tags: &'a [Tag], name: &str) -> impl Iterator<Item = &'a Tag>`
- `pub fn value<'a>(tags: &'a [Tag], name: &str) -> Option<&'a str>`
- `pub fn values<'a>(tags: &'a [Tag], name: &str) -> impl Iterator<Item = &'a str>`
- `pub fn has(tags: &[Tag], name: &str) -> bool`
6. **A word on markers.** NIP-10 puts thread markers at `tag[3]` on `e`
tags. Show how to read them with `tag.get(3)` without introducing a
typed marker. Note that reply-thread parsing is kind-aware and belongs
in a later chapter.
7. **The `Address` type.** For `a` tags:
```
"30023:npub...:my-article-slug"
```
- Struct: `kind: u16`, `pubkey: PublicKey`, `identifier: String`
- `Address::from_str` / `Display` — splits on `:`, validates kind and
pubkey hex, preserves identifier as-is
- `Address::to_tag(&self) -> Tag` — emits `["a", "kind:pubkey:d"]`
- `Address::from_tag(&Tag) -> Option<Address>` — reads index 1 of an
`a` tag and parses it
- `AddressError` enum with `InvalidFormat`, `InvalidKind`,
`InvalidPubkey`
8. **Retrofit `Event`.** The events chapter holds tags as
`Vec<Vec<String>>`. Replace that everywhere with `Vec<Tag>`. The
canonical JSON produced by `Sha256::digest(canonical(...))` must remain
byte-identical — `#[serde(transparent)]` on `Tag` guarantees this
because the canonical form goes through `serde_json::json!`, which
sees each `Tag` as its inner `Vec<String>`. Update all six structs in
the pipeline, the canonical helper, the `Visitor::visit_map`, and the
tests.
9. **Worked example.** Build an event with a few typed tags:
```rust
let tags = vec![
Tag::new("t", ["nostr"]),
Tag::new("p", [pubkey.to_hex()]),
Address { kind: 30023, pubkey, identifier: "slug".into() }.to_tag(),
];
```
Then show how to read them back with `tags::value(&event.tags, "t")`
etc. This is illustrative prose, not tangled.
10. **What's next.** Pointer toward kinds: the type that interprets what
a given tag collection *means* in the context of a particular event.
## API Design
New in `coracle-lib/src/tags.rs`:
```rust
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(transparent)]
pub struct Tag(pub Vec<String>);
impl Tag {
pub fn new<N, V, S>(name: N, values: V) -> Self
where N: Into<String>, V: IntoIterator<Item = S>, S: Into<String>;
pub fn name(&self) -> &str;
pub fn value(&self) -> &str;
pub fn values(&self) -> &[String];
pub fn get(&self, i: usize) -> Option<&str>;
pub fn len(&self) -> usize;
pub fn is_empty(&self) -> bool;
pub fn as_slice(&self) -> &[String];
}
impl From<Vec<String>> for Tag { ... }
impl From<Tag> for Vec<String> { ... }
impl std::ops::Deref for Tag { type Target = [String]; ... }
pub fn find<'a>(tags: &'a [Tag], name: &str) -> Option<&'a Tag>;
pub fn find_all<'a>(tags: &'a [Tag], name: &str)
-> impl Iterator<Item = &'a Tag>;
pub fn value<'a>(tags: &'a [Tag], name: &str) -> Option<&'a str>;
pub fn values<'a>(tags: &'a [Tag], name: &str)
-> impl Iterator<Item = &'a str>;
pub fn has(tags: &[Tag], name: &str) -> bool;
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AddressError {
InvalidFormat,
InvalidKind,
InvalidPubkey,
NotAnAddressTag,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Address {
pub kind: u16,
pub pubkey: PublicKey,
pub identifier: String,
}
impl Address {
pub fn to_tag(&self) -> Tag;
pub fn from_tag(tag: &Tag) -> Result<Self, AddressError>;
}
impl std::str::FromStr for Address { ... }
impl std::fmt::Display for Address { ... }
```
## Changes to `events.rs`
- `use crate::tags::Tag;` at the top.
- Every `tags: Vec<Vec<String>>` becomes `tags: Vec<Tag>`.
- The `canonical()` helper accepts `tags: &[Tag]` and still serializes
identically (Tag is `#[serde(transparent)]`).
- `Visitor::visit_map`'s `tags: Option<Vec<Vec<String>>>` becomes
`tags: Option<Vec<Tag>>`. Same serde behavior.
- The worked example in prose should use `Tag::new(...)`.
## Code Organization
- `coracle-lib/src/tags.rs` — new file, tangled from the chapter.
- `coracle-lib/src/lib.rs` — append `pub mod tags;` via a small
block in tags.md.
- `coracle-lib/src/events.rs` — existing file, modified by editing
`book/04-events.md` where its blocks are tangled.
- `coracle-lib/tests/tags.rs` — new hand-written integration tests.
- `coracle-lib/tests/events.rs` — updated to use `Tag::new` in
fixtures instead of `vec!["t".into(), "nostr".into()]`.
## Dependencies
All already present. No new crates.
## Narrative Notes
- **Lead with the philosophy, not the code.** Building-nostr's framing is
precise: lists of lists, ordering matters, data tags vs filter tags vs
behavior tags got conflated, single-letter tags are indexed. Putting
this first makes the eventual tiny type feel justified.
- **Justify the thinness.** Readers coming from rust-nostr may expect a
huge enum. Spell out why we don't: tag meaning is kind-dependent, new
tags appear constantly, and a `Custom` catch-all variant is a sign the
abstraction is in the wrong place.
- **Show retrofitting as routine.** Don't hide the fact that we're
revising `events.rs`. Literate programs are allowed to grow backward —
that's part of why they're literate.
- **Address gets special treatment.** It's the one tag shape worth a
struct: three fields that always appear together and can be validated
at parse time. Contrast with `e`/`p`/`t` where the payload is just a
single string and a struct would be gratuitous.
## Design Decisions
1. **Tuple struct, not named field.** Matches `PublicKey`/`SecretKey` from
chapter 02. Reads naturally and allows direct destructuring.
2. **`#[serde(transparent)]`.** The wire format is unchanged. Canonical
hash bytes are unchanged. Prior events round-trip byte-for-byte.
3. **Deref to `[String]`.** Gives iter, len, get for free without a
pile of forwarding methods.
4. **`name()`/`value()` return `&str`, never `Option`.** Empty tags would
be a protocol error. Returning `""` when the slice is empty keeps call
sites simple and matches how every reference implementation reads
tags.
5. **Free functions, not methods on a `Tags` newtype.** A `Tags(Vec<Tag>)`
wrapper would force users to convert back and forth for serde. Free
functions over slices compose with whatever container the caller
holds.
6. **No `TagKind` enum.** Single tag names stay as bare `&str`. Building
an enum would bake in the semantic-per-kind decision we explicitly
reject.
7. **No marker type.** NIP-10 markers live at a positional index and
their meaning depends on the event kind. Reply/thread parsing belongs
in a reply chapter.
8. **`Address` takes `u16` for kind, matching `Event::kind`.** Addresses
in the wild occasionally use kinds beyond 65535 — we match the event
type anyway for consistency and revisit only if it bites.
9. **`Address::from_tag` returns `Result`, not `Option`.** Distinguishing
"not an `a` tag" from "malformed `a` tag" is useful for error
messages and matches our existing error-enum style.
10. **No `relay` field on `Address`.** The third element of an `a` tag is
a relay hint, not part of the address. Relay hints are a concept we
introduce with relay selections later; folding them in here would be
premature.
## Open Questions
- **Should `Tag::new` take a slice literal instead of `IntoIterator`?**
`Tag::new("t", ["nostr"])` reads well with `IntoIterator` because
array literals satisfy it. Keeping `IntoIterator` to allow passing
iterators directly from callers.
- **Should we add `event.tag(name)` / `event.tag_value(name)` methods on
`Event`?** Tempting, but method clutter on `Event` grows fast. Sticking
to free functions in `tags::` that take `&event.tags`. Revisit if
ergonomics suffer in later chapters.
+103
View File
@@ -0,0 +1,103 @@
# Research: Addresses
## Topic Summary
Create an `Address` struct representing a nostr event address (`kind:pubkey:d-tag`). It should:
- Be constructable from an event via `from_event`
- Encode/decode to the `naddr` NIP-19 bech32 format (TLV-based)
- Encode/decode to the `kind:pubkey:identifier` string format
- Provide convenient property access (kind, pubkey, identifier)
The reader already understands kind ranges and addressable events from the kinds chapter.
## Philosophy
From `ref/building-nostr`:
- Events are content-addressable via their ID (hash), giving referential transparency. But replaceability broke this — a replaceable event's ID changes on every update, so you need a stable reference.
- An "address" is `kind:pubkey:identifier`, pointing to all events sharing the same kind, pubkey, and d-tag.
- The `a` tag carries this triple as its value, with an optional relay hint in position 2.
- Design principle: "you should have a slight bias towards regular, non-replaceable events because they don't break referential transparency."
- The address includes the author's pubkey, which enables outbox-model relay lookup — unlike a bare event ID.
## Reference Implementation Analysis
### applesauce
- No custom Address struct — re-exports `AddressPointer` from nostr-tools: `{ kind, pubkey, identifier, relays? }`
- Also has `AddressPointerWithoutD` for replaceable (non-addressable) events
- `createReplaceableAddress(kind, pubkey, identifier?)` concatenates with `:`
- `parseReplaceableAddress(str)` splits on `:`, handles identifiers containing colons by joining remaining parts
- `getAddressPointerForEvent(event, relays?)` extracts d-tag: `event.tags.find(t => t[0] === "d")?.[1] ?? ""`
- Delegates naddr encoding to nostr-tools
### ndk
- No dedicated Address type — handles addresses on NDKEvent directly
- `tagAddress()` returns `"${kind}:${pubkey}:${dTag ?? ""}"`
- `dTag` getter/setter on NDKEvent
- Auto-generates d-tag for param replaceable events if missing (random string)
- `filterFromId(id)` handles three input forms: raw address string, naddr bech32, or event ID
- Regex: `NIP33_A_REGEX = /^(\d+):([0-9A-Fa-f]+)(?::(.*))?$/`
- Delegates nip19 to nostr-tools
### nostr-gadgets
- No dedicated Address type — addresses are plain `kind:pubkey:d-tag` strings
- `isATag(input)` validates via regex: `/^\d+:[0-9a-f]{64}:[^:]+$/`
- Imports `AddressPointer` from nostr-tools when needed
- Inline parsing by splitting on `:`
- d-tag defaults to empty string
### nostrlib (Go)
- Type: `EntityPointer { PublicKey, Kind, Identifier, Relays []string }`
- `AsTagReference()`: `fmt.Sprintf("%d:%s:%s", ep.Kind, ep.PublicKey.Hex(), ep.Identifier)`
- `ParseAddrString(addr)`: splits on `:` (max 3 parts)
- `Tags.GetD()` returns first d tag value or ""
- `MatchesEvent` checks kind + pubkey + identifier
- NIP-19 TLV: identifier=TLV0, relays=TLV1, author=TLV2(32 bytes), kind=TLV3(u32 BE)
- No separate from_event — caller assembles EntityPointer
### nostr-tools
- `AddressPointer = { identifier: string, pubkey: string, kind: number, relays?: string[] }`
- naddr TLV: 0=identifier(UTF-8), 1=relay(multiple), 2=pubkey(32 bytes), 3=kind(u32 BE)
- Strict decode: TLV 0, 2, 3 required; relays default to []
- a tag parsed as `tag[1].split(':')`
### rust-nostr
- `Coordinate { kind: Kind, public_key: PublicKey, identifier: String }`
- `Nip19Coordinate { coordinate: Coordinate, relays: Vec<RelayUrl> }` (separate from core Coordinate)
- `from_kpi_format(str)` parses `kind:pubkey:d-tag` via split on `:`
- `parse(str)` tries kpi format, then bech32, then NIP-21 URI
- `Display` outputs `kind:pubkey:identifier`
- `FromStr` delegates to `parse()`
- `verify()` validates: replaceable must have empty identifier, addressable must have non-empty
- TLV encode: SPECIAL(0)=identifier, AUTHOR(2)=32-byte pubkey, KIND(3)=u32 BE, RELAY(1)=URL strings
- TLV decode: while loop consuming bytes, first occurrence wins for each field type
- `Coordinate` converts `Into<Tag>` and `Into<Filter>`
- `CoordinateBorrow<'a>` for zero-copy operations
### welshman
- `Address` class with `kind, pubkey, identifier, relays`
- `Address.from(str, relays?)` parses `kind:pubkey:identifier` via regex
- `Address.fromNaddr(naddr)` decodes via nostr-tools
- `Address.fromEvent(event, relays?)` extracts d-tag: `event.tags.find(t => t[0] === "d")?.[1] || ""`
- `toString()` returns `kind:pubkey:identifier`
- `toNaddr()` delegates to nostr-tools
- `getAddress(event)` shorthand
- `Address.isAddress(str)` regex test
## Common Patterns
1. **Three fields**: kind (u16), pubkey (32 bytes), identifier/d-tag (string). All implementations agree.
2. **String format**: `kind:pubkey:identifier` — universal, colon-separated, kind is decimal.
3. **NIP-19 TLV**: identifier=type 0, relay=type 1, author=type 2 (32 bytes), kind=type 3 (u32 big-endian).
4. **d-tag defaults to empty string**, not None/null — important for replaceable events that have no d-tag.
5. **from_event pattern**: extract d-tag from `tags.find("d")?.value() ?? ""`, combine with event.kind and event.pubkey.
6. **Relay hints are separate**: the core address is just the triple; relays are transport metadata. Most implementations keep them optional/separate.
7. **Parsing colon-split**: most implementations just split on `:` and take 3 parts. Some handle identifiers containing colons by joining remaining parts.
## Considerations for Our Implementation
- Keep the core `Address` struct to the three fields (kind, pubkey, identifier). Relay hints can be added later as a wrapper or in the naddr encoding methods.
- `from_event` should work on our existing `Event` type, using `tags.value("d")` which already exists.
- NIP-19 TLV encoding is new infrastructure — the keys chapter only used simple bech32 (no TLV). This chapter introduces the TLV pattern that will be reused for `nprofile` and `nevent` later.
- `Display`/`FromStr` should use the `kind:pubkey:identifier` format (the `a` tag value), since that's the most common string representation.
- Unlike rust-nostr, we don't need to validate that addressable kinds have non-empty identifiers — an address is just a reference triple, and empty identifiers are valid for replaceable events.
- The identifier field should handle colons in d-tags correctly during parsing.
+302
View File
@@ -0,0 +1,302 @@
# Research: Events
## Topic Summary
The events chapter covers the core nostr Event data structure: fields (id, pubkey,
created_at, kind, tags, content, sig), JSON serialization, canonical form, event ID
computation (sha256 of canonical JSON), unsigned vs signed event representations, and
structural validation. Signing is addressed by adding a `sign` method to `PrivateKey`
that signs an arbitrary string (not an event). The chapter includes an illustrative
example of event signing using this primitive, but does not tangle a sign method on
events themselves.
## Philosophy
From `ref/building-nostr/content/book.md` and `summary.md`:
**Events as referentially transparent facts.** An event's ID is the hash of its
content (NIP-01), making events content-addressable and immutable. If you have an
event ID, you know the exact content forever. This is the foundation of nostr's
break from server-centric architectures. Regular, non-replaceable events should be
the default because replaceable events break referential transparency.
**Permissionless validity through signatures.** Events gain legitimacy through
secp256k1 Schnorr signatures. Any event that is signed is valid simply by virtue of
existing — validation is limited to hash and signature verification plus a few
structural checks. The protocol explicitly rejects schema enforcement in favor of
social/reputational filtering.
**Minimal kernel.** Events are the entire nostr substrate. All application
semantics emerge from event kinds and tagged references. The protocol itself is "a
very humble protocol — it is little more than an empty shell which users can then
fill with content types that solve their use cases."
**Four-part structure.**
- Cryptographic core: id, pubkey, sig
- Metadata: created_at, kind
- Structured data: tags (array of string arrays)
- Human-readable: content
Content should hold human-readable payloads. Encoding JSON or encrypted data in
content is considered an antipattern (though encrypted DMs necessarily do this).
**Weak timestamps.** `created_at` is a second-granularity Unix timestamp supplied
unilaterally by the author. Ordering is delegated to the social layer; nostr refuses
to solve distributed clock problems.
**Fault-tolerant validation.** "Liberal in what you accept" applied conservatively:
tolerate malformed data but always verify signatures. Never repair data that
contradicts conventions.
## Reference Implementation Analysis
### applesauce
Re-exports core types (`NostrEvent`, `EventTemplate`, `UnsignedEvent`,
`VerifiedEvent`) from `nostr-tools/pure` rather than defining its own. Adds
kind-narrowing wrappers `KnownEvent<K>` etc. for type safety. Delegates
serialization, hashing, and signature verification entirely to nostr-tools.
The notable contribution is `EventFactory` (`packages/core/src/factories/event.ts`),
a fluent builder extending `Promise<T>` with chainable operations:
`fromKind(1).content("hello").stamp(signer).sign(signer)`. Three lifecycle states:
`EventTemplate``UnsignedEvent` (has pubkey) → `NostrEvent` (signed).
Structural validation (`isEvent`) checks string/number types and 64-char hex lengths
for id/pubkey but does NOT verify signatures. Signature verification is pluggable on
the `AsyncEventStore` via a `verifyEvent` callback, cached after first verification
via a symbol property. Attaches metadata via symbols to avoid polluting the event
object.
### ndk
Uses class-based design: `NDKEvent` extends `EventEmitter` with mutable public
fields. Provides inheritance hierarchy for kind-specific subclasses (NDKArticle,
NDKZap, etc.).
Canonical serialization in `core/src/events/serializer.ts` produces the NIP-01 JSON
array `[0, pubkey, created_at, kind, tags, content]`. Hash computed via
`@noble/hashes` sha256 plus hex encoding. Validation uses regex
`/^[a-f0-9]{64}$/` for pubkey format, and throws detailed errors with event
snapshots on structural failure.
Signature verification supports two modes: synchronous (blocking, via
`@noble/curves` schnorr) or Web-Worker offloaded for non-blocking verification in
browsers. Results cached in a 1000-entry LRU with 60s TTL. Type-safe signed vs
unsigned via discriminated unions (`NDKSignedEvent`, `NDKUnsignedEvent`) with guard
functions.
`toNostrEvent()` finalizes by computing id hash before signing; `sign()` delegates
to pluggable `NDKSigner`. Notable: in-place mutation is pervasive (setters on all
fields), trading safety for ergonomics.
### nostr-gadgets
Treats events as plain TS objects imported from `@nostr/tools/pure`; does not
define its own Event type. Consumes already-finalized events — ID computation,
signing, and verification are all delegated to nostr-tools.
Interesting only for its storage strategy: RedEventStore constructs JSON manually
(`'{"pubkey":"${event.pubkey}","id":"${event.id}",...}'`) to control field order
and append custom metadata like `seen_on`. Validation is limited to structural
bounds: `created_at > 0xffffffff || kind > 0xffff`. No signature verification at
the storage layer — that's assumed to happen upstream at the relay/pool boundary.
Uses Symbol properties (`isLocalSymbol`, `seenOnSymbol`) to attach metadata without
mutating events. No builder pattern — callers use `finalizeEvent` from
nostr-tools directly.
### nostr-tools
The de facto reference for TypeScript nostr. Defines the canonical `NostrEvent`
type as a plain object (not a class). Uses a `verifiedSymbol` for type
discrimination of verified events without wrapping them.
Three type variants via `Pick<>`:
- `EventTemplate`: `kind`, `tags`, `content`, `created_at`
- `UnsignedEvent`: adds `pubkey`
- `Event` / `VerifiedEvent`: complete with `id`, `sig`
Canonical serialization in `pure.ts`:
```typescript
function serializeEvent(evt: UnsignedEvent): string {
return JSON.stringify([0, evt.pubkey, evt.created_at, evt.kind, evt.tags, evt.content])
}
```
`getEventHash()` is sha256 over UTF-8 encoded canonical JSON, returned as hex.
`finalizeEvent(template, secretKey)` bridges template → signed event, mutating the
template in-place for efficiency.
Validation (`validateEvent`) checks field types, regex-validates pubkey format,
and verifies tag shape. `verifyEvent` performs both ID recomputation (catches
tampering) and Schnorr signature verification via `@noble/curves/secp256k1`. Result
cached via `verifiedSymbol`.
Dual backend: `pure.ts` uses pure-JS `@noble`; `wasm.ts` provides a WASM
alternative with identical API.
### rust-nostr
The most relevant reference, being Rust. Defines three event representations:
**`Event`** (`event/mod.rs`): `#[non_exhaustive]` struct with public immutable
fields. Implements `Clone`, `Eq`, `Ord` (by created_at desc, then id), `Hash`.
Construction only via `Event::new()`, protecting against future additive changes.
**`UnsignedEvent`** (`event/unsigned.rs`): Same fields minus `sig`, with `id` as
`Option<EventId>` — lazily computed on first access via interior mutability.
**`EventBorrow<'a>`** (`event/borrow.rs`): Zero-copy view for serialization and
validation without allocating; holds `&[u8; 32]` for id/pubkey/sig.
**Supporting types:** `EventId` is a newtype around `[u8; 32]`, serialized as hex;
`Kind` is an enum with named variants plus `Custom(u16)`; `Tag` wraps
`Vec<String>` with lazy standardization parsing via `OnceCell`.
**Serialization:** Uses an `EventIntermediate<'a>` struct with `Cow<'a, T>` fields
for zero-copy serialize and owned deserialize. Custom hex encoding for signatures
via `serialize_with`. Silently ignores unknown JSON fields (lenient for relay
compat).
**ID computation:** `EventId::new()` uses `serde_json::json!()` to build the
canonical `[0, pubkey, created_at, kind, tags, content]` array, serializes to
string, hashes with `bitcoin_hashes::sha256::Hash`, returns as 32-byte newtype.
**Validation:**
- `Event::verify_id()`: recompute and compare
- `Event::verify_signature()`: Schnorr via `secp256k1::verify_schnorr`
- `Event::verify()`: both, returning `Result<(), Error>`
**EventBuilder** (`event/builder.rs`): fluent builder that produces `UnsignedEvent`
via `.build(pubkey)`. Includes POW mining (optional multi-threaded via
`Arc<AtomicBool>`), self-tag filtering, tag deduplication, and custom timestamps.
Signing via `NostrSigner` trait (async) or `Keys` directly (sync).
**no_std support:** Conditional `core::cell::OnceCell` vs `std::sync::OnceLock`.
**Dependencies:** `secp256k1 0.29`, `serde/serde_json`, `bitcoin_hashes`,
`faster-hex`. Event module is self-contained aside from tight coupling to Tag/Kind
submodules.
### welshman
Data-oriented, composition-first design. Defines a layered type hierarchy:
```
EventTemplate → StampedEvent → OwnedEvent → HashedEvent → SignedEvent
(content) (+ created_at) (+ pubkey) (+ id) (+ sig)
```
Each is a structural type. Construction is a pipeline of pure functions:
`stamp(event, created_at) → own(event, pubkey) → hash(event) → sign(event, secret)`.
No builder class; composition via function calls.
Delegates crypto to `nostr-tools` and `@noble/curves`. `getHash` is a one-liner
wrapping `getEventHash`. Signature verification upgrades from pure-JS to WASM
asynchronously if available, caches results via the `verifiedSymbol`.
Adds `TrustedEvent`: an event accepted without verification (e.g., from canonical
relay storage). Represents the distinction between "cryptographically verified"
and "contextually trusted."
Structural type guards (`isEventTemplate`, `isSignedEvent`) check field types and
formats.
## Common Patterns
**Canonical serialization.** All implementations use NIP-01's JSON array form
`[0, pubkey, created_at, kind, tags, content]`. Order matters (deterministic
output).
**Event ID = sha256(canonical JSON).** Universal. Implementations differ only in
which sha256 library they use.
**Layered types for lifecycle states.** Everyone distinguishes template / unsigned
/ signed, though the exact type names vary (welshman has the most granular
hierarchy with five stages).
**Delegation.** Every TypeScript library delegates the hashing and signature
primitives to `@noble/curves` + `@noble/hashes` (via nostr-tools or directly).
Only rust-nostr and nostr-tools actually implement canonical serialization
themselves.
**Structural validation is cheap and separate from signature verification.**
Libraries uniformly distinguish "is this the right shape?" from "is the signature
valid?" — and often verify signatures lazily, cached, or on-demand.
**Tags are `string[][]`.** Every implementation represents tags as arrays of
string arrays. None impose a richer schema at the Event level.
**Plain data over classes.** TypeScript libraries mostly use plain objects
(nostr-tools, nostr-gadgets, welshman, applesauce). NDK is the outlier with a
class hierarchy. rust-nostr uses immutable structs with public fields.
**Divergence: mutability.** NDK mutates freely; nostr-tools mutates templates in
`finalizeEvent`; welshman and applesauce build new objects. rust-nostr is strictly
immutable after construction.
**Divergence: builder vs function composition.** rust-nostr and applesauce have
fluent builders. nostr-tools, welshman, and nostr-gadgets expose plain functions.
## Considerations for Our Implementation
**Use serde.** `serde` + `serde_json` is the obvious choice for a Rust
implementation. The canonical form `[0, pubkey, created_at, kind, tags, content]`
can be produced via `serde_json::json!` or a custom `Serialize` impl.
**Event struct.** A plain struct with public fields matches rust-nostr and the
philosophy of "events are data." Fields: `id`, `pubkey`, `created_at`, `kind`,
`tags`, `content`, `sig`.
**Field types.**
- `id: [u8; 32]` or a `EventId` newtype (rust-nostr uses newtype; keeps the API
clean and enforces length). Start simple — `String` of hex or `[u8; 32]`.
- `pubkey`: reuse the `PublicKey` already defined in the keys chapter.
- `created_at: u64` (Unix timestamp seconds).
- `kind: u16` (nostr kinds fit in 16 bits).
- `tags: Vec<Vec<String>>` — simple and correct.
- `content: String`.
- `sig`: a `Signature` type or `[u8; 64]` or hex string.
**Unsigned vs signed.** Two approaches:
1. Separate `UnsignedEvent` struct (no `id`, no `sig`), with a method to compute
the id and convert into a signed `Event`. Clean but verbose.
2. Single struct with `Option<String>` for id/sig. Mirrors welshman's approach but
loses compile-time guarantees.
Given the teaching context, option 1 is clearer: it makes the lifecycle visible
in the type system.
**Canonical serialization via serde_json::json!.** rust-nostr's approach is
compact and readable:
```rust
let canonical = json!([0, pubkey, created_at, kind, tags, content]).to_string();
let id = sha256(canonical.as_bytes());
```
**ID computation as a method on UnsignedEvent.** `fn id(&self) -> [u8; 32]`.
**Signing primitive on PrivateKey.** Per user's guidance: add
`PrivateKey::sign(&self, message: &[u8]) -> Signature` (or `&str`), signing an
arbitrary byte string via Schnorr. The chapter shows an example of combining
`unsigned.id()` with `private_key.sign(&id)` to produce a complete `Event`, but
this orchestration is illustrative prose, not tangled source.
**Verification.** `Event::verify_id()` recomputes and compares; `Event::verify()`
also verifies the Schnorr signature via the existing `PublicKey`-based primitive
(or we add `PublicKey::verify(message, signature)` as a companion to
`PrivateKey::sign`).
**Dependencies already in tree.** From chapter 02 and 03, `secp256k1` (or
`k256`), `sha2`, and `hex` crates should already be available. Need to confirm
and reuse. Add `serde`, `serde_json` if not yet in `coracle-lib`.
**Structural validation.** Keep minimal: lengths of id and sig, format of pubkey
(already enforced if `PublicKey` is a typed wrapper), non-negative timestamp
(enforced by `u64`).
**Kinds chapter is separate.** Don't enumerate kinds; just expose `u16`. A later
chapter adds an enum or constants.
**Tags chapter is separate.** Don't build tag helpers; just `Vec<Vec<String>>`.
+182
View File
@@ -0,0 +1,182 @@
# Research: Expiring Events
## Topic Summary
NIP-40 defines an optional `expiration` tag of the form
`["expiration", "<unix_timestamp_seconds>"]`. When present, it marks the
event as no longer valid after the given time. Relays SHOULD stop
serving expired events and MAY drop them from storage. Clients SHOULD
not display expired events. Expiration is a hint, not a cryptographic
guarantee — anyone who already has a copy can keep it.
This chapter should cover:
- Reading the `expiration` tag from an event's tags
- A predicate `is_expired` (and a testable `is_expired_at(now)` variant)
- Constructing an expiration tag
- Attaching free-function accessors to `HashedEvent` and `Event`
- Handling missing and malformed tags (treated as never-expires)
Mining, storage filtering, and relay-side enforcement are out of scope —
the chapter stays at the tag-parsing layer.
## Philosophy
From `ref/building-nostr`:
- The `expiration` tag is a **behavior tag** — orthogonal to kind,
applicable to any event. It requests special handling by relays
rather than encoding meaning in the payload.
- Expiration is a *superset* of ephemeral-kind (2XXXX) functionality.
Putting retention policy in a tag frees it from the kind range.
- Expiration, like deletion (kind 5), is a **request, not a guarantee**:
"once it's published, there's no way to un-sign or un-publish".
Clients and relays cooperate voluntarily.
- Different content has different natural lifespans. Expiration is a
design choice reflecting use case — chat messages can expire; wiki
entries shouldn't.
The framing matters for the chapter: we are not building a deletion
mechanism. We are parsing an opt-in hint that downstream layers
(storage, UI, relay policy) may act on.
## Reference Implementation Analysis
### applesauce
- `getExpirationTimestamp(event)` reads the first `expiration` tag and
parses its value as a number; caches the parsed value via a Symbol.
- `isExpired(event)` compares the cached timestamp to `unixNow()`
(`Math.round(Date.now() / 1000)`). Missing tag → returns `false`.
- `setExpirationTimestamp(n)` is an event-builder operation that
replaces any existing expiration tag with `["expiration", n.toString()]`.
- `ExpirationManager` is an injectable storage-layer component that
tracks non-expired events and emits an `expired$` observable; an
`EventStore` filters expired events out of queries unless
`keepExpired: true`.
- Time is wall-clock only (`Date.now()`).
### ndk
- Minimal. Expiration is handled at the model level (e.g., a
`P2POrderEvent` class has `expiration` getter/setter); no
framework-wide `isExpired()` method.
- No storage-layer filtering of expired events.
### nostr-gadgets
- Nothing. Zero references to `expiration` or NIP-40.
### nostrlib (Go)
- `nip40.GetExpiration(tags) Timestamp` returns -1 on missing/malformed
tag. Plain int64 parse.
- `khatru/expiration.go` is a separate opt-in relay-side manager:
min-heap of expiring events, periodic sweep (default 1 hour), deletes
on expiration. Not used at query time — strictly a background cleaner.
- Uses `nostr.Now()` = `time.Now().Unix()`; no injection.
### nostr-tools
- `getExpiration(event)``Date` object (from unix seconds × 1000).
- `isEventExpired(event)` compares via `Date.now()`; missing tag → false.
- Bonus: `waitForExpire(event)` / `onExpire(event, cb)` schedule a
`setTimeout` to fire when the event expires — async sugar, not
essential to the core API.
- No tag constructor.
### rust-nostr
The most structured implementation:
- Tag modeled as `TagStandard::Expiration(Timestamp)` variant and
`TagKind::Expiration` kind.
- Constructor: `Tag::expiration(timestamp: Timestamp) -> Tag`.
- Query: `Tags::expiration() -> Option<&Timestamp>`.
- Predicates on `Event`:
- `is_expired_at(&self, now: &Timestamp) -> bool` — compares strictly
with `<` (expiration equal to now is not yet expired).
- `is_expired(&self) -> bool` (std only) — delegates to `now()`.
- `Timestamp` is a `u64` newtype in seconds. `Timestamp::now()` uses
`SystemTime` with an unwrap-to-default fallback; no clock trait.
- Storage feature flag `event_expiration: bool` in the database trait;
currently all backends set it to `false`, so filtering is
application-level.
### welshman
- `Repository` maintains a separate `expired: Map<eventId, timestamp>`
populated at insert time (parses `expiration` tag, skips NaN).
- `isExpired(event)` looks up in the map; strict `<` comparison.
- No builder for the tag; no automatic filtering in query paths.
## Common Patterns
- **Graceful absence.** Every implementation treats a missing
expiration tag as "never expires". None returns an error on absent
tags; most surface `Option` / `null` / a sentinel.
- **Graceful malformed.** Most implementations silently ignore tags
whose value doesn't parse. None treats a malformed tag as expired.
- **Strict `<` comparison.** rust-nostr, welshman, and applesauce all
use strict-less-than: an event whose expiration equals `now` is still
valid. This is the natural reading of "expires at time T" (the event
is invalid *after* T).
- **Wall-clock time.** No implementation injects a clock. Time comes
from `SystemTime::now()` / `Date.now()` / `time.Now()`. Testable
variants (passing `now` explicitly) are common.
- **Separation of parse from enforcement.** Parsing the tag is cheap
and always available. Doing something about it (removing expired
events from storage, hiding them in UI, sweeping a relay database)
is a separate concern layered on top.
- **Minimal tag constructor.** Rust-nostr has one; most TS libraries
expect the caller to build the tag by hand. Adding one is trivial
and makes call sites readable.
## Considerations for Our Implementation
**Scope.** This crate is `coracle-lib`, which owns nostr types and
stateless utilities. The chapter stays at that layer: tag accessor,
predicate, tag constructor. Storage sweeping and UI hiding belong in
downstream crates (`coracle-storage`, future application code).
**API shape, closely tracking rust-nostr but flatter.** The library has
deliberately avoided a `TagStandard` enum in earlier chapters — `Tag`
is a thin `Vec<String>` wrapper with accessor methods. Following that
style:
- `get_expiration(tags: &Tags) -> Option<u64>` — free function.
- `is_expired_at(tags: &Tags, now: u64) -> bool`.
- `is_expired(tags: &Tags) -> bool` — uses `SystemTime::now()`.
- Methods `get_expiration / is_expired / is_expired_at` on both
`HashedEvent` and `Event`, delegating to the free functions.
- A tag constructor. Two plausible shapes:
- `Tag::expiration(ts: u64) -> Tag` — parallels `Tag::new` but
specialized.
- Just document that callers can write `Tag::new("expiration",
[ts.to_string()])`.
The existing `pow.rs` module builds the nonce tag inline via
`Tag::new`, so there is a precedent for the inline form. But an
expiration-specific constructor is small and keeps call sites more
readable, and there's nothing nonstandard to hide. Lean toward
providing it.
**Type for timestamps.** The rest of the library uses plain `u64` for
`created_at`. Consistency argues for `u64` here too; no new
`Timestamp` newtype.
**Comparison semantics.** Use strict `<`. Document it in the rustdoc.
**Graceful parsing.** Missing tag or unparseable value → `None` /
`false`. No errors surfaced to callers.
**Time source.** Use `SystemTime::now().duration_since(UNIX_EPOCH)`
directly in `is_expired`. Keep `is_expired_at` available as the
testable primitive. No clock trait — it would cost more than it buys
in a library this size.
**Dependencies.** None beyond what's already pulled in (`std::time`).
**Narrative.** The chapter should build on the proof-of-work chapter's
structure — free functions plus methods on event types — and lean on
the philosophy framing: expiration is a **behavior tag**, a request,
not a cryptographic property of the event. That framing prepares the
reader for later chapters where storage and relay layers act on it.
+241
View File
@@ -0,0 +1,241 @@
# Research: Filters
## Topic Summary
Filters are the NIP-01 data structure for matching events. They form an elegant primitive for
matching events independent of the client/relay context — not just for REQ messages, but as a
general-purpose event matching and querying abstraction. The chapter should cover the filter
structure, matching semantics (AND within a filter, OR across filters), tag filters, timestamp
constraints, limits, and programmatic construction/manipulation of filters.
## Philosophy
From `ref/building-nostr`:
**Filters as a conceptual category for tags**: The author identifies "filter tags" as one of
three tag categories (alongside data and behavior tags). Filter tags are "especially useful for
filtering and retrieval" and tend to be single-letter because relays only index single-letter
tags to reduce database overhead.
**Filter structure (NIP-01)**: Standard fields are `ids`, `authors`, `kinds`, `since`, `until`,
`limit`. Tag filters are created by prefixing tag names with `#` (e.g., `#p`, `#e`). Extensions
include prefix matching and NIP-50 `search`. Negative matches were proposed but rejected due to
relay performance concerns.
**Filters as a routing problem**: "Where to send a given filter" is distinct from "where to send
a given event." Filters have less information than events, making routing harder. A routing
heuristic "connects a filter that might be constructed to support a particular use case with the
relay where matching events are stored." This is analogous to database indexes.
**Design principles**:
- **Minimalism**: Filters match discrete criteria without complex negation or boolean logic
- **Decentralization**: Clients understand routing heuristics; relays don't need to understand intent
- **Extensibility**: The `#tag` convention allows arbitrary new filters without protocol changes
- **Trade-offs**: Single-letter indexing limits expressiveness but maintains relay scalability
- **Partition tolerance**: Missing a relay means missing some events, which is acceptable
## Reference Implementation Analysis
### applesauce
**Types**: `Filter` extends nostr-tools' `CoreFilter` with NIP-91 AND operator support (`&`-prefixed
tag names) and NIP-50 `search` field. All standard NIP-01 fields present.
**Matching**: `matchFilter(filter, event)` checks basic fields first for early rejection, then
uses `getIndexableTags(event)` to build a cached `Set<string>` of `"tagName:value"` pairs on the
event (via Symbol key). NIP-91 AND tags processed before OR tags. `matchFilters` implements OR
across filter arrays.
**Utilities**: `mergeFilters(...filters)` unions array fields and deduplicates; takes minimum
`limit`, minimum `since`, maximum `until`. `isFilterEqual` uses `fast-deep-equal` for subscription
deduplication. `createFilterMap` distributes filters across relays by author.
**SQL layer**: Separate `buildFilterConditions` translates filters to SQL WHERE clauses. AND tags
use `GROUP BY`/`HAVING COUNT` subqueries. OR tags use `IN` subqueries.
**Patterns**: Symbol-based memoization of tag indexes on event objects. Early exit optimization.
Only indexes single-letter tags. RxJS integration for streaming filter results.
### ndk
**Types**: `NDKFilter<K extends NDKKind>` — generic parameterized type with all NIP-01 fields plus
`search`. Dynamic tag properties via `[key: #${string}]`.
**Matching**: `matchFilter(filter, event)` uses `indexOf()` for array membership. Special case:
`#t` (hashtag) tags are case-insensitive. Does NOT check `limit` or `search` — these are
submission constraints, not matching criteria. Short-circuits on first mismatch.
**Utilities**:
- `mergeFilters()` — unions arrays, deduplicates via Set. Preserves filters with `limit` separately (limits can't be merged). Returns array.
- `filterFingerprint()` — deterministic hash of filter structure for subscription grouping
- `compareFilter()` — checks if one filter is a subset of another (for cache-hit validation)
- `filterFromId()` — converts bech32 identifiers to filters
- `filterForEventsTaggingId()` — creates tag filters for events referencing a given ID
**Validation**: Three modes (VALIDATE/FIX/IGNORE). Checks for undefined values, type correctness,
hex format, kind range. Guardrails catch common mistakes: empty filters, bech32 in hex arrays,
`since > until`, `#t` with literal `#` prefix.
**Patterns**: Generic type parameterization. Pluggable validation modes. Readable subscription IDs
generated from filter structure.
### nostr-gadgets
Uses `nostr-tools` filter types and functions directly — no custom filter implementation.
**Construction patterns**: Mutable accumulation (`filter.authors?.push(target)`), inline literals,
spread-based composition (`{ ...f, authors: [pubkey], since: newest }`).
**Filter-based deletion**: Converts event tags to filter arrays for batch deletion operations.
**Multi-level filtering**: Filters used at query construction, relay permission checking (purgatory),
and client-side event matching.
### nostrlib
**Types**: Go struct with `IDs []ID`, `Kinds []Kind`, `Authors []PubKey`, `Tags TagMap`,
`Since Timestamp`, `Until Timestamp`, `Limit int`, `Search string`, `LimitZero bool`. Uses
fixed-size byte arrays for IDs/PubKeys.
**Matching**: Two methods:
- `Matches(event)` — full matching including timestamp constraints
- `MatchesIgnoringTimestampConstraints(event)``[//go:inline]` optimized, used for live events after EOSE
Tag matching via `tags.ContainsAny(tagName, values)`. Uses `slices.Contains()` for array membership.
**Utilities**: `Clone()` deep copies. `FilterEqual()` order-independent comparison. `GetTheoreticalLimit()`
estimates max results considering replaceability. No merging functions.
**Serialization**: Custom easyjson codec with `xhex` for fast hex encoding. `LimitZero` bool
distinguishes `"limit": 0` from omitted limit.
**Patterns**: Pure data structure with stateless methods. Subscription switches matching function
after EOSE. Query optimizer scores tags by "goodness" for index selection.
### nostr-tools
**Types**: Simple TypeScript type with all NIP-01 fields. Index signature `[key: #${string}]` for
dynamic tag filters. All properties optional.
**Matching**: `matchFilter(filter, event)` — conjunctive (AND) matching. Uses `indexOf()` for
membership. Iterates filter properties for `#`-prefixed tag filters. Both `since` and `until` are
inclusive. `matchFilters` implements OR across array.
**Utilities**:
- `mergeFilters(...filters)` — unions array properties, takes max `limit`, min `since`, max `until`
- `getFilterLimit(filter)` — computes intrinsic limit considering replaceability:
- Empty arrays → 0
- IDs → `ids.length`
- Replaceable kinds → `authors.length * kinds.length`
- Addressable kinds → `authors.length * kinds.length * #d.length`
- Returns minimum across all applicable constraints
**Patterns**: Minimalist, functional, no external dependencies for filter logic. Pure JavaScript.
Early exit on mismatch. Kind classification integration for limit calculation.
**Design**: Self-contained. No validation. `search` field defined but not used in matching logic.
### rust-nostr
**Types**: `Filter` struct with `Option<BTreeSet<T>>` for all set fields. Uses `BTreeSet` for
O(log n) lookups and deterministic serialization. `generic_tags: BTreeMap<SingleLetterTag, BTreeSet<String>>`
for dynamic tag filters.
**Matching**: `match_event(&self, event, opts)` with `MatchEventOptions` controlling which fields
to check (7 boolean flags). Individual match methods are `#[inline]`. Tag matching uses lazy-initialized
`event.tags.indexes()` (OnceCell pattern). NIP-50 search: case-insensitive substring via `.windows()`.
**Builder pattern**: Fluent chainable methods consuming `self`: `Filter::new().kind(k).author(pk)`.
Convenience methods for common tags: `.event()`, `.pubkey()`, `.hashtag()`, `.identifier()`,
`.coordinate()`. Generic `.custom_tag()` for arbitrary tags. Remove methods return `None` if set
becomes empty.
**Option semantics**: `None` = no constraint (matches all). `Some(empty_set)` = matches nothing.
This distinction is explicitly documented (GitHub issue #302).
**Utilities**: `is_empty()`, `extract_public_keys()`. No merging/combining API — multiple filters
handled at protocol layer.
**Patterns**: no_std compatible (uses `alloc`). BTreeSet for deterministic ordering. Custom serde
with `#[serde(flatten)]` for generic tags.
### welshman
**Types**: Standard NIP-01 filter type. `neverFilter = {ids: []}` constant for "matches nothing."
**Matching**: Delegates to nostr-tools for NIP-01 matching. Extends with search: splits by
whitespace, case-insensitive, requires ALL terms match (AND logic).
**Utilities**:
- `getFilterId(filter)` — deterministic hash for deduplication (sort keys, join, hash)
- `calculateFilterGroup(filter)` — groups by matching space (structural fields vs temporal)
- `unionFilters(filters)` — groups by `calculateFilterGroup`, merges arrays within groups
- `intersectFilters(groups)` — Cartesian product across filter groups with intelligent merging (max `since`, min `until`, max `limit`, concatenate `search`)
- `getIdFilters(idsOrAddresses)` — converts mix of IDs and addresses to filters
- `getReplyFilters(events)` — generates filters for replies (#e for regular, #a for replaceable)
- `addRepostFilters(filters)` — adds repost kind variants
- `trimFilter(filter)` — caps array fields at 1000 items with random sampling
- `getFilterGenerality()` — heuristic score 0 (specific) to 1 (general)
**Patterns**: Functional composition. Immutable transformations. Hash-based deduplication.
Domain-driven builders for common query patterns.
## Common Patterns
1. **Type structure**: All implementations use optional fields. Missing = no constraint. The `#tag`
convention for dynamic tag filters is universal.
2. **AND/OR semantics**: Universal agreement — AND within a single filter, OR across an array of
filters. This is fundamental to NIP-01.
3. **Matching order**: Most implementations check scalar fields first (ids, kinds, authors) for
early exit before the more expensive tag matching.
4. **Tag indexing**: Several implementations build cached indexes on events for efficient repeated
matching (applesauce: Symbol-based Set cache; rust-nostr: OnceCell BTreeMap).
5. **No negation**: No implementation supports negative matching (NOT). This aligns with protocol
design — rejected for relay performance reasons.
6. **Limit semantics**: `limit` is not a matching criterion — it's a result count constraint.
Most matching functions ignore it. `getFilterLimit`/`GetTheoreticalLimit` computes intrinsic
upper bounds based on kind replaceability.
7. **Merging**: Most implementations provide union-style filter merging. Array fields are unioned
and deduplicated. Scalar fields use min/max logic.
8. **Option vs empty**: rust-nostr explicitly distinguishes `None` (no constraint) from
`Some(empty)` (matches nothing). Other implementations handle this implicitly.
## Considerations for Our Implementation
1. **Filter as a standalone primitive**: Frame filters independent of REQ messages. They're a
general-purpose matching predicate over events.
2. **Struct design**: Use `Option<BTreeSet<T>>` following rust-nostr's approach — it correctly
models the distinction between "no constraint" and "empty constraint." BTreeSet gives
deterministic serialization and O(log n) lookups.
3. **Matching function**: Implement `matches(&self, event: &Event) -> bool` with early exit on
scalar fields. Tag matching should use event tag indexes.
4. **Builder pattern**: Fluent API for construction: `Filter::new().kind(1).author(pk)`.
Convenience methods for common tags (#e, #p, #d, #a, #t).
5. **Generic tag filters**: Support arbitrary single-letter tag filters via
`BTreeMap<SingleLetterTag, BTreeSet<String>>` or similar.
6. **Serialization**: Custom JSON serialization to flatten generic tags as `#tag` keys. Handle
`limit: 0` vs omitted limit.
7. **No merging in core**: Following rust-nostr, keep the filter primitive simple. Merging and
combining can live in higher-level utilities if needed.
8. **Limit calculation**: Consider `getFilterLimit`-style intrinsic limit computation based on
kind replaceability — useful for query optimization.
9. **Dependencies**: Filter should depend only on existing types (Event, EventId, Pubkey, Kind,
Timestamp, Tags). Self-contained within coracle-lib.
10. **Test strategy**: Test matching logic thoroughly — all field types, AND semantics, tag
matching, timestamp boundaries, empty sets vs None, edge cases.
+256
View File
@@ -0,0 +1,256 @@
# Research: Kinds
## Topic Summary
The kinds chapter should explain what kinds are without going into detail on any kind in
particular (that's saved for domain chapters). Instead, the chapter should create some kind
of factory/registry/enum that can be used to add behavior to a given kind — validation,
event template factories, interfaces for decryption, methods for converting an event of a
particular kind into a domain-specific struct, etc. Applesauce's factory pattern is preferred
over welshman's ad-hoc utilities approach.
## Philosophy
From building-nostr, several key principles inform this chapter:
**Numbers Over Names:** Event kinds are 16-bit integers. Using numbers instead of names is
deliberate — in a distributed system built on signed data, names can't change. Numbers are
meaningless, which allows subjective interpretations while keeping meaning intact. This frees
implementers to assign their own terms based on understanding.
**Kind Ranges Encode Behavior:** The protocol defines ranges with storage/replacement semantics:
- Regular (1-9999, with exceptions): standard events stored by relays
- Replaceable (0, 3, 10000-19999): only latest per (pubkey, kind) kept
- Ephemeral (20000-29999): not stored, just broadcast
- Addressable (30000-39999): latest per (pubkey, kind, d-tag) kept
The author criticizes this design as coupling kind numbers with behavior — these policies are
orthogonal to content type and "should have belonged in behavior tags."
**More Kinds, Less Ambiguity:** Prefer creating specific event kinds over overloading existing
ones. "Overloading a single kind for multiple purposes is always a recipe for disaster."
**Implementation-First:** Nostr takes an implementation-first approach. It's only through
implementation that what is actually needed can be understood.
## Reference Implementation Analysis
### applesauce
The most sophisticated factory pattern found. Uses a layered architecture:
**EventFactory base class** extends `Promise<T>` for fluent chaining. Core capabilities:
- `EventFactory.fromKind<K>(kind)` — create from kind number
- `EventFactory.fromEvent<K>(event)` — create from existing event (modify)
- `.content()`, `.created()`, `.meta()` — base operations
- `.modifyPublicTags()`, `.modifyHiddenTags()` — tag manipulation
- `.encryptedContent()` — NIP-44 encryption
- `.as(signer)` — set signer, propagated through chain
- `.stamp()`, `.sign()` — finalize event
**Typed Factory subclasses** per kind:
```typescript
export class NoteFactory extends EventFactory<kinds.ShortTextNote, NoteTemplate> {
static create(content?: string): NoteFactory
static reply(parent: NostrEvent, content?: string): NoteFactory
text(content, options?)
mention(pubkey)
subject(subject)
addHashtag(hashtag)
}
```
**Dual system for create vs consume:**
- **Factories** — for creating/modifying events (builder pattern)
- **Casts** (`EventCast<KnownEvent<KIND>>`) — for consuming events, providing typed
properties, validation, and observable streams for relationships
**List factory hierarchy** for NIP-51:
- `ListFactory``NIP51RelayListFactory` / `NIP51UserListFactory` / `NIP51ItemListFactory`
- Concrete: `BookmarkSetFactory extends NIP51ItemListFactory`
**Validation** via type guard functions:
```typescript
function isValidComment(event): event is CommentEvent {
return event.kind === COMMENT_KIND && hasRoot(event) && hasReply(event);
}
```
**No centralized registry** — factories distributed across modules, aggregated via index.ts exports.
### ndk
Uses class-based kind wrappers with inheritance:
**NDKKind enum** with 200+ named variants. Each kind gets a dedicated wrapper class extending
`NDKEvent`:
```typescript
export class NDKArticle extends NDKEvent {
static kind = NDKKind.Article;
static kinds = [NDKKind.Article]; // supports multi-kind classes
static from(event: NDKEvent): NDKArticle
get title(): string | undefined
set title(title: string | undefined)
}
```
**Multi-kind classes:** `NDKVideo` handles kinds 21, 34235, 34236, 22. `NDKRepost` handles 6
and 16.
**40+ kind wrapper files** in `core/src/events/kinds/`.
Kind-aware behavior methods on base `NDKEvent`: `isReplaceable()`, `isEphemeral()`,
`isParamReplaceable()`.
### nostr-gadgets
Functional/storage-oriented approach:
**Higher-order factory functions:**
```typescript
export const loadFollowsList = makeListFetcher<string>(3, RELAYS, itemsFromTags(...))
export const loadFollowSets = makeSetFetcher<string>(30000, itemsFromTags(...))
```
**Generic fetcher factories** take kind number + processor function, return typed fetchers.
Storage interface returns different shapes based on kind classification (addressable vs
replaceable).
### nostrlib
Go library with multi-layered kind awareness:
**`Kind` type** is `uint16` with categorization methods: `IsRegular()`, `IsReplaceable()`,
`IsEphemeral()`, `IsAddressable()`.
**Factory map pattern** for domain objects:
```go
var moderationActionFactories = map[nostr.Kind]func(nostr.Event) (Action, error){
nostr.KindSimpleGroupPutUser: func(evt nostr.Event) (Action, error) { ... },
}
func PrepareModerationAction(evt nostr.Event) (Action, error) {
factory, ok := moderationActionFactories[evt.Kind]
if !ok { return nil, fmt.Errorf("unsupported kind %d", evt.Kind) }
return factory(evt)
}
```
**Schema-based validation** from YAML specs per kind (content format, required tags, d-tag rules).
**Generic list/set patterns** with `fetchGenericList[V, I](kind, index, parseTag, cache)`.
**Dataloader registries** — index-based arrays for 18 replaceable and 4 addressable kinds.
### nostr-tools
Foundational kind system:
**Const + typeof pattern:**
```typescript
export const ShortTextNote = 1;
export type ShortTextNote = typeof ShortTextNote;
```
**Classification functions:** `classifyKind()`, `isRegularKind()`, `isReplaceableKind()`,
`isEphemeralKind()`, `isAddressableKind()`.
**Type guard:** `isKind<T>(event, kind): event is NostrEvent & { kind: T }`.
65+ kind constants. No factory pattern — this is the lowest-level library.
### rust-nostr
Most comprehensive Rust implementation:
**Macro-generated `Kind` enum** with 130+ named variants plus `Custom(u16)` fallback:
```rust
pub enum Kind { Metadata, TextNote, ContactList, ..., Custom(u16) }
```
**Kind range constants:**
```rust
pub const REPLACEABLE_RANGE: Range<u16> = 10_000..20_000;
pub const EPHEMERAL_RANGE: Range<u16> = 20_000..30_000;
pub const ADDRESSABLE_RANGE: Range<u16> = 30_000..40_000;
```
**Behavior methods:** `is_regular()`, `is_replaceable()`, `is_ephemeral()`, `is_addressable()`.
**EventBuilder** with 140+ kind-specific factory methods:
```rust
pub fn text_note(content) -> Self
pub fn metadata(metadata) -> Self
pub fn job_request(kind) -> Result<Self, Error> // validates kind range
```
**Coordinate validation** enforces kind constraints (replaceable vs addressable, d-tag rules).
**No central registry** — behavior statically encoded in kind ranges and builder methods.
### welshman
Ad-hoc utilities approach:
**Kind constants** in centralized `Kinds.ts` (60+ constants).
**Classification predicates:** `isRegularKind()`, `isPlainReplaceableKind()`,
`isParameterizedReplaceableKind()`, `isEphemeralKind()`, `isDVMKind()`.
**Domain types per kind** with read/create/edit functions:
- `readProfile(event) -> PublishedProfile`
- `createProfile(profile) -> EventTemplate`
- `editProfile(profile) -> EventTemplate`
- `readRoomMeta(event) -> PublishedRoomMeta` (validates kind in reader)
- `readList(event) -> PublishedList` (generic across list kinds)
**Store integration:** `deriveItemsByKey({ eventToItem: readProfile, filters: [{kinds: [PROFILE]}] })`
**Encryptable wrapper** for events with encrypted content (lists with private tags).
Pattern: each kind gets constants + types + read/create/edit functions + store wiring. Scattered
across files rather than unified in a registry.
## Common Patterns
1. **Kind as integer wrapper** — all implementations use u16/number with named constants
2. **Range-based classification** — universal: regular/replaceable/ephemeral/addressable predicates
3. **Factory methods for creation** — builder/factory pattern to construct events of specific kinds
4. **Parser functions for consumption** — event → domain struct conversion
5. **Validation at boundaries** — kind checks when parsing events, kind range validation for coordinates
6. **No centralized registry** — most implementations distribute behavior across modules rather
than maintaining a central kind→behavior map
**Divergences:**
- applesauce uses class hierarchy with fluent builder; welshman uses standalone functions
- rust-nostr uses macro-generated enum; Go uses typed constants
- NDK couples kind behavior to event subclasses; others keep them separate
- Only nostrlib uses external schema-based validation
## Considerations for Our Implementation
1. **Trait-based factory pattern** — Rust traits map well to the factory/cast dual pattern from
applesauce. A `KindHandler` trait could define validation, parsing, and creation methods.
2. **Kind enum with Custom fallback** — Follow rust-nostr's `Kind` enum pattern with named
variants + `Custom(u16)`. This gives compile-time safety for known kinds while remaining
extensible.
3. **Range classification as methods**`Kind::is_replaceable()` etc. are universally needed
and straightforward to implement.
4. **Separate create vs parse** — The applesauce pattern of factories (create) vs casts (parse)
maps to Rust traits: one trait for building `EventTemplate` from domain data, another for
extracting domain data from `Event`.
5. **Registry for extensibility** — A `HashMap<Kind, Box<dyn KindHandler>>` or similar registry
would let domain chapters register their handlers without modifying the kinds chapter.
Alternatively, use static dispatch with an enum if the set of kinds is known at compile time.
6. **Literate programming flow** — The chapter should introduce kinds conceptually, define the
`Kind` type, implement classification methods, then build the trait/registry infrastructure
that later chapters will use. Keep actual kind implementations (profiles, lists, etc.) for
domain chapters.
7. **Dependencies** — This chapter depends on the events chapter (for `Event`, `EventTemplate`
types). It should not depend on any domain-specific types.
+195
View File
@@ -0,0 +1,195 @@
# Research: Proof of Work
## Topic Summary
Full implementation of NIP-13 proof-of-work: validation/querying and generation. The mining function receives an `OwnedEvent` with a target difficulty and hashes until the difficulty is achieved, returning a `HashedEvent`. The design should support running in a separate thread (native) or a Web Worker (wasm).
## Philosophy
From `ref/building-nostr`:
- PoW is framed as an **optional, client-level heuristic** — not a protocol requirement. Users can "use web of trust or proof of work heuristics to filter posts."
- Clients should have user-configurable policies for handling adversarial data. PoW is one tool among several (reputation, payment, WoT).
- Economic spam prevention (relay charging) is presented as an alternative to computational PoW.
- NIP-13 is not mentioned by name; the philosophy emphasizes user agency and optional mechanisms over mandated solutions.
## Reference Implementation Analysis
### applesauce
No PoW implementation found. The applesauce libraries focus on client UI utilities and do not include NIP-13 support.
### ndk
No PoW mining or validation in NDK core. NDK does parse `min_pow_difficulty` from NIP-11 relay info documents (`NDKRelayInformation.limitation.min_pow_difficulty`), but does not act on it. PoW mining is delegated to external packages.
### nostr-gadgets
No PoW implementation found. The library focuses on hints scoring, database utilities, event sets, and outbox logic.
### nostrlib
No PoW-specific findings from the Go library search. The library may handle it elsewhere but no dedicated NIP-13 module was found.
### nostr-tools
**File:** `nip13.ts`
Complete standalone NIP-13 implementation:
- **`getPow(hex: string): number`** — Counts leading zero bits from hex string by iterating 8-bit nibbles.
- **`getPowFromBytes(hash: Uint8Array): number`** — Byte-based version using `Math.clz32()` for the partial byte.
- **`minePow(unsigned: UnsignedEvent, difficulty: number)`** — Synchronous mining on main thread:
- Appends `["nonce", count, difficulty]` tag
- Increments nonce counter until SHA-256 hash meets difficulty threshold
- Uses `@noble/hashes` for SHA-256
- Mutates `created_at` when time rolls over
- Returns event with computed `id` field
**Design:** Simple, synchronous, single-threaded. No worker support. Minimal dependencies.
**Test vectors:** Validates `getPow` against known hashes with specific difficulties (073 bits). Tests `minePow` with difficulty=10.
### rust-nostr
**Files:**
- `crates/nostr/src/nips/nip13.rs` — Core PoW utilities
- `crates/nostr/src/event/builder.rs` — Mining loop
- `crates/nostr/src/event/id.rs` — Validation
- `crates/nostr/src/event/tag/standard.rs` — POW tag structure
**Data Structures:**
```rust
TagStandard::POW { nonce: u128, difficulty: u8 }
// Serializes to: ["nonce", "<nonce>", "<difficulty>"]
EventBuilder { pow: Option<u8>, ... }
```
**Core Algorithm — `get_leading_zero_bits<T: AsRef<[u8]>>(h: T) -> u8`:**
- Iterates bytes: full zero byte → +8 bits; non-zero byte → `leading_zeros()` CPU intrinsic; return
- Range 0255
**Prefix Generation — `get_prefixes_for_difficulty(bits: u8) -> Vec<String>`:**
- Converts bit difficulty to valid hex prefixes for filtering/querying
- Formula: `prefix_count = 2^(hex_bits - difficulty_bits)`
**Single-Threaded Mining (`mine_pow_single_thread`):**
- Increments nonce from 0
- For each nonce: push POW tag → compute EventId (SHA-256) → check leading zeros ≥ difficulty → pop tag if fail
- Returns `UnsignedEvent` on success
**Multi-Threaded Mining (`mine_pow_multi_thread`, feature `pow-multi-thread`):**
- Spawns `thread::available_parallelism()` threads
- Each thread: starting nonce = `thread_id`, stride = `num_threads`
- Coordination: `Arc<AtomicBool>` with `Ordering::Relaxed`
- First thread to find solution signals others to stop
- Main thread busy-waits on atomic flag
- Falls back to single-threaded if 1 core or spawn failure
**Validation:**
```rust
impl EventId {
pub fn check_pow(&self, difficulty: u8) -> bool {
nip13::get_leading_zero_bits(self.as_bytes()) >= difficulty
}
}
```
**Relay enforcement:** `nostr-relay-builder` checks `min_pow` before accepting events.
**Design decisions:**
- `nonce: u128` — virtually inexhaustible nonce space
- `difficulty: u8` — matches SHA-256 bit width
- Feature-gated multi-threading keeps `no_std` compatibility
- Busy-wait polling (not ideal but simple)
- Timestamp captured once before mining, shared across threads
### welshman
**File:** `packages/util/src/Pow.ts`
Worker-based asynchronous mining:
- **`makePow(event: OwnedEvent, difficulty: number): ProofOfWork`**
- Returns `{ worker: Worker, result: Promise<HashedEvent> }`
- Creates Web Worker from inline `Blob` URL
- Worker receives event + difficulty via `postMessage`
- Worker runs mining loop using `crypto.subtle.digest("SHA-256", ...)`
- Nonce tag mutated in-place for performance: `tag[1] = count.toString()`
- Supports `start` and `step` parameters for distributing across multiple workers
- **`getPow(event: HashedEvent): number`** — Validates difficulty from event hash. Counts leading zero bytes, then uses `Math.clz32()` on final byte.
- **`estimateWork(difficulty: number)`** — Cost estimation using benchmark: `benchmark_ms * 2^(difficulty - benchmarkDifficulty)`. `benchmarkDifficulty = 15`.
**Integration in `thunk.ts`:**
- PoW applied before signing (necessary because nonce changes the event ID)
- For gift-wrapped events (NIP-59): PoW on the wrapper
- Uses `AbortSignal.timeout(30_000)` for timeout protection
- Logs warning if event already signed (PoW would change ID, invalidating signature)
**Design:** Non-blocking via Workers. Single worker by default but architecture supports multi-worker distribution via start/step. Uses Web Crypto API (available in workers).
## Common Patterns
1. **Difficulty metric:** All implementations use leading zero bits in SHA-256 hash (NIP-13 standard). Not byte-aligned — allows fine-grained difficulty.
2. **Tag format:** Universal `["nonce", "<counter>", "<target_difficulty>"]` tag.
3. **Mining loop:** Increment nonce, recompute hash, check leading zeros. Simple brute force — no shortcuts possible.
4. **PoW before signing:** All implementations compute PoW before the event is signed, since the nonce tag changes the event ID which is what gets signed.
5. **Leading zero counting:** Two approaches:
- Byte iteration + CPU intrinsic (`leading_zeros()` / `Math.clz32()`)
- Hex string nibble iteration (nostr-tools)
6. **Parallelism strategies diverge:**
- rust-nostr: OS threads with atomic bool coordination
- welshman: Web Workers with message passing
- nostr-tools: no parallelism (synchronous)
## Considerations for Our Implementation
### API Design
The chapter should take an `OwnedEvent` (has pubkey but no hash yet) and return a `HashedEvent` (has computed ID meeting difficulty). This fits naturally into the event lifecycle from chapter 05.
The mining function should be **pure and blocking** — it takes inputs and returns a result. Threading/worker dispatch is the caller's responsibility. This keeps the core function platform-agnostic:
- Native: caller wraps in `std::thread::spawn` or `tokio::spawn_blocking`
- WASM: caller runs in a Web Worker
### Validation
Two levels:
1. `get_leading_zero_bits(hash: &[u8]) -> u8` — pure utility
2. `check_pow(event: &HashedEvent, difficulty: u8) -> bool` — checks event's nonce tag difficulty matches actual hash difficulty
### Querying
`get_prefixes_for_difficulty(difficulty: u8) -> Vec<String>` enables relay-side filtering by event ID prefix. Useful for REQ filters.
### Dependencies
- SHA-256 already available from the events chapter
- No additional crates needed for single-threaded PoW
- `u128` nonce type is standard Rust
### Threading Considerations
Rather than building threading into the mining function:
- Keep `mine_pow(event: OwnedEvent, difficulty: u8) -> HashedEvent` as a blocking function
- Document how callers can parallelize (thread per core, strided nonces via start/step parameters)
- For WASM: the function runs inside a worker; the host page dispatches to it
This matches welshman's approach and avoids platform-specific code in the core library.
### Nonce Strategy
Support `start` and `step` parameters to enable callers to distribute work:
```rust
mine_pow(event: OwnedEvent, difficulty: u8, start: u64, step: u64) -> HashedEvent
```
- Single thread: `start=0, step=1`
- N threads: thread i uses `start=i, step=N`
+230
View File
@@ -0,0 +1,230 @@
# Research: Protected Events
## Topic Summary
NIP-70 defines an optional, valueless `["-"]` tag — the "protected" marker.
Its presence asks relays to accept the event *only* when it is published
directly by its author, where "directly by its author" means the publishing
connection has authenticated via NIP-42 and the authenticated pubkey matches
the event's `pubkey`. A relay that receives a protected event over an
unauthenticated connection should issue a NIP-42 `AUTH` challenge and reject
the event for now; a relay that receives one from an authenticated connection
whose pubkey is *not* the author should reject it outright.
Like NIP-40 expiration, this is a **behavior tag**: orthogonal to the event's
kind, it requests cooperation from the network rather than altering what the
event means. It is a hint, not a guarantee — once an event is signed and held
by anyone, no tag can claw it back.
This chapter adds tag-layer support only, mirroring the Expiring Events
chapter: a `Tag::protected()` constructor, an `is_protected(&Tags)` predicate,
and `is_protected()` method facades on `HashedEvent` and `Event`. Relay-side
enforcement (the NIP-42 auth-then-compare logic) is deferred to the Relay
Authentication chapter (ch 26). The unusual property of this tag — that it
carries no value — is the only thing distinguishing the API from expiration:
there is nothing to parse, so there is no `get_*` function, just a boolean.
## Philosophy
`ref/building-nostr` addresses NIP-70 directly and frames it the way this
chapter should.
- **Behavior tags are cooperative requests.** The book classifies tags as
data, filter, or behavior tags, and names NIP-70 explicitly: *"the NIP 70
protected tag `['-']` has no values. The presence of the tag merely indicates
that relays should only accept such event if they're published directly by
their author."* Behavior tags "determine how implementations should handle an
event, independent of the specification which defines the event's kind." This
is exactly the framing already used for expiration — and a reader arriving
from chapter 9 will recognize it.
- **Signed data cannot be unpublished.** *"Once it's published, there's no way
to un-sign or un-publish the data. Anyone who has a copy can keep it."* The
protected tag does not make an event secret; it asks relays to limit who they
will *accept* it from going forward. The chapter must be honest that this is
intent-signaling, not enforcement.
- **Access control is the relay's job, via AUTH.** *"There is one additional
responsibility that relays can't really delegate: access control. Access
control on Nostr is implemented by the `AUTH` verb."* This is why NIP-70's
enforcement story belongs to the networking layer and NIP-42, not to the
stateless tag layer. The library's job here is only to read and write the
marker.
- **Censorship resistance vs. access control.** *"An architecture optimized for
censorship resistance is necessarily not optimized for access controls or
privacy."* Protected events ask relays to *limit* distribution, a deliberate
deviation from nostr's broadcast default — useful framing for why this tag
exists at all.
- **Routing implications.** The book notes that access-controlled content (it
cites NIP-29 groups) should *not* be routed by the usual outbox heuristic,
since that would "leak" content meant to be protected. A brief forward
reference: protected events interact with relay selection, but that is a later
concern.
## Reference Implementation Analysis
### applesauce
Supported, client-side only, in `applesauce-core`:
- `isProtectedEvent(event)``helpers/event.ts:160`:
`return event.tags.some((t) => t[0] === "-");`
- `setProtected(set = true)``operations/event.ts:91`: adds the `["-"]`
singleton tag, or removes it, via `setSingletonTag(["-"])` /
`removeSingletonTag("-")`.
- `protected(isProtected)``factories/event.ts:181`: fluent builder method on
`EventFactory`.
The `["-"]` tag is treated like any other singleton tag (`expiration`, `alt`),
with no special-casing. No relay enforcement, no NIP-42 coupling. Notably,
`isProtectedEvent` is never *called* elsewhere in the library except in tests —
it is a primitive offered to consumers.
### ndk
Supported as a getter/setter on `NDKEvent` (`core/src/events/index.ts:868`):
```typescript
set isProtected(val: boolean) { this.removeTag("-"); if (val) this.tags.push(["-"]); }
get isProtected(): boolean { return this.hasTag("-"); }
```
The one place protection is *consumed* is reposts (`repost.ts:24`): when
building a NIP-18 repost, a protected original is **not** JSON-embedded in the
repost content. No relay-side enforcement; NDK is a client library, so the
relay rule is out of scope by design.
### nostr-gadgets
**Not implemented.** No references to nip70/protected/`["-"]` anywhere. The
library is a high-level client utility layer (profile loading, lists, relay
hints, outbox) that delegates event construction to `@nostr/tools` and never
touches the protected tag.
### nostrlib
The most complete implementation, including relay enforcement. Dedicated
`nip70/nip70.go`:
- `IsProtected(event) bool` — iterates tags, returns true if any tag is exactly
`["-"]`.
- `HasEmbeddedProtected(event) bool` — for reposts (kind 6/16), detects an
embedded `["-"]` in the stringified content via substring search.
Relay enforcement lives in the khatru handler (`khatru/handlers.go:192`):
```go
if nip70.IsProtected(env.Event) {
authed, is := GetAuthed(ctx)
if !is {
RequestAuth(ctx) // NIP-42 challenge
// OK:false "auth-required: must be published by authenticated event author"
} else if authed != env.Event.PubKey {
// OK:false "blocked: must be published by event author"
}
} else if nip70.HasEmbeddedProtected(env.Event) {
// OK:false "blocked: can't repost nip70 protected"
}
```
There is also an opt-in policy `OnlyAllowNIP70ProtectedEvents`. No client-side
builder — callers append `["-"]` by hand. The auth-then-compare flow is the
canonical relay logic and is what our ch 26 will implement.
### nostr-tools
Minimal: the only handling is one inline check inside NIP-18 reposts
(`nip18.ts:41`):
```typescript
content: t.content === '' || reposted.tags?.find(tag => tag[0] === '-') ? '' : JSON.stringify(reposted),
```
If a reposted event is protected, the repost content is emptied. No standalone
detector, no builder, no relay rules — consistent with the library's low-level,
minimal philosophy. A single test covers the repost-emptying behavior.
### rust-nostr
First-class and the closest analog to our target design. The `-` tag is a named
enum variant rather than a generic string tag:
- `TagKind::Protected``"-"` (`event/tag/kind.rs`).
- `TagStandard::Protected`, parsed from `["-"]` (`event/tag/standard.rs`).
- `Tag::protected() -> Self` (`event/tag/mod.rs:469`).
- `Tag::is_protected(&self) -> bool` (`mod.rs:529`) —
`matches!(self.as_standardized(), Some(TagStandard::Protected))`.
- `Event::is_protected(&self) -> bool` (`event/mod.rs:263`) —
`self.tags.find_standardized(TagKind::Protected).is_some()`.
Relay enforcement in `nostr-relay-builder` mirrors nostrlib: if protected and
not NIP-42-authenticated, send AUTH; if authed pubkey ≠ event pubkey, reject
with "this event may only be published by its author". A `test_protected_event`
round-trips `{"tags":[["-"]]}` JSON. Clean, minimal, self-contained; no separate
NIP-70 module because the feature is small enough to live in the tag system.
### welshman
**Not implemented.** No nip70/protected/`["-"]` handling anywhere. Welshman has
generic tag utilities (`getTag`, `getTagValue`, specialized `p`/`e`/`a`/`t`
helpers) and a NIP-42 auth event builder (`makeRelayAuth`), but nothing wires
them to a protected-event rule. As the sibling library to this one, its absence
is a small data point that NIP-70 is often skipped — but our chapter adds it.
## Common Patterns
- **The tag is `["-"]`, valueless.** Every implementation that handles it agrees
presence alone is the signal; there is nothing to parse. This is the single
most important fact for our API: a boolean predicate, no `get_*`.
- **Detection is a one-liner.** Universally "does any tag's first element equal
`"-"`" — i.e., `tags.has("-")` in our existing `Tags` vocabulary.
- **Construction is trivial.** A constructor that returns `["-"]`. rust-nostr
gives it a named `Tag::protected()`; applesauce/ndk inline `["-"]`.
- **Method on the event is the ergonomic surface.** rust-nostr, ndk, and
applesauce all expose detection at the event level (`is_protected` /
`isProtected`), which is where callers actually hold the data.
- **Enforcement is relay-side and needs NIP-42.** nostrlib and rust-nostr
implement the identical flow: protected + unauthenticated → AUTH challenge +
reject; protected + authed-but-wrong-pubkey → reject. Client libraries
(applesauce, ndk, nostr-tools, welshman) do *not* enforce; they only mark.
- **The NIP-18 repost interaction recurs.** applesauce, ndk, nostr-tools, and
nostrlib all special-case reposts so a protected event isn't re-broadcast by
being embedded in someone else's repost content. This is a real, named
consequence of the tag worth a forward reference, but it belongs to a reposts
chapter, not here.
- **Two libraries skip it entirely** (nostr-gadgets, welshman), confirming the
feature is small and often deferred — which is why a focused, correct
tag-layer treatment is worth having.
## Considerations for Our Implementation
- **Mirror the expiration chapter's shape.** Free function on `&Tags`, a
`Tag::protected()` constructor in `tags.rs`, method facades on `HashedEvent`
and `Event`, a new `protected.rs` module, a hand-written integration test.
Consistency with ch 9 is a feature: the reader has just seen this exact shape.
- **No `get_*`, no parsing.** The tag has no value. The whole API is a boolean.
`is_protected(&Tags)` is `tags.has("-")` — our `Tags::has` already does the
work. The chapter should acknowledge that the free function is a thin wrapper
and justify it on consistency and discoverability grounds (a named NIP-70
function the reader can find), the same justification used for the expiration
constructor.
- **No `Timestamp`/value type concerns.** Unlike expiration there is no number,
so none of expiration's parsing/`<` discussion applies. The chapter is
shorter.
- **The constructor goes in `tags.rs`,** appended as another `impl Tag` block,
exactly like `Tag::expiration`. Discoverable next to `Tag::new`; no import
from `protected` just to build a tag.
- **Defer enforcement explicitly.** State the relay rule in prose — protected +
authed-author-only — and name where it will be implemented (Relay
Authentication, ch 26, which needs the NIP-42 machinery). Do not write
acceptance logic here; the tag layer is stateless and has no notion of a
connection or an authenticated pubkey.
- **Mention the repost consequence briefly,** as a forward reference, without
implementing it: a protected event should not be embedded in a NIP-18 repost.
- **No dependencies.** Pure std + existing crate types. No new `Cargo.toml`
entries, no workspace changes.
- **The `"-"` tag name is unusual** (a single dash, not a word). Worth one
sentence: it is intentionally terse and visually distinct, and it has no
values, which is why it does not look like other tags.
+307
View File
@@ -0,0 +1,307 @@
# Research: Search
## Topic Summary
NIP-50 adds an optional full-text `search` field to the subscription filter
introduced in chapter 11. A relay that supports the capability interprets the
query string against event content (and, for some kinds, other fields),
returning results ordered by relevance rather than `created_at`. The query may
carry structured extensions in the form of `key:value` pairs — `domain:`,
`language:`, `sentiment:`, `nsfw:`, `include:spam` — which relays may support or
ignore.
The chapter will:
1. Add a `search` field to the existing `Filter` type, wiring it through
construction, serialization, hashing, grouping, and the union/intersect
utilities.
2. Introduce a typed `SearchQuery` model that splits free-text terms from
`key:value` extensions, so applications can build and inspect queries safely
instead of stringly-typed concatenation. (This is a deliberate departure
from every reference, which treats the query as an opaque string.)
3. Implement a best-effort, case-insensitive local matcher over event content,
while documenting that real ranking and extension semantics are
relay-defined.
The code lives in `coracle-lib`: the `search` field extends `filters.rs`, and
the query model gets a dedicated `search.rs` module.
## Philosophy
From `ref/building-nostr`, the framing relevant to search is that **content
discovery on nostr is client-initiated routing through relay selection**, not a
query against a global index. Searching is "knowing where to send queries." A
relay that supports NIP-50 is exercising an *optional, relay-authored
capability* — like content curation or access control — and defines its own
matching semantics, including which extensions it honors. This mirrors the NIP's
own "relays SHOULD ignore extensions they don't support."
Three principles bear directly on the chapter's voice:
- **No guaranteed completeness.** "No implementation will have a complete view
of every heuristic that is applicable" — so search results are neither global
nor exhaustive. A client queries the relays it knows support search and
accepts a partial, spontaneous view. This should be stated honestly, not hidden.
- **Indexing is the curator's responsibility, not the user's.** Authors publish
signed events; relays (or indexing services) that *want* content discoverable
maintain the index. Clients do nothing special beyond sending a `search`
filter to a search-capable relay.
- **Publicity, not privacy.** Full-text indexing makes content patterns
discoverable and gives relay operators visibility into queries. The honest
framing: search is a publicity feature.
The takeaway for our library: model `search` as a first-class but optional
filter field, keep the query structured enough that applications can reason
about it, and be candid that local matching is a best-effort approximation of a
relay-defined operation.
## Reference Implementation Analysis
### applesauce
`search` is an optional string on an extended `Filter` type
(`packages/core/src/helpers/filter.ts`): `Filter = CoreFilter & { search?: string }`,
extending nostr-tools' base type. **Opaque** — no extension parsing.
Dual-mode: relay subscriptions pass the string through verbatim; a local SQLite
backend (`packages/sqlite`) indexes content into an FTS5 table and runs
`events_search MATCH ?` with the raw string double-quote-escaped. Local
client-side `matchFilter()` **ignores** the search field entirely. Pluggable
"search content formatters" decide what gets indexed (default: `content`;
enhanced: kind-0 profile fields plus `t`/`subject`/`title`/`summary`/`d` tags).
Supports `order: "created_at" | "rank"` for FTS5 ranking. Low coupling; SQLite
is optional. No query-extension awareness anywhere.
### ndk
`search?: string` on `NDKFilter` (`core/src/subscription/index.ts:30`).
**Opaque, relay-only.** No parsing, no validation (filter-validation pipeline
skips it), no client-side matching (delegates to nostr-tools' `matchFilters`,
which ignores search). No helper functions for building search filters; callers
construct `{ search: "..." }` by hand. The field is serialized and sent to
relays as-is. No NIP-11 capability negotiation or fallback. Minimal by design.
### nostr-gadgets
Re-uses `@nostr/tools`' `Filter` type (`search?: string`). **Opaque,
relay-only.** Notably its local stores *reject* search: the in-memory store
returns an empty set if `filter.search` is present, and the RedEventStore docs
state "any filters supported (except 'search')." Provides a hardcoded
`SEARCH_RELAYS` constant (`defaults.ts`): `relay.nostr.band`, `nostr.wine`,
`relay.noswhere.com`, `relay.nos.today`. No query builders, no dynamic relay
capability detection.
### nostrlib (Go)
`Search string` on the `Filter` struct (`filter.go`), (de)serialized as a plain
`"search"` JSON key. The core `Filter.Matches` / `MatchesIgnoringTimestampConstraints`
**ignores** search — matching is delegated to eventstore backends. Key-value
backends (BoltDB, LMDB, MMM) return nothing for search queries; only the **Bleve**
backend implements real full-text search: per-document language auto-detection
(lingua-go, 22 languages), per-language analyzers, boolean query syntax
(`AND/OR/NOT`, parens, quoted phrases), NIP-27 reference extraction with 2× boost,
and case-insensitive substring validation of quoted phrases. Kind-0 profiles index
name/display_name/about; reposts unpack inner events. Khatru relay policies
`NoSearchQueries`/`RemoveSearchQueries` let operators disable search. SDK
`SearchUsers()` just sends a `Search` filter to designated user-search relays. No
NIP-50 *extension* parsing (treats `domain:x` as a regular word); a 2-char minimum
query length is enforced by Bleve.
### nostr-tools
`search?: string` on the base `Filter` (`filter.ts`). **The canonical
"defined-but-unused" implementation.** `matchFilter()`/`matchFilters()` do not
check search at all; `mergeFilters()` drops it entirely. No parsing, no
validation, no helpers, no tests for the field. Strictly a transport-layer
placeholder so applications can send search filters to relays. Minimal-deps
philosophy: search is purely a relay concern.
### rust-nostr
The most directly relevant reference (also Rust). In
`crates/nostr/src/filter.rs`:
```rust
/// A string describing a query in a human-readable form, i.e. "best nostr apps"
/// <https://github.com/nostr-protocol/nips/blob/master/50.md>
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(default)]
pub search: Option<String>,
```
Builder API: `search<S: Into<String>>(self, value: S) -> Self` and
`remove_search(self) -> Self` — symmetric, generic, `#[inline]`. **Opaque** (no
extension parsing).
Local matching (`search_match`):
```rust
fn search_match(&self, event: &Event) -> bool {
match &self.search {
Some(query) => event.content.as_bytes()
.windows(query.len())
.any(|window| window.eq_ignore_ascii_case(query.as_bytes())),
None => true,
}
}
```
Case-insensitive **ASCII** substring via sliding window; `None` matches
everything. Gated by a `MatchEventOptions { nip50: bool, .. }` flag (default
true). Notably, the SDK relay sets `.nip50(false)` with the comment "Skip NIP-50
matches since they may create issues and ban non-malicious relays" — i.e.
client-side re-matching of a relay's search results can wrongly drop valid hits.
DB backends (LMDB, SQLite) extend matching to a fixed set of searchable tags —
`title`, `description`, `subject`, `name` — lowercasing the query once up front;
empty search → no results. A `Features { full_text_search: bool }` flag declares
backend capability.
Patterns worth emulating: `Into<String>` builder, `skip_serializing_if` for a
clean wire format, an explicit opt-out for search matching, ASCII case folding
for speed.
### welshman
The TypeScript toolkit our library descends from. `search?: string` on `Filter`
(`packages/util/src/Filters.ts`). It is the **only reference that matches search
locally and threads it through filter utilities**:
```typescript
export const matchFilter = (filter, event) => {
if (!nostrToolsMatchFilter(filter, event)) return false
if (filter.search) {
const content = event.content.toLowerCase()
const terms = filter.search.toLowerCase().split(/\s+/g)
for (const term of terms) {
if (content.includes(term)) return true
return false // <-- bug: returns after first term
}
}
return true
}
```
The intent is term-splitting + case-insensitive substring, but the early
`return false` means only the first term is ever checked. **A correct version
should decide AND vs OR across terms explicitly** — this is the one place we can
clearly improve on the reference.
Filter utilities (directly parallel to our `group`/`union_filters`/`intersect_filters`):
- `calculateFilterGroup` pushes `search:${search}` into the group key — **a
filter with a search is only mergeable with an identical search.**
- `unionFilters` treats `search` (like `since`/`until`/`limit`) as a scalar
preserved from the first filter in the group, **not merged**.
- `intersectFilters` concatenates differing searches with a space
(`[a, b].join(" ")`) — modeling "must match both" as a compound query — and
takes whichever is present otherwise.
- `getFilterId` includes search in the deterministic hash, so different searches
never dedupe.
Search-relay selection lives in the router: `getSearchRelays()` returns relays
whose NIP-11 `supported_nips` includes `"50"`. No extension parsing.
## Common Patterns
- **`search` is universally an optional plain string.** Every reference models
it as `Option<String>` / `search?: string`. None parse the `key:value`
extensions — they treat the whole query as opaque and let the relay interpret
it. Our typed `SearchQuery` is therefore a value-add, not a port.
- **Local matching is the exception, not the rule.** nostr-tools, ndk,
applesauce (in `matchFilter`), and nostrlib's core `Filter` all *ignore*
search locally; matching happens relay-side (or in a dedicated index like
Bleve/FTS5). Only rust-nostr and welshman attempt local matching, both with
case-insensitive substring over `content`.
- **Where matching exists, it's case-insensitive substring** — rust-nostr does
ASCII-only `eq_ignore_ascii_case` over byte windows (whole query as one
needle); welshman lowercases and splits on whitespace into terms (intending
multi-term, buggily). DB backends additionally search a small fixed set of
metadata tags (`title`, `description`, `subject`, `name`).
- **Search makes filters un-mergeable.** Both welshman (group key) and the
general intuition agree: two filters with different search strings can't be
unioned without changing semantics. rust-nostr sidesteps merging at this layer
entirely.
- **Client-side re-matching is risky.** rust-nostr's SDK disables NIP-50
matching when filtering relay results, because a relay's notion of a match
(ranked, fuzzy, multi-field, extension-aware) is richer than a client's
substring check — re-filtering can drop legitimate hits.
- **Relay selection by NIP-11.** Search-capable relays are discovered via
`supported_nips` containing `50` (welshman) or a hardcoded allowlist
(nostr-gadgets). This is an application/networking concern, out of scope for
`coracle-lib`.
## Considerations for Our Implementation
**Filter field.** Add `pub search: Option<String>` to `Filter`. Follow
rust-nostr: `add_search<S: Into<String>>(self, S)` and `clear_search(self)` to
match the existing `add_*`/`clear_*` builder vocabulary (our methods are named
`add_since`/`clear_since`, etc., so `add_search`/`clear_search` fits better than
rust-nostr's `search`/`remove_search`). The field already participates in the
derived `Hash` (so `id()` covers it for free), but serialization, `group()`,
`union_filters`, `intersect_filters`, and `matches()` all need explicit updates.
**Serialization.** Our `Filter` has hand-written serde (to flatten `#tag` keys).
Add `search` as a plain `"search"` key — emit only when `Some` (mirroring
`since`/`until`/`limit`), and read it in the visitor's match arm. A round-trip
test must cover it.
**Grouping / union / intersect.** Per welshman: include `search` in the
`group()` hash so filters with different searches land in different groups (never
merged). In `union_filters`, since group members share an identical search by
construction, the search carries over via the `or_insert_with(|| filter.clone())`
seed — no special merge needed, but worth a comment. In `combine_pair`
(intersect), decide how to combine two searches: welshman concatenates with a
space. Concatenation is defensible ("must match both") but lossy and surprising;
a cleaner rule for a typed model is to **merge two `SearchQuery` values** (union
their terms and extensions) or, if we keep the field as a string at this layer,
to concatenate with a space and document it. Recommend: concatenate with a space
when both present and differ, matching welshman, and note the limitation.
**Local matching.** Extend `Filter::matches` to test `search` *after* the cheap
scalar checks. Best-effort, case-insensitive. Two design choices to settle in
planning:
1. Whole-query substring (rust-nostr) vs. term-split AND/OR (welshman, fixed).
A typed `SearchQuery` makes term-split natural: match the free-text terms
(AND across terms reads as the intuitive "all words present"; document it),
and treat `key:value` extensions as *unenforceable locally* — i.e. ignored by
the local matcher, since we can't evaluate `sentiment:` or `domain:` without
external data. This honesty matches the NIP.
2. ASCII (`eq_ignore_ascii_case`) vs. Unicode lowercasing. ASCII is what
rust-nostr ships and is allocation-free; Unicode `to_lowercase` is more
correct for non-Latin content but allocates. Given nostr's multilingual
content, prefer Unicode `to_lowercase` for the local matcher — correctness
over micro-optimization, consistent with our "clarity over cleverness" rule —
and note the trade-off.
Also document, per rust-nostr's SDK, that local matching is a *fallback*:
relay results should generally be trusted as-is rather than re-filtered.
**`SearchQuery` model (new `search.rs`).** A struct splitting a query into
free-text `terms: Vec<String>` and `extensions: Vec<(String, String)>` (ordered;
NIP-50 doesn't forbid repeats, and order can matter to relays). Parsing: split on
whitespace, treat a token containing `:` (with a non-empty key before it) as an
extension, everything else as a term. Provide:
- `SearchQuery::parse(&str) -> SearchQuery` (total, never fails — unknown shapes
fall back to terms).
- `Display` / `to_string()` that re-renders to the wire string (terms first or
preserve order; planning to decide).
- Builder helpers: `term`, `extension`, plus typed convenience for the
spec-defined extensions (`domain`, `language`, `sentiment`, `nsfw`,
`include_spam`) — optional, decide scope in planning.
- A bridge to `Filter`: `Filter::add_search` can accept `impl Into<String>` so
both a raw string and `query.to_string()` work; optionally
`Filter::search_query()` to parse the field back out.
Keep `sentiment`/`nsfw` values as strings (or small enums) — leaning toward
strings to stay forward-compatible with relay-specific values, with named
constructors for the common cases.
**Dependencies.** None new. Parsing is plain string handling; matching uses std.
Avoid pulling in a real FTS engine — out of scope and against the
minimal-dependency rule.
**Out of scope (defer / mention only).** Real relevance ranking; relay-side
indexing; NIP-11 search-relay discovery (a networking concern); the `order`
hint from applesauce; multi-field/tag matching beyond `content` (could mention
`title`/`subject` as a possible extension but keep the matcher content-only for
clarity).
+248
View File
@@ -0,0 +1,248 @@
# Research: Tags
## Topic Summary
The tags chapter introduces a typed representation of nostr tags to replace
the `Vec<Vec<String>>` used in the events chapter. Tags are arrays of
strings whose first element names the tag and whose subsequent elements
carry values, relay hints, and markers. The chapter should cover:
- A `Tag` wrapper around `Vec<String>` with accessors for name, value, and
the rest of the entries
- Helpers that read and filter tags on an event (`find`, `find_all`,
`values`, `value`, `has`)
- The distinction between indexed single-letter tags and multi-character
tags
- Parsing and constructing address tags (`kind:pubkey:identifier`) and
`EventPointer`/`ProfilePointer`/`AddressPointer` conveniences
- NIP-10 markers on e-tags (`root`, `reply`, `mention`) and how to read
them positionally
- Integration with the `Event`/`EventContent`/etc. types from the events
chapter — swap `Vec<Vec<String>>` for `Vec<Tag>`
We want an ergonomic but minimal type. Not rust-nostr's 60-variant enum; a
thin wrapper plus free functions on slices, close in spirit to nostrlib or
welshman.
## Philosophy
From `ref/building-nostr`:
**Tags are the structured data half of events.** An event's content is
generally human-readable; tags hold structured data. Encoding JSON into
content is an antipattern. Conversely, tags are where every reference,
index, or machine-readable annotation should live.
**Lists of lists, not maps.** Tags are arrays of arrays of strings by
design. This preserves two properties a dictionary cannot: keys may repeat
(important for multiple `e` or `p` references), and order is preserved.
The parallel drawn by building-nostr is to URL query parameters and Python
ordered dicts.
**Keep tags short.** "In general, tags should be as short as is reasonable.
Two to three entries is all you really need; if you have more than that,
you're probably trying to pack more data into a single tag than really
belongs." Prefer multiple tags over positional fields.
**Three categories, conflated.** Building-nostr identifies three
categories of tag that were conflated in the original design: data tags
(for display/handling), filter tags (single-letter, queryable via `#x`),
and behavior tags (like `expiration`, `-`, `h` — affect implementation
handling orthogonally to kind). The conflation is called out as "a design
mistake" but we have to live with it.
**Single-letter = indexed.** Single-letter tag names (`a``z`, `A``Z`)
are the ones relays index and expose via `#e`, `#p`, etc. filters.
Multi-character names (`imeta`, `alt`, `expiration`) are typically not
indexed. The tag-name convention is therefore meaningful: naming a tag
with a single letter asserts it's intended for filtering.
**The `e` tag is overloaded.** Eight different NIPs use `e` for different
things (reply, fork, transaction reference, report target, list member,
approval, merge, mention). Building-nostr warns: when resolving a tag's
meaning, always consult the kind spec first, then tag specs — never the
other way around. Our library should stay neutral about semantics and let
callers interpret based on kind.
**Design general-purpose tags cautiously.** Broad tags can conflict with
kind-specific semantics. Our tag type should not bake in interpretation.
## Reference Implementation Analysis
### applesauce (TypeScript)
- Tags remain `string[][]` throughout; no wrapper class.
- Type-level annotation via `NameValueTag<Name>` generic tuple; runtime
type guards (`isETag`, `isPTag`, ...) identify kinds.
- Markers (`"root" | "reply" | "mention" | ""`) as a union type.
- A-tag parsing lives in `parseReplaceableAddress(address)` returning
`AddressPointer | null`, with an inverse
`getReplaceableAddressFromPointer`.
- Operations-as-functions: `TagOperation = (tags) => tags`. Events expose
`modifyPublicTags(...ops)` that pipes operations.
- Helpers: `addEventPointerTag`, `addProfilePointerTag`,
`addAddressPointerTag`, `ensureSingletonTag`, `ensureNamedValueTag`,
`fillAndTrimTag` (normalizes nulls and trailing blanks).
### ndk (TypeScript)
- `NDKTag = string[]`, raw; `NDKEvent.tags: NDKTag[]`.
- Accessors on event: `getMatchingTags(name, marker?)`, `hasTag`,
`tagValue` (returns index 1 or undefined), plus `removeTag`, `replaceTag`.
- Address tags: `tagAddress()` constructs `${kind}:${pubkey}:${dTag}`;
`tagId()` returns event id or address depending on replaceability;
`tagType()` returns `"e" | "a"`.
- NIP-10 markers at `tag[3]`: `getRootTag`, `getReplyTag` fall back to
positional interpretation when markers are absent.
- `referenceTags(marker?)` emits `[["a", addr], ["e", id, relay, marker, pubkey]]`.
- `generateContentTags` auto-tags `npub`/`note`/`nevent`/`naddr`/hashtags
from content.
### nostr-gadgets (TypeScript, JSR)
- Raw `string[]` tags, documented by convention.
- Single helper: `getTagOr(event, tagName, dflt)`.
- Validators: `isHex32`, `isATag` (regex `^\d+:[0-9a-f]{64}:[^:]+$`).
- Composition pattern: `itemsFromTags<I>(processor)` factory — each
fetcher passes a per-tag processor to build typed items.
- Deletion kind-5: switch on `tag[0]` for `e` (id filter) vs `a`
(kind+author+#d filter).
### nostrlib (Go, fiatjaf)
- `Tag = []string`, `Tags = []Tag`; embedded directly in `Event`.
- Helpers on `Tags`:
- `Find(key)`, `FindLast(key)`
- `FindWithValue(key, value)`, `FindLastWithValue`
- `FindAll(key)` returns `iter.Seq[Tag]` (lazy)
- `Has(key)`, `ContainsAny(key, values)`
- `GetD()` for the `d` identifier on parameterized replaceables
- Pointer interface: `ProfilePointer`, `EventPointer`, `EntityPointer`
all share `AsTag`, `AsTagReference`, `AsFilter`, `MatchesEvent`.
- Address parsing: `ParseAddrString("kind:pubkey:d")` splits on `:`,
validates kind (0..65535) and pubkey (hex), preserves identifier.
- Standard library only (`iter`, `slices`, `strconv`). No tag taxonomy
enum; NIPs implement their own parsing helpers over raw slices.
- Thread markers (`root`/`reply`/`mention`) and relay-list markers
(`read`/`write`) are read via index, never via typed fields.
### nostr-tools (TypeScript)
- Plain `tags: string[][]`, no wrapper.
- Direct indexing throughout: `tag[0]` name, `tag[1]` value, `tag[2]`
relay, `tag[3]` marker, `tag[4]` pubkey hint.
- Address-tag parsing inline per NIP:
`let [kind, pubkey, identifier] = tag[1].split(':')`.
- NIP-10 supports both explicit markers and legacy positional fallback
(oldest/newest heuristic).
- Each NIP module owns its own tag construction and parsing; no central
tag API.
### rust-nostr (Rust)
- `Tag` wraps `Vec<String>` plus `OnceCell<Option<TagStandard>>` for
lazy parsed enum.
- `TagStandard` enum has 60+ variants covering most NIPs (`Event`,
`PublicKey`, `Coordinate`, `Kind`, `Amount`, `Image`, `Title`, ...).
- `TagKind<'a>` categorizes: named variants, `SingleLetter(SingleLetterTag)`
with case tracking, `Custom(Cow<'a, str>)`.
- E-tag parser is position-aware: `tag[3]` attempts Marker first, falls
back to PublicKey (NIP-01 legacy); `tag[4]` is PublicKey only if `[3]`
was a marker.
- A-tag parser uses `Coordinate::from_str`.
- `Tags` collection (not `Vec<Tag>`) maintains a
`BTreeMap<SingleLetterTag, BTreeSet<String>>` index for dedup and
indexed lookup, plus helpers `event_ids()`, `public_keys()`,
`coordinates()`.
- Trade-offs: extensibility (every new tag type touches the enum),
OnceCell overhead per tag, case-preservation fields. Very thorough
but heavy.
- **We should not replicate the enum approach.** Prefer a thin wrapper
over `Vec<String>` and let callers parse.
### welshman (TypeScript — predecessor of this library)
- No wrapper class; raw `string[][]`.
- 50+ pure functions in `/util/src/Tags.ts`:
- Filters: `getTags(tagName, tags)`, `getTag(tagName, tags)`
- Value extractors: `getTagValues`, `getTagValue`
- Type-specific: `getEventTags`, `getPubkeyTags`, `getAddressTags`,
`getRelayTags`, `getTopicTags`, `getKindTags`
- Reply logic: `getReplyTags`, `getCommentTags` (NIP-10 + NIP-22
uppercase/lowercase dual-tag)
- `uniqTags` dedup, `tagger` factory
- Dedicated `Address` class with `kind`, `pubkey`, `identifier`, `relays`;
factories `from`, `fromNaddr`, `fromEvent`; `isAddress` regex
`^\d+:\w+:.*$`; `toString` and `toNaddr`.
- Event envelope types (`EventContent`, `EventTemplate`, `StampedEvent`,
...) match our exact Rust hierarchy — this is where we borrowed it.
Tags stay as `string[][]`.
- High-level builders in `/app/src/tags.ts`: `tagEventForReply`,
`tagEventForComment`, `tagEventForQuote`, `tagEventForReaction`.
## Common Patterns
**Raw lists dominate.** Every library except rust-nostr keeps tags as the
native string array. The rust-nostr enum is an outlier, and its heaviness
is visible (extensibility pain, memory overhead).
**Free functions over methods.** Welshman and applesauce both prefer pure
functions that take tags and return tags or values. Method-on-type
approaches (ndk) tend to get cluttered.
**Address tags get their own type.** Nostrlib (`EntityPointer`), welshman
(`Address`), applesauce (`AddressPointer`), rust-nostr (`Coordinate`)
all introduce a small struct for `kind:pubkey:d`. This is consistently
the one tag type worth parsing eagerly because it combines three fields
that are always used together.
**Markers are positional.** No library introduces a `Marker` enum
dependency that leaks into the base tag type. Marker interpretation
happens at the reader site (`getReplyTags` etc.), not at construction
time.
**Single-letter indexing matters for filters.** Nostrlib and rust-nostr
explicitly model the single-letter vs multi-character distinction.
Applesauce and welshman rely on convention.
## Considerations for Our Implementation
Given our literate-programming posture and existing style (thin wrappers
over bytes in `keys`, struct pipelines in `events`), we should:
1. **Introduce `Tag(Vec<String>)` as a tuple wrapper.** Provide `name()`,
`value()` (second element or empty), `values()` (all after the first),
`get(i)`, `len()`, `as_slice()`, plus `From<Vec<String>>`,
`IntoIterator`, `Serialize/Deserialize` that flatten transparently to
an array. `new` constructor that takes a name and variadic values.
2. **Free functions on `&[Tag]`.** `find(tags, name)`, `find_all(tags, name)`,
`values(tags, name)`, `value(tags, name)`, `has(tags, name)` as
standalone helpers. Keep them name-agnostic — `name` is `&str`.
3. **An `Address` struct for `a` tags.** Fields `kind: u16`,
`pubkey: PublicKey`, `identifier: String`, plus optional `relays`.
Implement `FromStr`/`Display` for the `kind:pubkey:d` form, and an
`Address::to_tag()` / `Address::from_tag()` pair. Keep it minimal —
no `naddr` yet (that lands in the bech32/entities chapter).
4. **Update `Event` and friends to use `Vec<Tag>`.** The events chapter
left tags as `Vec<Vec<String>>` explicitly because the `Tag` type
wasn't ready. Swap it now and keep the canonical hash bytes identical
(serialize `Tag` transparently as `Vec<String>` in the canonical form).
5. **Stay neutral on semantics.** No `TagKind` enum, no marker parsing
baked into `Tag`. Building-nostr is explicit that tags must be
interpreted in the context of the event kind; a generic type should
not try to know better.
6. **Brief section on markers.** Show how to read NIP-10 markers
positionally — `tag.get(3)` — without introducing a marker type. The
marker-aware reply threading will belong in a later chapter.
7. **No hidden-tag / modify pipelines.** That belongs later, with
encryption of private tag lists.
The goal is a type that disappears when you're not using it and becomes
helpful the moment you are — exactly the "little more than an empty
shell" that building-nostr describes for nostr itself.
+208
View File
@@ -0,0 +1,208 @@
use coracle_lib::addresses::{Address, AddressError};
use coracle_lib::events::EventContent;
use coracle_lib::keys::SecretKey;
use coracle_lib::tags::Tag;
fn fixed_secret() -> SecretKey {
let bytes: [u8; 32] = [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
25, 26, 27, 28, 29, 30, 31, 32,
];
SecretKey::from_hex(&hex::encode(bytes)).unwrap()
}
fn make_signed_event(kind: u16, tags: Vec<Tag>) -> coracle_lib::events::Event {
let sk = fixed_secret();
let pk = sk.public_key();
let hashed = EventContent::new()
.content("test")
.tags(tags)
.kind(kind)
.stamp(1_700_000_000)
.own(pk)
.hash();
let sig = sk.sign(&hashed.id);
hashed.sign(sig)
}
// --- Construction ---
#[test]
fn new_basic() {
let pk = fixed_secret().public_key();
let addr = Address::new(30023, pk, "my-article");
assert_eq!(addr.kind, 30023);
assert_eq!(addr.pubkey, pk);
assert_eq!(addr.identifier, "my-article");
assert!(addr.hints.is_empty());
}
#[test]
fn hints_builder() {
let pk = fixed_secret().public_key();
let addr = Address::new(30023, pk, "my-article")
.hints(["wss://relay.example.com", "wss://other.relay"]);
assert_eq!(addr.hints.len(), 2);
assert_eq!(addr.hints[0], "wss://relay.example.com");
}
#[test]
fn equality_ignores_hints() {
let pk = fixed_secret().public_key();
let a = Address::new(30023, pk, "slug");
let b = Address::new(30023, pk, "slug").hints(["wss://relay.example.com"]);
assert_eq!(a, b);
}
// --- from_event ---
#[test]
fn from_event_addressable() {
let event = make_signed_event(30023, vec![Tag::new("d", ["my-article"])]);
let addr = Address::from_event(&event).unwrap();
assert_eq!(addr.kind, 30023);
assert_eq!(addr.identifier, "my-article");
assert_eq!(addr.pubkey, event.pubkey);
assert!(addr.hints.is_empty());
}
#[test]
fn from_event_replaceable_kind_0() {
let event = make_signed_event(0, vec![]);
let addr = Address::from_event(&event).unwrap();
assert_eq!(addr.kind, 0);
assert_eq!(addr.identifier, "");
}
#[test]
fn from_event_replaceable_kind_3() {
let event = make_signed_event(3, vec![]);
let addr = Address::from_event(&event).unwrap();
assert_eq!(addr.kind, 3);
assert_eq!(addr.identifier, "");
}
#[test]
fn from_event_replaceable_range() {
let event = make_signed_event(10002, vec![]);
let addr = Address::from_event(&event).unwrap();
assert_eq!(addr.kind, 10002);
assert_eq!(addr.identifier, "");
}
#[test]
fn from_event_replaceable_ignores_d_tag() {
let event = make_signed_event(0, vec![Tag::new("d", ["should-be-ignored"])]);
let addr = Address::from_event(&event).unwrap();
assert_eq!(addr.kind, 0);
assert_eq!(addr.identifier, "");
}
#[test]
fn from_event_regular_returns_none() {
let event = make_signed_event(1, vec![]);
assert!(Address::from_event(&event).is_none());
}
#[test]
fn from_event_ephemeral_returns_none() {
let event = make_signed_event(20000, vec![]);
assert!(Address::from_event(&event).is_none());
}
// --- Display / FromStr ---
#[test]
fn display_roundtrip() {
let pk = fixed_secret().public_key();
let addr = Address::new(30023, pk, "my-article");
let s = addr.to_string();
assert!(s.starts_with("30023:"));
assert!(s.ends_with(":my-article"));
let parsed: Address = s.parse().unwrap();
assert_eq!(parsed, addr);
}
#[test]
fn fromstr_empty_identifier() {
let pk = fixed_secret().public_key();
let addr = Address::new(0, pk, "");
let s = addr.to_string();
let parsed: Address = s.parse().unwrap();
assert_eq!(parsed.identifier, "");
assert_eq!(parsed, addr);
}
#[test]
fn fromstr_identifier_with_colons() {
let pk = fixed_secret().public_key();
let addr = Address::new(30023, pk, "has:colons:inside");
let s = addr.to_string();
let parsed: Address = s.parse().unwrap();
assert_eq!(parsed.identifier, "has:colons:inside");
assert_eq!(parsed, addr);
}
#[test]
fn fromstr_invalid() {
assert!("not-an-address".parse::<Address>().is_err());
assert!("123".parse::<Address>().is_err());
assert!("abc:def:ghi".parse::<Address>().is_err());
}
// --- naddr encoding ---
#[test]
fn naddr_roundtrip_no_relays() {
let pk = fixed_secret().public_key();
let addr = Address::new(30023, pk, "my-article");
let naddr = addr.to_naddr();
assert!(naddr.starts_with("naddr1"));
let decoded = Address::from_naddr(&naddr).unwrap();
assert_eq!(decoded, addr);
assert!(decoded.hints.is_empty());
}
#[test]
fn naddr_roundtrip_with_relays() {
let pk = fixed_secret().public_key();
let addr = Address::new(30023, pk, "my-article")
.hints(["wss://relay.example.com", "wss://other.relay"]);
let naddr = addr.to_naddr();
let decoded = Address::from_naddr(&naddr).unwrap();
assert_eq!(decoded, addr);
assert_eq!(
decoded.hints,
vec!["wss://relay.example.com", "wss://other.relay"]
);
}
#[test]
fn naddr_roundtrip_empty_identifier() {
let pk = fixed_secret().public_key();
let addr = Address::new(10002, pk, "");
let naddr = addr.to_naddr();
let decoded = Address::from_naddr(&naddr).unwrap();
assert_eq!(decoded, addr);
}
#[test]
fn fromstr_accepts_naddr() {
let pk = fixed_secret().public_key();
let addr = Address::new(30023, pk, "my-article")
.hints(["wss://relay.example.com"]);
let naddr = addr.to_naddr();
let parsed: Address = naddr.parse().unwrap();
assert_eq!(parsed, addr);
assert_eq!(parsed.hints, vec!["wss://relay.example.com"]);
}
#[test]
fn from_naddr_wrong_prefix() {
let pk = fixed_secret().public_key();
let npub = pk.to_npub();
let err = Address::from_naddr(&npub).unwrap_err();
assert!(matches!(err, AddressError::WrongPrefix { .. }));
}
+120
View File
@@ -0,0 +1,120 @@
use coracle_lib::events::{
Event, EventContent, EventError, EventTemplate, HashedEvent, OwnedEvent, StampedEvent,
};
use coracle_lib::keys::SecretKey;
use coracle_lib::tags::{Tag, Tags};
fn fixed_secret() -> SecretKey {
let bytes: [u8; 32] = [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,
26, 27, 28, 29, 30, 31, 32,
];
SecretKey::from_hex(&hex::encode(bytes)).unwrap()
}
fn build_hashed(content: &str, tags: Vec<Tag>) -> HashedEvent {
EventContent::new()
.content(content)
.tags(tags)
.kind(1)
.stamp(1_700_000_000)
.own(fixed_secret().public_key())
.hash()
}
fn sign_fixture() -> Event {
let sk = fixed_secret();
let hashed = build_hashed("hello nostr", vec![Tag::new("t", ["nostr"])]);
let sig = sk.sign(&hashed.id);
hashed.sign(sig)
}
#[test]
fn pipeline_progresses_through_types() {
let content = EventContent::new().content("hi").tags(Tags::new());
let template: EventTemplate = content.kind(1);
let stamped: StampedEvent = template.stamp(1_700_000_000);
let owned: OwnedEvent = stamped.own(fixed_secret().public_key());
let hashed: HashedEvent = owned.hash();
let sig = fixed_secret().sign(&hashed.id);
let event: Event = hashed.sign(sig);
assert_eq!(event.content, "hi");
assert_eq!(event.kind, 1);
}
#[test]
fn id_is_deterministic() {
let a = build_hashed("hello", vec![]);
let b = build_hashed("hello", vec![]);
assert_eq!(a.id, b.id);
}
#[test]
fn id_changes_with_content() {
let a = build_hashed("hello", vec![]);
let b = build_hashed("goodbye", vec![]);
assert_ne!(a.id, b.id);
}
#[test]
fn sign_and_verify_roundtrip() {
let event = sign_fixture();
assert!(event.verify_id());
event.verify().expect("signature should verify");
}
#[test]
fn tampered_content_fails_id_check() {
let mut event = sign_fixture();
event.content = "something else".to_string();
assert!(!event.verify_id());
assert_eq!(event.verify(), Err(EventError::InvalidId));
}
#[test]
fn tampered_sig_fails_verify() {
let mut event = sign_fixture();
event.sig[0] ^= 0x01;
assert_eq!(event.verify(), Err(EventError::InvalidSignature));
}
#[test]
fn json_roundtrip() {
let event = sign_fixture();
let json = serde_json::to_string(&event).unwrap();
let parsed: Event = serde_json::from_str(&json).unwrap();
assert_eq!(parsed, event);
parsed.verify().expect("roundtripped event still verifies");
}
#[test]
fn json_contains_hex_fields() {
let event = sign_fixture();
let value: serde_json::Value = serde_json::to_value(&event).unwrap();
let obj = value.as_object().unwrap();
assert_eq!(obj["id"].as_str().unwrap().len(), 64);
assert_eq!(obj["pubkey"].as_str().unwrap().len(), 64);
assert_eq!(obj["sig"].as_str().unwrap().len(), 128);
assert_eq!(obj["kind"].as_u64().unwrap(), 1);
assert_eq!(obj["content"].as_str().unwrap(), "hello nostr");
assert!(obj["tags"].is_array());
}
#[test]
fn json_ignores_unknown_fields() {
let event = sign_fixture();
let mut value: serde_json::Value = serde_json::to_value(&event).unwrap();
value
.as_object_mut()
.unwrap()
.insert("custom_field".to_string(), serde_json::json!("ignored"));
let reparsed: Event = serde_json::from_value(value).unwrap();
assert_eq!(reparsed, event);
}
#[test]
fn secret_key_sign_is_deterministic() {
let sk = fixed_secret();
let msg = [42u8; 32];
assert_eq!(sk.sign(&msg), sk.sign(&msg));
}
+165
View File
@@ -0,0 +1,165 @@
use coracle_lib::events::{EventContent, HashedEvent};
use coracle_lib::expiration::{get_expiration, is_expired, is_expired_at};
use coracle_lib::prelude::*;
use coracle_lib::keys::SecretKey;
use coracle_lib::tags::{Tag, Tags};
fn fixed_secret() -> SecretKey {
let bytes: [u8; 32] = [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,
26, 27, 28, 29, 30, 31, 32,
];
SecretKey::from_hex(&hex::encode(bytes)).unwrap()
}
fn build_hashed(tags: Tags) -> HashedEvent {
EventContent::new()
.content("hi")
.tags(tags)
.kind(1)
.stamp(1_700_000_000)
.own(fixed_secret().public_key())
.hash()
}
// --- Tag::expiration constructor ---
#[test]
fn tag_constructor_shape() {
let t = Tag::expiration(1_700_000_000);
assert_eq!(t.name(), "expiration");
assert_eq!(t.value(), "1700000000");
assert_eq!(t.len(), 2);
}
// --- get_expiration ---
#[test]
fn get_expiration_returns_none_when_tag_absent() {
let tags = Tags::new();
assert_eq!(get_expiration(&tags), None);
}
#[test]
fn get_expiration_parses_numeric_value() {
let tags = Tags::from(vec![Tag::expiration(1_700_000_000)]);
assert_eq!(get_expiration(&tags), Some(1_700_000_000));
}
#[test]
fn get_expiration_returns_none_when_malformed() {
let tags = Tags::from(vec![Tag::new("expiration", ["not-a-number"])]);
assert_eq!(get_expiration(&tags), None);
}
#[test]
fn get_expiration_reads_first_matching_tag() {
let tags = Tags::from(vec![
Tag::new("t", ["nostr"]),
Tag::expiration(100),
Tag::expiration(200),
]);
assert_eq!(get_expiration(&tags), Some(100));
}
// --- is_expired_at ---
#[test]
fn is_expired_at_false_when_tag_absent() {
let tags = Tags::new();
assert!(!is_expired_at(&tags, u64::MAX));
}
#[test]
fn is_expired_at_false_when_now_before_expiration() {
let tags = Tags::from(vec![Tag::expiration(100)]);
assert!(!is_expired_at(&tags, 99));
}
#[test]
fn is_expired_at_false_when_now_equals_expiration() {
// Strict `<`: equal is not yet expired.
let tags = Tags::from(vec![Tag::expiration(100)]);
assert!(!is_expired_at(&tags, 100));
}
#[test]
fn is_expired_at_true_when_now_past_expiration() {
let tags = Tags::from(vec![Tag::expiration(100)]);
assert!(is_expired_at(&tags, 101));
}
#[test]
fn is_expired_at_false_when_malformed() {
let tags = Tags::from(vec![Tag::new("expiration", ["nope"])]);
assert!(!is_expired_at(&tags, u64::MAX));
}
// --- is_expired (system clock) ---
#[test]
fn is_expired_true_for_unix_epoch_tag() {
// Unless the system clock is wildly wrong, `now` is far past 1.
let tags = Tags::from(vec![Tag::expiration(1)]);
assert!(is_expired(&tags));
}
#[test]
fn is_expired_false_for_far_future_tag() {
let tags = Tags::from(vec![Tag::expiration(u64::MAX)]);
assert!(!is_expired(&tags));
}
#[test]
fn is_expired_false_when_tag_absent() {
let tags = Tags::new();
assert!(!is_expired(&tags));
}
// --- HashedEvent method facades ---
#[test]
fn hashed_event_reads_expiration() {
let tags = Tags::from(vec![Tag::expiration(1_700_000_500)]);
let event = build_hashed(tags);
assert_eq!(event.get_expiration(), Some(1_700_000_500));
assert!(!event.is_expired_at(1_700_000_500));
assert!(event.is_expired_at(1_700_000_501));
}
#[test]
fn hashed_event_no_expiration_without_tag() {
let event = build_hashed(Tags::new());
assert_eq!(event.get_expiration(), None);
assert!(!event.is_expired_at(u64::MAX));
assert!(!event.is_expired());
}
// --- Event method facades ---
#[test]
fn signed_event_reads_expiration() {
let sk = fixed_secret();
let tags = Tags::from(vec![Tag::expiration(1_700_000_500)]);
let hashed = build_hashed(tags);
let sig = sk.sign(&hashed.id);
let signed = hashed.sign(sig);
assert_eq!(signed.get_expiration(), Some(1_700_000_500));
assert!(signed.is_expired_at(1_700_000_501));
assert!(!signed.is_expired_at(1_700_000_499));
}
#[test]
fn expiration_tag_survives_hashing_and_signing() {
// The tag is part of the canonical hash input, so it has to
// round-trip through `hash()` and `sign()` intact.
let sk = fixed_secret();
let ts = 1_700_000_500;
let hashed = build_hashed(Tags::from(vec![Tag::expiration(ts)]));
let sig = sk.sign(&hashed.id);
let signed = hashed.sign(sig);
assert!(signed.verify_id());
assert_eq!(signed.get_expiration(), Some(ts));
}
+675
View File
@@ -0,0 +1,675 @@
use std::collections::BTreeSet;
use coracle_lib::events::{Event, EventContent};
use coracle_lib::filters::{intersect_filters, matches_any, union_filters, Filter, TagMatch};
use coracle_lib::keys::SecretKey;
use coracle_lib::tags::Tag;
fn fixed_secret() -> SecretKey {
let bytes: [u8; 32] = [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
25, 26, 27, 28, 29, 30, 31, 32,
];
SecretKey::from_hex(&hex::encode(bytes)).unwrap()
}
fn make_event(kind: u16, timestamp: u64, tags: Vec<Tag>) -> Event {
let sk = fixed_secret();
let hashed = EventContent::new()
.content("hello")
.tags(tags)
.kind(kind)
.stamp(timestamp)
.own(sk.public_key())
.hash();
hashed.clone().sign(sk.sign(&hashed.id))
}
// --- Matching ---
#[test]
fn empty_filter_matches_everything() {
let event = make_event(1, 1_700_000_000, vec![]);
assert!(Filter::new().matches(&event));
}
#[test]
fn filter_matches_by_id() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter::new().add_id(event.id);
assert!(filter.matches(&event));
}
#[test]
fn filter_rejects_wrong_id() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter::new().add_id([0u8; 32]);
assert!(!filter.matches(&event));
}
#[test]
fn filter_matches_by_kind() {
let event = make_event(1, 1_700_000_000, vec![]);
assert!(Filter::new().add_kind(1).matches(&event));
assert!(!Filter::new().add_kind(2).matches(&event));
}
#[test]
fn filter_matches_by_author() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter::new().add_author(fixed_secret().public_key());
assert!(filter.matches(&event));
}
#[test]
fn filter_rejects_wrong_author() {
let event = make_event(1, 1_700_000_000, vec![]);
let other_sk = SecretKey::generate();
let filter = Filter::new().add_author(other_sk.public_key());
assert!(!filter.matches(&event));
}
#[test]
fn filter_matches_by_tag() {
let event = make_event(
1,
1_700_000_000,
vec![Tag::new("t", ["nostr"]), Tag::new("p", ["abc123"])],
);
assert!(Filter::new().add_tag(TagMatch::Any, "t", "nostr").matches(&event));
assert!(!Filter::new().add_tag(TagMatch::Any, "e", "something").matches(&event));
assert!(!Filter::new().add_tag(TagMatch::Any, "t", "bitcoin").matches(&event));
}
#[test]
fn filter_tag_or_within_same_name() {
let event = make_event(1, 1_700_000_000, vec![Tag::new("t", ["nostr"])]);
let filter = Filter::new().add_tags(TagMatch::Any, "t", ["nostr", "bitcoin"]);
assert!(filter.matches(&event));
}
#[test]
fn filter_tag_and_across_names() {
let event = make_event(
1,
1_700_000_000,
vec![Tag::new("t", ["nostr"]), Tag::new("p", ["abc"])],
);
let filter = Filter::new().add_tag(TagMatch::Any, "t", "nostr").add_tag(TagMatch::Any, "p", "abc");
assert!(filter.matches(&event));
let filter = Filter::new().add_tag(TagMatch::Any, "t", "nostr").add_tag(TagMatch::Any, "e", "missing");
assert!(!filter.matches(&event));
}
#[test]
fn filter_and_tag_requires_all_values() {
let both = make_event(
1,
1_700_000_000,
vec![Tag::new("t", ["meme"]), Tag::new("t", ["cat"])],
);
let one = make_event(1, 1_700_000_000, vec![Tag::new("t", ["meme"])]);
let filter = Filter::new().add_tags(TagMatch::All, "t", ["meme", "cat"]);
assert!(filter.matches(&both));
assert!(!filter.matches(&one)); // missing "cat"
}
#[test]
fn filter_or_and_keys_combine() {
// `#t` (OR) and `&t` (AND) are independent entries; both must hold.
let filter = Filter::new()
.add_tags(TagMatch::All, "t", ["meme", "cat"])
.add_tags(TagMatch::Any, "t", ["black", "white"]);
// Has meme and cat but neither black nor white: the `#t` OR fails.
let no_color = make_event(
1,
1_700_000_000,
vec![Tag::new("t", ["meme"]), Tag::new("t", ["cat"])],
);
assert!(!filter.matches(&no_color));
// meme, cat, black: both the AND and the OR are satisfied.
let ok = make_event(
1,
1_700_000_000,
vec![
Tag::new("t", ["meme"]),
Tag::new("t", ["cat"]),
Tag::new("t", ["black"]),
],
);
assert!(filter.matches(&ok));
}
#[test]
fn filter_since_inclusive() {
let event = make_event(1, 1000, vec![]);
assert!(Filter::new().add_since(1000).matches(&event));
assert!(Filter::new().add_since(999).matches(&event));
assert!(!Filter::new().add_since(1001).matches(&event));
}
#[test]
fn filter_until_inclusive() {
let event = make_event(1, 1000, vec![]);
assert!(Filter::new().add_until(1000).matches(&event));
assert!(Filter::new().add_until(1001).matches(&event));
assert!(!Filter::new().add_until(999).matches(&event));
}
#[test]
fn filter_limit_ignored_in_matching() {
let event = make_event(1, 1_700_000_000, vec![]);
assert!(Filter::new().add_limit(0).matches(&event));
}
#[test]
fn filter_and_semantics() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter::new()
.add_kind(1)
.add_author(fixed_secret().public_key());
assert!(filter.matches(&event));
let filter = Filter::new()
.add_kind(2)
.add_author(fixed_secret().public_key());
assert!(!filter.matches(&event));
}
#[test]
fn empty_set_matches_nothing() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter {
ids: Some(BTreeSet::new()),
..Filter::new()
};
assert!(!filter.matches(&event));
let filter = Filter {
kinds: Some(BTreeSet::new()),
..Filter::new()
};
assert!(!filter.matches(&event));
}
// --- matches_any ---
#[test]
fn matches_any_or_semantics() {
let event = make_event(1, 1_700_000_000, vec![]);
let filters = vec![Filter::new().add_kind(2), Filter::new().add_kind(1)];
assert!(matches_any(&filters, &event));
}
#[test]
fn matches_any_empty_slice() {
let event = make_event(1, 1_700_000_000, vec![]);
assert!(!matches_any(&[], &event));
}
// --- Construction: add ---
#[test]
fn builder_chaining() {
let pk = fixed_secret().public_key();
let filter = Filter::new()
.add_kind(1)
.add_kinds([2, 3])
.add_author(pk)
.add_tag(TagMatch::Any, "t", "nostr")
.add_since(1000)
.add_until(2000)
.add_limit(10);
assert_eq!(filter.kinds.as_ref().unwrap().len(), 3);
assert!(filter.kinds.as_ref().unwrap().contains(&1));
assert!(filter.kinds.as_ref().unwrap().contains(&2));
assert!(filter.kinds.as_ref().unwrap().contains(&3));
assert_eq!(filter.since, Some(1000));
assert_eq!(filter.until, Some(2000));
assert_eq!(filter.limit, Some(10));
}
#[test]
fn address_convenience() {
let pk = fixed_secret().public_key();
let addr = coracle_lib::addresses::Address::new(30023, pk, "my-article");
let filter = Filter::new().add_address(&addr);
assert!(filter.kinds.as_ref().unwrap().contains(&30023));
assert!(filter.authors.as_ref().unwrap().contains(&pk));
assert!(filter.tags.get("#d").unwrap().contains("my-article"));
}
// --- Construction: remove ---
#[test]
fn remove_id() {
let id = [1u8; 32];
let filter = Filter::new().add_id(id).add_id([2u8; 32]).remove_id(&id);
assert!(!filter.ids.as_ref().unwrap().contains(&id));
assert!(filter.ids.as_ref().unwrap().contains(&[2u8; 32]));
}
#[test]
fn remove_author() {
let pk = fixed_secret().public_key();
let filter = Filter::new().add_author(pk).remove_author(&pk);
// Set exists but is empty — matches nothing
assert!(filter.authors.as_ref().unwrap().is_empty());
}
#[test]
fn remove_kind() {
let filter = Filter::new()
.add_kinds([1, 2, 3])
.remove_kind(&2);
let kinds = filter.kinds.as_ref().unwrap();
assert!(kinds.contains(&1));
assert!(!kinds.contains(&2));
assert!(kinds.contains(&3));
}
#[test]
fn remove_tag_value() {
let filter = Filter::new()
.add_tags(TagMatch::Any, "t", ["nostr", "bitcoin"])
.remove_tag(TagMatch::Any, "t", "bitcoin");
let values = filter.tags.get("#t").unwrap();
assert!(values.contains("nostr"));
assert!(!values.contains("bitcoin"));
}
#[test]
fn remove_tag_last_value_removes_entry() {
let filter = Filter::new()
.add_tag(TagMatch::Any, "t", "nostr")
.remove_tag(TagMatch::Any, "t", "nostr");
assert!(!filter.tags.contains_key("#t"));
}
#[test]
fn remove_on_none_is_noop() {
// Removing from a field that was never set should not panic
let filter = Filter::new()
.remove_id(&[0u8; 32])
.remove_kind(&1)
.remove_author(&fixed_secret().public_key())
.remove_tag(TagMatch::Any, "t", "nostr");
assert!(filter.ids.is_none());
assert!(filter.kinds.is_none());
assert!(filter.authors.is_none());
}
// --- Construction: clear ---
#[test]
fn clear_ids() {
let filter = Filter::new().add_id([1u8; 32]).clear_ids();
assert!(filter.ids.is_none());
}
#[test]
fn clear_authors() {
let filter = Filter::new()
.add_author(fixed_secret().public_key())
.clear_authors();
assert!(filter.authors.is_none());
}
#[test]
fn clear_kinds() {
let filter = Filter::new().add_kind(1).clear_kinds();
assert!(filter.kinds.is_none());
}
#[test]
fn clear_tag() {
let filter = Filter::new()
.add_tag(TagMatch::Any, "t", "nostr")
.add_tag(TagMatch::Any, "e", "abc")
.clear_tag(TagMatch::Any, "t");
assert!(!filter.tags.contains_key("#t"));
assert!(filter.tags.contains_key("#e"));
}
#[test]
fn clear_tags() {
let filter = Filter::new()
.add_tag(TagMatch::Any, "t", "nostr")
.add_tag(TagMatch::Any, "e", "abc")
.clear_tags();
assert!(filter.tags.is_empty());
}
#[test]
fn clear_since() {
let filter = Filter::new().add_since(1000).clear_since();
assert!(filter.since.is_none());
}
#[test]
fn clear_until() {
let filter = Filter::new().add_until(2000).clear_until();
assert!(filter.until.is_none());
}
#[test]
fn clear_limit() {
let filter = Filter::new().add_limit(10).clear_limit();
assert!(filter.limit.is_none());
}
// --- Serialization ---
#[test]
fn json_round_trip() {
let pk = fixed_secret().public_key();
let filter = Filter::new()
.add_kind(1)
.add_author(pk)
.add_tag(TagMatch::Any, "t", "nostr")
.add_tag(TagMatch::Any, "e", "abc123")
.add_since(1000)
.add_until(2000)
.add_limit(10);
let json = serde_json::to_string(&filter).unwrap();
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(filter, parsed);
}
#[test]
fn json_tag_flattening() {
let filter = Filter::new().add_tag(TagMatch::Any, "t", "nostr");
let json = serde_json::to_string(&filter).unwrap();
assert!(json.contains("\"#t\""));
assert!(json.contains("\"nostr\""));
}
#[test]
fn json_and_tag_flattening() {
let filter = Filter::new()
.add_tags(TagMatch::All, "t", ["meme", "cat"])
.add_tags(TagMatch::Any, "t", ["black", "white"]);
let json = serde_json::to_string(&filter).unwrap();
assert!(json.contains("\"&t\""), "json was: {json}");
assert!(json.contains("\"#t\""), "json was: {json}");
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(filter, parsed);
}
#[test]
fn json_empty_filter() {
let filter = Filter::new();
let json = serde_json::to_string(&filter).unwrap();
assert_eq!(json, "{}");
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(filter, parsed);
}
#[test]
fn json_limit_zero() {
let filter = Filter::new().add_limit(0);
let json = serde_json::to_string(&filter).unwrap();
assert!(json.contains("\"limit\":0"));
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.limit, Some(0));
}
#[test]
fn json_ids_as_hex() {
let filter = Filter::new().add_id([0xab; 32]);
let json = serde_json::to_string(&filter).unwrap();
assert!(json.contains(&hex::encode([0xab; 32])));
}
#[test]
fn json_ignores_unknown_keys() {
let json = r#"{"kinds": [1], "unknown_field": true}"#;
let filter: Filter = serde_json::from_str(json).unwrap();
assert_eq!(filter.kinds, Some(BTreeSet::from([1])));
}
// --- Identity and grouping ---
#[test]
fn filter_id_deterministic() {
let f1 = Filter::new().add_kind(1).add_tag(TagMatch::Any, "t", "nostr");
let f2 = Filter::new().add_kind(1).add_tag(TagMatch::Any, "t", "nostr");
assert_eq!(f1.id(), f2.id());
}
#[test]
fn filter_id_differs_for_different_filters() {
let f1 = Filter::new().add_kind(1);
let f2 = Filter::new().add_kind(2);
assert_ne!(f1.id(), f2.id());
}
#[test]
fn filter_group_same_shape() {
let f1 = Filter::new().add_kind(1).add_tag(TagMatch::Any, "t", "nostr");
let f2 = Filter::new().add_kind(2).add_tag(TagMatch::Any, "t", "bitcoin");
assert_eq!(f1.group(), f2.group());
}
#[test]
fn filter_group_different_shape() {
let f1 = Filter::new().add_kind(1).add_tag(TagMatch::Any, "t", "nostr");
let f2 = Filter::new().add_kind(1).add_tag(TagMatch::Any, "e", "abc");
assert_ne!(f1.group(), f2.group());
}
#[test]
fn group_key_same_time_window_mergeable() {
let f1 = Filter::new().add_kind(1).add_since(1000).add_until(2000);
let f2 = Filter::new().add_kind(2).add_since(1000).add_until(2000);
assert_eq!(f1.group(), f2.group());
}
#[test]
fn group_key_different_time_window_not_mergeable() {
let f1 = Filter::new().add_kind(1).add_since(1000);
let f2 = Filter::new().add_kind(1).add_since(2000);
assert_ne!(f1.group(), f2.group());
}
#[test]
fn group_key_limit_always_unique() {
let f1 = Filter::new().add_kind(1).add_limit(10);
let f2 = Filter::new().add_kind(1).add_limit(10);
// Even identical limits produce different group keys
assert_ne!(f1.group(), f2.group());
}
#[test]
fn group_key_no_limit_is_mergeable() {
let f1 = Filter::new().add_kind(1);
let f2 = Filter::new().add_kind(2);
assert_eq!(f1.group(), f2.group());
}
#[test]
fn group_distinguishes_or_and_keys() {
// `#t` and `&t` are different keys, so they never share a group.
let or = Filter::new().add_kind(1).add_tag(TagMatch::Any, "t", "meme");
let and = Filter::new().add_kind(1).add_tag(TagMatch::All, "t", "meme");
assert_ne!(or.group(), and.group());
}
#[test]
fn group_distinct_and_values_not_mergeable() {
// AND values are part of the group identity, so they can't be merged.
let f1 = Filter::new().add_kind(1).add_tag(TagMatch::All, "t", "meme");
let f2 = Filter::new().add_kind(1).add_tag(TagMatch::All, "t", "cat");
assert_ne!(f1.group(), f2.group());
}
// --- union_filters ---
#[test]
fn union_merges_same_shape() {
let pk1 = fixed_secret().public_key();
let pk2 = SecretKey::generate().public_key();
let filters = vec![
Filter::new().add_kind(1).add_author(pk1),
Filter::new().add_kind(1).add_author(pk2),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 1);
let merged = &result[0];
assert!(merged.authors.as_ref().unwrap().contains(&pk1));
assert!(merged.authors.as_ref().unwrap().contains(&pk2));
assert!(merged.kinds.as_ref().unwrap().contains(&1));
}
#[test]
fn union_keeps_different_shapes_separate() {
let filters = vec![
Filter::new().add_kind(1),
Filter::new().add_kind(1).add_tag(TagMatch::Any, "t", "nostr"),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 2);
}
#[test]
fn union_keeps_different_time_windows_separate() {
let filters = vec![
Filter::new().add_kind(1).add_since(1000),
Filter::new().add_kind(2).add_since(2000),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 2);
}
#[test]
fn union_keeps_limited_filters_separate() {
let filters = vec![
Filter::new().add_kind(1).add_limit(10),
Filter::new().add_kind(2).add_limit(10),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 2);
}
#[test]
fn union_empty_input() {
let result = union_filters(&[]);
assert!(result.is_empty());
}
#[test]
fn union_merges_tag_values() {
let filters = vec![
Filter::new().add_tag(TagMatch::Any, "t", "nostr"),
Filter::new().add_tag(TagMatch::Any, "t", "bitcoin"),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 1);
let values = result[0].tags.get("#t").unwrap();
assert!(values.contains("nostr"));
assert!(values.contains("bitcoin"));
}
#[test]
fn union_keeps_distinct_and_values_separate() {
let filters = vec![
Filter::new().add_kind(1).add_tag(TagMatch::All, "t", "meme"),
Filter::new().add_kind(1).add_tag(TagMatch::All, "t", "cat"),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 2);
}
// --- intersect_filters ---
#[test]
fn intersect_cartesian_product() {
let groups = vec![
vec![Filter::new().add_kind(1), Filter::new().add_kind(2)],
vec![Filter::new().add_tag(TagMatch::Any, "t", "nostr")],
];
let result = intersect_filters(&groups);
// Two combinations merged by union into one: kinds(1,2)+tag(t)
assert_eq!(result.len(), 1);
assert!(result[0].kinds.as_ref().unwrap().contains(&1));
assert!(result[0].kinds.as_ref().unwrap().contains(&2));
assert!(result[0].tags.contains_key("#t"));
}
#[test]
fn intersect_incompatible_shapes_stay_separate() {
let groups = vec![
vec![Filter::new().add_kind(1)],
vec![
Filter::new().add_tag(TagMatch::Any, "t", "nostr"),
Filter::new().add_tag(TagMatch::Any, "e", "abc"),
],
];
let result = intersect_filters(&groups);
// kind+#t and kind+#e are different shapes, can't merge
assert_eq!(result.len(), 2);
}
#[test]
fn intersect_tightens_time_window() {
let groups = vec![
vec![Filter::new().add_since(1000).add_until(3000)],
vec![Filter::new().add_since(2000).add_until(4000)],
];
let result = intersect_filters(&groups);
assert_eq!(result.len(), 1);
assert_eq!(result[0].since, Some(2000)); // max
assert_eq!(result[0].until, Some(3000)); // min
}
#[test]
fn intersect_accumulates_and_values() {
// Each group requires a different `&t` value; the intersection must
// require both, so the values accumulate into one filter.
let groups = vec![
vec![Filter::new().add_tag(TagMatch::All, "t", "meme")],
vec![Filter::new().add_tag(TagMatch::All, "t", "cat")],
];
let result = intersect_filters(&groups);
assert_eq!(result.len(), 1);
let required = result[0].tags.get("&t").unwrap();
assert!(required.contains("meme"));
assert!(required.contains("cat"));
let both = make_event(
1,
1_700_000_000,
vec![Tag::new("t", ["meme"]), Tag::new("t", ["cat"])],
);
assert!(result[0].matches(&both));
let one = make_event(1, 1_700_000_000, vec![Tag::new("t", ["meme"])]);
assert!(!result[0].matches(&one));
}
#[test]
fn intersect_empty_groups() {
let result = intersect_filters(&[]);
assert!(result.is_empty());
}
#[test]
fn intersect_single_group_passthrough() {
let groups = vec![vec![
Filter::new().add_kind(1),
Filter::new().add_kind(2),
]];
let result = intersect_filters(&groups);
// Should union the two compatible filters
assert_eq!(result.len(), 1);
assert!(result[0].kinds.as_ref().unwrap().contains(&1));
assert!(result[0].kinds.as_ref().unwrap().contains(&2));
}
+77
View File
@@ -0,0 +1,77 @@
use coracle_lib::kinds::{is_addressable, is_ephemeral, is_regular, is_replaceable};
use coracle_lib::kinds::KindError;
// --- Free function classification tests ---
#[test]
fn regular_kind_1() {
assert!(is_regular(1));
}
#[test]
fn regular_kind_9999() {
assert!(is_regular(9999));
}
#[test]
fn kind_0_is_replaceable_not_regular() {
assert!(!is_regular(0));
assert!(is_replaceable(0));
}
#[test]
fn kind_3_is_replaceable_not_regular() {
assert!(!is_regular(3));
assert!(is_replaceable(3));
}
#[test]
fn replaceable_range() {
assert!(is_replaceable(10_000));
assert!(is_replaceable(19_999));
assert!(!is_replaceable(20_000));
}
#[test]
fn ephemeral_range() {
assert!(is_ephemeral(20_000));
assert!(is_ephemeral(29_999));
assert!(!is_ephemeral(30_000));
}
#[test]
fn addressable_range() {
assert!(is_addressable(30_000));
assert!(is_addressable(39_999));
assert!(!is_addressable(40_000));
}
#[test]
fn ranges_are_mutually_exclusive_for_regular() {
for k in [1, 100, 5000, 9999] {
assert!(is_regular(k));
assert!(!is_replaceable(k));
assert!(!is_ephemeral(k));
assert!(!is_addressable(k));
}
}
#[test]
fn high_kinds_are_none_of_the_above() {
assert!(!is_regular(40_000));
assert!(!is_replaceable(40_000));
assert!(!is_ephemeral(40_000));
assert!(!is_addressable(40_000));
}
#[test]
fn kind_error_display() {
let err = KindError::WrongKind {
expected: 5,
got: 1,
};
assert_eq!(err.to_string(), "expected kind 5, got 1");
let err = KindError::InvalidContent("bad json".into());
assert_eq!(err.to_string(), "invalid content for kind: bad json");
}
+220
View File
@@ -0,0 +1,220 @@
use coracle_lib::events::{EventContent, OwnedEvent};
use coracle_lib::keys::SecretKey;
use coracle_lib::prelude::*;
use coracle_lib::pow::{get_leading_zero_bits, get_pow, mine_pow, mine_pow_batch};
use coracle_lib::tags::{Tag, Tags};
fn fixed_secret() -> SecretKey {
let bytes: [u8; 32] = [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
25, 26, 27, 28, 29, 30, 31, 32,
];
SecretKey::from_hex(&hex::encode(bytes)).unwrap()
}
fn test_event() -> OwnedEvent {
EventContent::new()
.content("hello nostr")
.tags(Tags::new())
.kind(1)
.stamp(1_700_000_000)
.own(fixed_secret().public_key())
}
// --- get_leading_zero_bits ---
#[test]
fn leading_zeros_empty_slice() {
assert_eq!(get_leading_zero_bits(&[]), 0);
}
#[test]
fn leading_zeros_all_zeros() {
assert_eq!(get_leading_zero_bits(&[0, 0, 0, 0]), 32);
}
#[test]
fn leading_zeros_first_byte_nonzero() {
assert_eq!(get_leading_zero_bits(&[0x80, 0x00]), 0);
assert_eq!(get_leading_zero_bits(&[0x01, 0x00]), 7);
assert_eq!(get_leading_zero_bits(&[0x0F]), 4);
}
#[test]
fn leading_zeros_with_zero_prefix() {
assert_eq!(get_leading_zero_bits(&[0x00, 0x01]), 15);
assert_eq!(get_leading_zero_bits(&[0x00, 0x00, 0x80]), 16);
assert_eq!(get_leading_zero_bits(&[0x00, 0x00, 0x00, 0x01]), 31);
}
#[test]
fn leading_zeros_known_hash() {
// "000006d8..." → 0x00, 0x00, 0x06 → 16 + 5 = 21
let hash = hex::decode("000006d8c378af1779d2feebc7603a125d99eca0ccf1085959b307f64e5dd358")
.unwrap();
assert_eq!(get_leading_zero_bits(&hash), 21);
}
// --- get_pow (free function) ---
#[test]
fn get_pow_returns_min_of_hash_and_claim() {
let event = test_event();
let hashed = mine_pow(&event, 8);
// The free function should return at least 8
assert!(get_pow(&hashed.id, &hashed.tags) >= 8);
}
#[test]
fn get_pow_zero_without_nonce_tag() {
let hashed = test_event().hash();
assert_eq!(get_pow(&hashed.id, &hashed.tags), 0);
}
#[test]
fn get_pow_capped_by_claimed_difficulty() {
// Mine at difficulty 4 — even if hash has more leading zeros,
// get_pow returns at most 4
let event = test_event();
let hashed = mine_pow(&event, 4);
assert!(get_pow(&hashed.id, &hashed.tags) >= 4);
// Actual leading zeros might exceed 4, but get_pow is min(leading, claimed)
assert_eq!(get_pow(&hashed.id, &hashed.tags), 4);
}
// --- HashedEvent::get_pow method ---
#[test]
fn hashed_event_get_pow_method() {
let event = test_event();
let hashed = mine_pow(&event, 8);
assert!(hashed.get_pow() >= 8);
}
#[test]
fn hashed_event_get_pow_zero_without_nonce() {
let hashed = test_event().hash();
assert_eq!(hashed.get_pow(), 0);
}
// --- Event::get_pow method ---
#[test]
fn signed_event_get_pow_method() {
let sk = fixed_secret();
let event = test_event();
let hashed = mine_pow(&event, 8);
let sig = sk.sign(&hashed.id);
let signed = hashed.sign(sig);
assert!(signed.get_pow() >= 8);
}
// --- OwnedEvent::mine_pow method ---
#[test]
fn owned_event_mine_pow_method() {
let event = test_event();
let result = event.mine_pow(8);
assert!(result.get_pow() >= 8);
}
// --- OwnedEvent::mine_pow_batch method ---
#[test]
fn owned_event_mine_pow_batch_method() {
let event = test_event();
let result = event.mine_pow_batch(1, 0, 1_000_000);
assert!(result.is_some());
assert!(result.unwrap().get_pow() >= 1);
}
// --- mine_pow (free function) ---
#[test]
fn mine_pow_returns_valid_result() {
let event = test_event();
let result = mine_pow(&event, 8);
assert!(get_leading_zero_bits(&result.id) >= 8);
let nonce_tag = result.tags.find("nonce").expect("should have nonce tag");
assert_eq!(nonce_tag.get(2), Some("8"));
let _nonce: u64 = nonce_tag.value().parse().expect("nonce should be a number");
}
#[test]
fn mine_pow_preserves_event_fields() {
let event = test_event();
let result = mine_pow(&event, 4);
assert_eq!(result.content, event.content);
assert_eq!(result.kind, event.kind);
assert_eq!(result.created_at, event.created_at);
assert_eq!(result.pubkey, event.pubkey);
}
#[test]
fn mine_pow_preserves_existing_tags() {
let mut event = test_event();
event.tags.0.push(Tag::new("t", ["nostr"]));
let result = mine_pow(&event, 4);
assert!(result.tags.find("t").is_some());
assert!(result.tags.find("nonce").is_some());
}
#[test]
fn mine_pow_does_not_mutate_input() {
let event = test_event();
let tags_before = event.tags.clone();
let _result = mine_pow(&event, 4);
assert_eq!(event.tags, tags_before);
}
// --- mine_pow_batch (free function) ---
#[test]
fn mine_pow_batch_returns_none_when_exhausted() {
let event = test_event();
let result = mine_pow_batch(&event, 64, 0, 10);
assert!(result.is_none());
}
#[test]
fn mine_pow_batch_finds_solution_in_range() {
let event = test_event();
let result = mine_pow_batch(&event, 1, 0, 1_000_000);
assert!(result.is_some());
assert!(get_leading_zero_bits(&result.unwrap().id) >= 1);
}
#[test]
fn mine_pow_batch_respects_start_offset() {
let event = test_event();
let a = mine_pow_batch(&event, 4, 0, 1_000_000);
let b = mine_pow_batch(&event, 4, 500_000, 1_000_000);
assert!(a.is_some());
assert!(b.is_some());
let nonce_a = a.unwrap().tags.find("nonce").unwrap().value().to_string();
let nonce_b = b.unwrap().tags.find("nonce").unwrap().value().to_string();
assert_ne!(nonce_a, nonce_b);
}
// --- Integration: mine then sign then verify ---
#[test]
fn mined_event_can_be_signed_and_verified() {
let sk = fixed_secret();
let event = test_event();
let hashed = mine_pow(&event, 8);
let sig = sk.sign(&hashed.id);
let signed = hashed.sign(sig);
assert!(signed.verify_id());
signed.verify().expect("mined and signed event should verify");
assert!(signed.get_pow() >= 8);
}
+126
View File
@@ -0,0 +1,126 @@
use coracle_lib::events::{EventContent, HashedEvent};
use coracle_lib::keys::SecretKey;
use coracle_lib::prelude::*;
use coracle_lib::protected::is_protected;
use coracle_lib::tags::{Tag, Tags};
fn fixed_secret() -> SecretKey {
let bytes: [u8; 32] = [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,
26, 27, 28, 29, 30, 31, 32,
];
SecretKey::from_hex(&hex::encode(bytes)).unwrap()
}
fn build_hashed(tags: Tags) -> HashedEvent {
EventContent::new()
.content("hi")
.tags(tags)
.kind(1)
.stamp(1_700_000_000)
.own(fixed_secret().public_key())
.hash()
}
// --- Tag::protected constructor ---
#[test]
fn tag_constructor_shape() {
let t = Tag::protected();
assert_eq!(t.name(), "-");
assert_eq!(t.len(), 1);
assert!(t.values().is_empty());
}
#[test]
fn tag_constructor_serializes_as_single_element() {
let t = Tag::protected();
assert_eq!(serde_json::to_string(&t).unwrap(), r#"["-"]"#);
}
// --- is_protected ---
#[test]
fn is_protected_false_when_tag_absent() {
let tags = Tags::new();
assert!(!is_protected(&tags));
}
#[test]
fn is_protected_false_with_unrelated_tags() {
let tags = Tags::from(vec![Tag::new("t", ["nostr"]), Tag::expiration(100)]);
assert!(!is_protected(&tags));
}
#[test]
fn is_protected_true_when_tag_present() {
let tags = Tags::from(vec![Tag::protected()]);
assert!(is_protected(&tags));
}
#[test]
fn is_protected_true_among_other_tags() {
let tags = Tags::from(vec![
Tag::new("t", ["nostr"]),
Tag::protected(),
Tag::expiration(100),
]);
assert!(is_protected(&tags));
}
#[test]
fn is_protected_detects_a_raw_dash_tag() {
// The marker is identified by name alone; a hand-built ["-"] is
// equivalent to Tag::protected().
let tags = Tags::from(vec![Tag::new("-", Vec::<String>::new())]);
assert!(is_protected(&tags));
}
// --- HashedEvent method facade ---
#[test]
fn hashed_event_reports_protected() {
let event = build_hashed(Tags::from(vec![Tag::protected()]));
assert!(event.is_protected());
}
#[test]
fn hashed_event_reports_unprotected() {
let event = build_hashed(Tags::new());
assert!(!event.is_protected());
}
// --- Event method facade ---
#[test]
fn signed_event_reports_protected() {
let sk = fixed_secret();
let hashed = build_hashed(Tags::from(vec![Tag::protected()]));
let sig = sk.sign(&hashed.id);
let signed = hashed.sign(sig);
assert!(signed.is_protected());
}
#[test]
fn protected_tag_survives_hashing_and_signing() {
// The tag is part of the canonical hash input, so it must round-trip
// through hash() and sign() intact and still verify.
let sk = fixed_secret();
let hashed = build_hashed(Tags::from(vec![Tag::protected()]));
let sig = sk.sign(&hashed.id);
let signed = hashed.sign(sig);
assert!(signed.verify_id());
assert!(signed.is_protected());
}
#[test]
fn unprotected_event_stays_unprotected_through_signing() {
let sk = fixed_secret();
let hashed = build_hashed(Tags::new());
let sig = sk.sign(&hashed.id);
let signed = hashed.sign(sig);
assert!(!signed.is_protected());
}
+250
View File
@@ -0,0 +1,250 @@
use coracle_lib::events::{Event, EventContent};
use coracle_lib::filters::{intersect_filters, union_filters, Filter};
use coracle_lib::keys::SecretKey;
use coracle_lib::search::SearchQuery;
fn fixed_secret() -> SecretKey {
let bytes: [u8; 32] = [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
25, 26, 27, 28, 29, 30, 31, 32,
];
SecretKey::from_hex(&hex::encode(bytes)).unwrap()
}
fn make_event(content: &str, created_at: u64) -> Event {
let sk = fixed_secret();
let hashed = EventContent::new()
.content(content)
.kind(1)
.stamp(created_at)
.own(sk.public_key())
.hash();
hashed.clone().sign(sk.sign(&hashed.id))
}
fn approx(a: f64, b: f64) -> bool {
(a - b).abs() < 1e-9
}
// --- SearchQuery parsing ---
#[test]
fn parse_splits_on_whitespace() {
let q = SearchQuery::parse("best nostr apps");
assert_eq!(q.terms, vec!["best", "nostr", "apps"]);
}
#[test]
fn parse_treats_extensions_as_terms() {
// We don't interpret NIP-50 extensions; every token is just a term.
let q = SearchQuery::parse("nostr domain:example.com language:en");
assert_eq!(q.terms, vec!["nostr", "domain:example.com", "language:en"]);
}
#[test]
fn parse_is_empty_for_blank_input() {
assert!(SearchQuery::parse(" ").is_empty());
assert!(SearchQuery::parse("").is_empty());
assert!(SearchQuery::new().is_empty());
assert!(!SearchQuery::parse("nostr").is_empty());
}
#[test]
fn display_joins_terms() {
let q = SearchQuery::parse("nostr best apps");
assert_eq!(q.to_string(), "nostr best apps");
}
#[test]
fn display_round_trips_through_parse() {
let q = SearchQuery::parse("nostr best apps language:en");
assert_eq!(SearchQuery::parse(&q.to_string()), q);
}
// --- Scoring ---
#[test]
fn score_full_match_is_one() {
let q = SearchQuery::parse("nostr apps");
assert!(approx(q.score("i love nostr and apps"), 1.0));
}
#[test]
fn score_no_match_is_zero() {
let q = SearchQuery::parse("nostr");
assert!(approx(q.score("no match here"), 0.0));
}
#[test]
fn score_partial_match_is_fraction() {
let q = SearchQuery::parse("nostr apps");
assert!(approx(q.score("only nostr here"), 0.5));
}
#[test]
fn score_is_case_insensitive() {
let q = SearchQuery::parse("NOSTR");
assert!(approx(q.score("the Nostr protocol"), 1.0));
}
#[test]
fn score_extension_like_term_matches_as_text() {
let q = SearchQuery::parse("language:en");
assert!(approx(q.score("posted with language:en today"), 1.0));
assert!(approx(q.score("no marker here"), 0.0));
}
#[test]
fn score_empty_query_is_one() {
assert!(approx(SearchQuery::parse("").score("anything at all"), 1.0));
assert!(approx(SearchQuery::new().score(""), 1.0));
}
#[test]
fn score_frequency_bonus_orders_partial_matches() {
let q = SearchQuery::parse("alpha beta");
let once = q.score("alpha only");
let many = q.score("alpha alpha alpha");
assert!(approx(once, 0.5));
assert!(many > once, "repeated term should score higher: {many} vs {once}");
assert!(many < 1.0, "a partial match must stay below a full match");
}
#[test]
fn score_never_exceeds_one() {
let q = SearchQuery::parse("nostr");
// Heavy repetition of a full match is still capped at 1.0.
assert!(approx(q.score("nostr nostr nostr nostr nostr"), 1.0));
}
#[test]
fn score_is_bounded() {
let q = SearchQuery::parse("alpha beta gamma");
for content in ["", "alpha", "alpha beta", "alpha alpha gamma gamma", "alpha beta gamma"] {
let s = q.score(content);
assert!((0.0..=1.0).contains(&s), "score {s} out of range for {content:?}");
}
}
// --- Filter integration ---
#[test]
fn add_and_clear_search() {
let f = Filter::new().add_search("nostr");
assert_eq!(f.search.as_deref(), Some("nostr"));
assert_eq!(f.clear_search().search, None);
}
#[test]
fn search_score_without_search_is_one() {
let event = make_event("anything", 1);
assert!(approx(Filter::new().search_score(&event), 1.0));
}
#[test]
fn search_score_with_search() {
let event = make_event("the nostr protocol", 1);
assert!(approx(Filter::new().add_search("nostr").search_score(&event), 1.0));
assert!(approx(Filter::new().add_search("missing").search_score(&event), 0.0));
}
#[test]
fn matches_ignores_search() {
// Structural matching and search scoring are independent.
let event = make_event("hello", 1);
let filter = Filter::new().add_kind(1).add_search("not-in-content");
assert!(filter.matches(&event), "matches must not consider the search field");
assert!(approx(filter.search_score(&event), 0.0));
}
// --- Serialization ---
#[test]
fn search_round_trips_through_json() {
let filter = Filter::new().add_kind(1).add_search("best nostr apps");
let json = serde_json::to_string(&filter).unwrap();
assert!(json.contains("\"search\":\"best nostr apps\""));
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(filter, parsed);
}
#[test]
fn search_absent_when_none() {
let json = serde_json::to_string(&Filter::new().add_kind(1)).unwrap();
assert!(!json.contains("search"));
}
// --- Grouping and set algebra ---
#[test]
fn group_distinguishes_searches() {
let f1 = Filter::new().add_kind(1).add_search("alpha");
let f2 = Filter::new().add_kind(1).add_search("beta");
let f3 = Filter::new().add_kind(1).add_search("alpha");
assert_ne!(f1.group(), f2.group());
assert_eq!(f1.group(), f3.group());
}
#[test]
fn union_keeps_different_searches_separate() {
let filters = vec![
Filter::new().add_kind(1).add_search("alpha"),
Filter::new().add_kind(1).add_search("beta"),
];
assert_eq!(union_filters(&filters).len(), 2);
}
#[test]
fn union_merges_same_search() {
let a = SecretKey::generate().public_key();
let b = SecretKey::generate().public_key();
let filters = vec![
Filter::new().add_search("x").add_author(a),
Filter::new().add_search("x").add_author(b),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 1);
assert_eq!(result[0].search.as_deref(), Some("x"));
let authors = result[0].authors.as_ref().unwrap();
assert!(authors.contains(&a) && authors.contains(&b));
}
#[test]
fn intersect_keeps_conflicting_searches_separate() {
let groups = vec![
vec![Filter::new().add_search("nostr")],
vec![Filter::new().add_search("bitcoin")],
];
let result = intersect_filters(&groups);
assert_eq!(result.len(), 2);
let searches: std::collections::BTreeSet<_> =
result.iter().map(|f| f.search.clone()).collect();
assert!(searches.contains(&Some("nostr".to_string())));
assert!(searches.contains(&Some("bitcoin".to_string())));
}
#[test]
fn intersect_combines_when_one_side_has_search() {
let author = SecretKey::generate().public_key();
let groups = vec![
vec![Filter::new().add_author(author)],
vec![Filter::new().add_search("nostr")],
];
let result = intersect_filters(&groups);
assert_eq!(result.len(), 1);
assert_eq!(result[0].search.as_deref(), Some("nostr"));
assert!(result[0].authors.as_ref().unwrap().contains(&author));
}
#[test]
fn intersect_combines_equal_searches() {
let groups = vec![
vec![Filter::new().add_kind(1).add_search("x")],
vec![Filter::new().add_kind(2).add_search("x")],
];
let result = intersect_filters(&groups);
assert_eq!(result.len(), 1);
assert_eq!(result[0].search.as_deref(), Some("x"));
let kinds = result[0].kinds.as_ref().unwrap();
assert!(kinds.contains(&1) && kinds.contains(&2));
}
+121
View File
@@ -0,0 +1,121 @@
use coracle_lib::tags::{Tag, Tags};
#[test]
fn new_tag_accessors() {
let t = Tag::new("e", ["abc", "wss://relay.example", "reply"]);
assert_eq!(t.name(), "e");
assert_eq!(t.value(), "abc");
assert_eq!(t.get(2), Some("wss://relay.example"));
assert_eq!(t.get(3), Some("reply"));
assert_eq!(t.get(4), None);
assert_eq!(t.len(), 4);
assert_eq!(t.values().len(), 3);
assert!(!t.is_empty());
}
#[test]
fn empty_tag_returns_empty_strings() {
let t = Tag(Vec::new());
assert_eq!(t.name(), "");
assert_eq!(t.value(), "");
assert!(t.values().is_empty());
assert!(t.is_empty());
}
#[test]
fn single_entry_tag_has_empty_value() {
let t = Tag::new("-", std::iter::empty::<String>());
assert_eq!(t.name(), "-");
assert_eq!(t.value(), "");
assert_eq!(t.len(), 1);
}
#[test]
fn from_and_into_vec_string() {
let raw = vec!["t".to_string(), "nostr".to_string()];
let tag: Tag = raw.clone().into();
assert_eq!(tag.name(), "t");
let back: Vec<String> = tag.into();
assert_eq!(back, raw);
}
#[test]
fn tag_serde_is_transparent() {
let tag = Tag::new("t", ["nostr"]);
let json = serde_json::to_string(&tag).unwrap();
assert_eq!(json, r#"["t","nostr"]"#);
let parsed: Tag = serde_json::from_str(r#"["p","abcdef"]"#).unwrap();
assert_eq!(parsed.name(), "p");
assert_eq!(parsed.value(), "abcdef");
}
fn sample_tags() -> Tags {
Tags::from(vec![
Tag::new("t", ["nostr"]),
Tag::new("t", ["rust"]),
Tag::new("p", ["abcd"]),
Tag::new("e", ["ffff", "wss://relay.example", "root"]),
])
}
#[test]
fn tags_serde_is_transparent() {
let tags = sample_tags();
let json = serde_json::to_string(&tags).unwrap();
// outer type is just an array of arrays
assert!(json.starts_with("[["));
let parsed: Tags = serde_json::from_str(&json).unwrap();
assert_eq!(parsed, tags);
}
#[test]
fn find_returns_first_match() {
let ts = sample_tags();
let t = ts.find("t").unwrap();
assert_eq!(t.value(), "nostr");
assert!(ts.find("x").is_none());
}
#[test]
fn find_all_iterates_matches() {
let ts = sample_tags();
let found: Vec<&str> = ts.find_all("t").map(Tag::value).collect();
assert_eq!(found, vec!["nostr", "rust"]);
assert_eq!(ts.find_all("p").count(), 1);
assert_eq!(ts.find_all("x").count(), 0);
}
#[test]
fn value_and_values_helpers() {
let ts = sample_tags();
assert_eq!(ts.value("t"), Some("nostr"));
assert_eq!(ts.value("p"), Some("abcd"));
assert_eq!(ts.value("x"), None);
let all_t: Vec<&str> = ts.values("t").collect();
assert_eq!(all_t, vec!["nostr", "rust"]);
}
#[test]
fn has_checks_presence() {
let ts = sample_tags();
assert!(ts.has("p"));
assert!(ts.has("e"));
assert!(!ts.has("q"));
}
#[test]
fn tags_deref_to_slice() {
let ts = sample_tags();
// Deref lets us use slice methods directly.
assert_eq!(ts.len(), 4);
assert_eq!(ts[0].name(), "t");
let names: Vec<&str> = ts.iter().map(Tag::name).collect();
assert_eq!(names, vec!["t", "t", "p", "e"]);
}
#[test]
fn tags_new_is_empty() {
let ts = Tags::new();
assert!(ts.is_empty());
assert_eq!(ts.len(), 0);
}
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "coracle-wasm"
version = "0.1.0"
edition = "2021"
description = "Web bindings for coracle-lib via wasm-bindgen"
[lib]
crate-type = ["cdylib", "rlib"]
[dependencies]
coracle-lib = { path = "../coracle-lib" }
wasm-bindgen = "0.2"
serde-wasm-bindgen = "0.6"
serde_json = "1"
hex = "0.4"
secp256k1 = "0.29"
+266
View File
@@ -0,0 +1,266 @@
//! Web bindings for `coracle-lib` via `wasm-bindgen`.
//!
//! Wraps the core nostr types — keys, NIP-44 encryption, events — in
//! `#[wasm_bindgen]` shims so they can be consumed from JavaScript. The
//! wrappers hold the underlying `coracle-lib` value by composition and
//! translate its errors into `JsError`.
use wasm_bindgen::prelude::*;
use coracle_lib::encryption as ce;
use coracle_lib::events as cev;
use coracle_lib::keys as ck;
fn js_err<E: std::fmt::Display>(e: E) -> JsError {
JsError::new(&e.to_string())
}
// ---------- Keys ----------
/// A nostr public key: the x-only 32-byte secp256k1 coordinate.
#[wasm_bindgen]
pub struct PublicKey(ck::PublicKey);
#[wasm_bindgen]
impl PublicKey {
#[wasm_bindgen(js_name = fromHex)]
pub fn from_hex(s: &str) -> Result<PublicKey, JsError> {
ck::PublicKey::from_hex(s).map(PublicKey).map_err(js_err)
}
#[wasm_bindgen(js_name = fromNpub)]
pub fn from_npub(s: &str) -> Result<PublicKey, JsError> {
ck::PublicKey::from_npub(s).map(PublicKey).map_err(js_err)
}
#[wasm_bindgen(js_name = toHex)]
pub fn to_hex(&self) -> String {
self.0.to_hex()
}
#[wasm_bindgen(js_name = toNpub)]
pub fn to_npub(&self) -> String {
self.0.to_npub()
}
#[wasm_bindgen(js_name = toBytes)]
pub fn to_bytes(&self) -> Vec<u8> {
self.0.as_bytes().to_vec()
}
}
/// A nostr secret key. Exporting the raw bytes is explicit via `toHex` /
/// `toNsec` — there is no implicit string conversion.
#[wasm_bindgen]
pub struct SecretKey(ck::SecretKey);
#[wasm_bindgen]
impl SecretKey {
/// Generate a new secret key from the OS RNG.
#[wasm_bindgen]
pub fn generate() -> SecretKey {
SecretKey(ck::SecretKey::generate())
}
#[wasm_bindgen(js_name = fromHex)]
pub fn from_hex(s: &str) -> Result<SecretKey, JsError> {
ck::SecretKey::from_hex(s).map(SecretKey).map_err(js_err)
}
#[wasm_bindgen(js_name = fromNsec)]
pub fn from_nsec(s: &str) -> Result<SecretKey, JsError> {
ck::SecretKey::from_nsec(s).map(SecretKey).map_err(js_err)
}
/// Decrypt a NIP-49 `ncryptsec1…` string.
#[wasm_bindgen(js_name = fromNcryptsec)]
pub fn from_ncryptsec(s: &str, password: &str) -> Result<SecretKey, JsError> {
ck::SecretKey::from_ncryptsec(s, password)
.map(SecretKey)
.map_err(js_err)
}
#[wasm_bindgen(js_name = toHex)]
pub fn to_hex(&self) -> String {
self.0.to_hex()
}
#[wasm_bindgen(js_name = toNsec)]
pub fn to_nsec(&self) -> String {
self.0.to_nsec()
}
/// Encrypt this key as a NIP-49 `ncryptsec1…` string.
#[wasm_bindgen(js_name = toNcryptsec)]
pub fn to_ncryptsec(
&self,
password: &str,
log_n: u8,
security_byte: u8,
) -> Result<String, JsError> {
self.0
.to_ncryptsec(password, log_n, security_byte)
.map_err(js_err)
}
#[wasm_bindgen(js_name = publicKey)]
pub fn public_key(&self) -> PublicKey {
PublicKey(self.0.public_key())
}
#[wasm_bindgen(js_name = nip44Encrypt)]
pub fn nip44_encrypt(&self, pk: &PublicKey, plaintext: &str) -> Result<String, JsError> {
self.0.nip44_encrypt(&pk.0, plaintext).map_err(js_err)
}
#[wasm_bindgen(js_name = nip44Decrypt)]
pub fn nip44_decrypt(&self, pk: &PublicKey, payload: &str) -> Result<String, JsError> {
self.0.nip44_decrypt(&pk.0, payload).map_err(js_err)
}
}
// ---------- Encryption ----------
/// Raw 32-byte ECDH shared secret between a secret and a public key.
#[wasm_bindgen(js_name = sharedSecret)]
pub fn shared_secret(sk: &SecretKey, pk: &PublicKey) -> Vec<u8> {
ce::shared_secret(&sk.0, &pk.0).to_vec()
}
/// A reusable NIP-44 conversation key. Derive once per `(sk, pk)` pair and
/// hold it when sending many messages to the same counterparty.
#[wasm_bindgen]
pub struct ConversationKey(ce::nip44::ConversationKey);
#[wasm_bindgen]
impl ConversationKey {
#[wasm_bindgen]
pub fn derive(sk: &SecretKey, pk: &PublicKey) -> ConversationKey {
ConversationKey(ce::nip44::ConversationKey::derive(&sk.0, &pk.0))
}
#[wasm_bindgen(js_name = toBytes)]
pub fn to_bytes(&self) -> Vec<u8> {
self.0.as_bytes().to_vec()
}
#[wasm_bindgen]
pub fn encrypt(&self, plaintext: &str) -> Result<String, JsError> {
self.0.encrypt(plaintext).map_err(js_err)
}
#[wasm_bindgen(js_name = encryptWithNonce)]
pub fn encrypt_with_nonce(&self, plaintext: &str, nonce: &[u8]) -> Result<String, JsError> {
let nonce: &[u8; 32] = nonce
.try_into()
.map_err(|_| JsError::new("nonce must be exactly 32 bytes"))?;
self.0.encrypt_with_nonce(plaintext, nonce).map_err(js_err)
}
#[wasm_bindgen]
pub fn decrypt(&self, payload: &str) -> Result<String, JsError> {
self.0.decrypt(payload).map_err(js_err)
}
}
// ---------- Events ----------
/// A signed nostr event.
#[wasm_bindgen]
pub struct Event(cev::Event);
#[wasm_bindgen]
impl Event {
/// Build and sign a new event under `sk`. `tags` must be a JS array of
/// string arrays (`string[][]`).
#[wasm_bindgen(js_name = signNew)]
pub fn sign_new(
kind: u16,
content: &str,
tags: JsValue,
created_at: u64,
sk: &SecretKey,
) -> Result<Event, JsError> {
use coracle_lib::tags::{Tag, Tags};
let raw_tags: Vec<Vec<String>> =
serde_wasm_bindgen::from_value(tags).map_err(|e| JsError::new(&e.to_string()))?;
let tags = Tags::from(
raw_tags
.into_iter()
.map(|v| Tag::from(v))
.collect::<Vec<_>>(),
);
let hashed = cev::EventContent::new()
.content(content)
.tags(tags)
.kind(kind)
.stamp(created_at)
.own(sk.0.public_key())
.hash();
let sig = sk.0.sign(&hashed.id);
Ok(Event(hashed.sign(sig)))
}
/// Parse an event from its JSON representation.
#[wasm_bindgen(js_name = fromJson)]
pub fn from_json(s: &str) -> Result<Event, JsError> {
serde_json::from_str::<cev::Event>(s)
.map(Event)
.map_err(js_err)
}
/// Serialize the event to JSON.
#[wasm_bindgen(js_name = toJson)]
pub fn to_json(&self) -> Result<String, JsError> {
serde_json::to_string(&self.0).map_err(js_err)
}
#[wasm_bindgen(getter)]
pub fn id(&self) -> String {
hex::encode(self.0.id)
}
#[wasm_bindgen(getter)]
pub fn pubkey(&self) -> String {
self.0.pubkey.to_hex()
}
#[wasm_bindgen(getter, js_name = createdAt)]
pub fn created_at(&self) -> u64 {
self.0.created_at
}
#[wasm_bindgen(getter)]
pub fn kind(&self) -> u16 {
self.0.kind
}
#[wasm_bindgen(getter)]
pub fn content(&self) -> String {
self.0.content.clone()
}
#[wasm_bindgen(getter)]
pub fn sig(&self) -> String {
hex::encode(self.0.sig)
}
#[wasm_bindgen(getter)]
pub fn tags(&self) -> Result<JsValue, JsError> {
serde_wasm_bindgen::to_value(&self.0.tags).map_err(|e| JsError::new(&e.to_string()))
}
/// Verify both the id and the Schnorr signature.
#[wasm_bindgen]
pub fn verify(&self) -> bool {
self.0.verify().is_ok()
}
/// Check whether the recomputed id matches the stored id.
#[wasm_bindgen(js_name = verifyId)]
pub fn verify_id(&self) -> bool {
self.0.verify_id()
}
}