coracle/coracle-rust
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add filters chapter

Browse Source
This commit is contained in:
Jon Staab
2026-04-21 12:08:55 -07:00
parent c8f6bc1652
commit a8a57a3d77
6 changed files with 1876 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
book/02-keys.md
+1 -1
View File
@@ -108,7 +108,7 @@ belongs to us rather than a foreign crate.
/// ///
/// This is the "name" half of a nostr identity. It's safe to log, share, and /// This is the "name" half of a nostr identity. It's safe to log, share, and
/// store — it identifies an author but grants no ability to speak as them. /// store — it identifies an author but grants no ability to speak as them.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct PublicKey(secp256k1::XOnlyPublicKey); pub struct PublicKey(secp256k1::XOnlyPublicKey);
impl PublicKey { impl PublicKey {
book/11-filters.md
+820
View File
@@ -0,0 +1,820 @@
# Filters
A filter is a predicate over events. Given an event, a filter answers one
question: does this event match? The protocol uses filters inside `REQ`
messages to tell relays what to send, but the data structure itself is
equally useful for querying a local database, routing events to the right
handler, or deciding which events to display — anywhere you need to select
a subset of events from a larger set.
A single filter expresses a conjunction: every field that is present must
match. An array of filters expresses a disjunction: the event must match
at least one. Between the two, you can describe any positive selection
over the event fields that nostr exposes — ids, authors, kinds, tags,
and time ranges. There is no negation; the protocol rejected it early on
to keep relay implementations simple.
## The module
```rust {file=coracle-lib/src/lib.rs}
pub mod filters;
```
```rust {file=coracle-lib/src/filters.rs}
//! Event filters: the [`Filter`] type for matching events by id, author,
//! kind, tags, and time range, plus utilities for hashing, grouping, and
//! estimating result cardinality.
use std::collections::{BTreeMap, BTreeSet};
use std::fmt;
use serde::de::{self, MapAccess, Visitor};
use serde::ser::SerializeMap;
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use crate::addresses::Address;
use crate::events::Event;
use crate::keys::PublicKey;
```
## The `Filter` struct
Each field corresponds to one axis of selection. The set fields — `ids`,
`authors`, `kinds` — use `Option<BTreeSet<T>>`. The `Option` layer
carries meaning: `None` says "I have no constraint on this field" and
matches everything, while `Some(empty set)` says "the value must be a
member of this empty set" and matches nothing. The distinction matters
for composition — merging two filters into one relies on being able to
tell "don't care" from "impossible."
`BTreeSet` rather than `HashSet` gives two things: O(log n) membership
checks regardless of the key type, and deterministic iteration order for
serialization and hashing.
The `tags` field is a `BTreeMap` from tag name to a set of acceptable
values. Tag names are arbitrary strings — not restricted to single
letters. Single-letter indexing is a relay optimization, not a protocol
constraint, and the filter type should not encode relay policy.
```rust {file=coracle-lib/src/filters.rs}
/// A predicate over nostr events.
///
/// Every present field must match for the filter to match (AND semantics).
/// An array of filters matches if any single filter matches (OR semantics).
///
/// `None` on a set field means "no constraint" — it matches any value.
/// `Some(empty set)` means "must be a member of the empty set" — it
/// matches nothing. This distinction is important for filter composition.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Filter {
/// Event IDs to match.
pub ids: Option<BTreeSet<[u8; 32]>>,
/// Author public keys to match.
pub authors: Option<BTreeSet<PublicKey>>,
/// Event kinds to match.
pub kinds: Option<BTreeSet<u16>>,
/// Tag filters: for each entry, the event must have at least one tag
/// with that name whose value appears in the set.
pub tags: BTreeMap<String, BTreeSet<String>>,
/// Lower bound on `created_at` (inclusive).
pub since: Option<u64>,
/// Upper bound on `created_at` (inclusive).
pub until: Option<u64>,
/// Maximum number of events a consumer should return. This is not a
/// matching criterion — [`matches`](Filter::matches) ignores it.
pub limit: Option<usize>,
}
```
The `limit` field is part of the NIP-01 filter object, so it belongs in
the struct. But it is a result-count constraint for consumers (relays,
storage engines), not a predicate over individual events. The `matches`
method ignores it entirely.
## Matching
Matching walks each present field and returns `false` as soon as one
fails. Scalar checks — ids, kinds, authors — come first because they
are a simple set-membership test and reject most non-matching events
immediately, before the more involved tag iteration.
Tag matching checks every entry in the filter's `tags` map. For each
tag name, the event must contain at least one tag with that name whose
value appears in the filter's set. Within a single tag name the values
are disjunctive (OR): any match suffices. Across tag names the
constraints are conjunctive (AND): all must be satisfied.
```rust {file=coracle-lib/src/filters.rs}
impl Filter {
/// Test whether an event satisfies this filter.
///
/// All present fields must match (AND semantics). The `limit` field
/// is ignored — it is a hint for result-set sizing, not a predicate.
pub fn matches(&self, event: &Event) -> bool {
if let Some(ids) = &self.ids {
if !ids.contains(&event.id) {
return false;
}
}
if let Some(kinds) = &self.kinds {
if !kinds.contains(&event.kind) {
return false;
}
}
if let Some(authors) = &self.authors {
if !authors.contains(&event.pubkey) {
return false;
}
}
for (name, values) in &self.tags {
let has_match = event
.tags
.find_all(name)
.any(|tag| values.contains(tag.value()));
if !has_match {
return false;
}
}
if let Some(since) = self.since {
if event.created_at < since {
return false;
}
}
if let Some(until) = self.until {
if event.created_at > until {
return false;
}
}
true
}
}
```
A convenience free function handles the common case of testing an event
against an array of filters — the OR-across-filters semantics that
NIP-01 defines for `REQ` subscriptions.
```rust {file=coracle-lib/src/filters.rs}
/// Test whether an event matches any filter in the slice.
///
/// Returns `true` if at least one filter matches. An empty slice matches
/// nothing.
pub fn matches_any(filters: &[Filter], event: &Event) -> bool {
filters.iter().any(|f| f.matches(event))
}
```
## Construction
An empty filter matches everything — no constraints means no rejections.
The `add_*` methods insert values into the constraint sets, consuming and
returning `self` so calls can be chained. Matching `remove_*` methods
take values back out, and `clear_*` methods reset a field to `None` —
removing the constraint entirely.
```rust {file=coracle-lib/src/filters.rs}
impl Filter {
/// Create an empty filter that matches every event.
pub fn new() -> Self {
Filter {
ids: None,
authors: None,
kinds: None,
tags: BTreeMap::new(),
since: None,
until: None,
limit: None,
}
}
/// Add an event id to the constraint set.
pub fn add_id(mut self, id: [u8; 32]) -> Self {
self.ids.get_or_insert_with(BTreeSet::new).insert(id);
self
}
/// Add multiple event ids to the constraint set.
pub fn add_ids(mut self, ids: impl IntoIterator<Item = [u8; 32]>) -> Self {
self.ids.get_or_insert_with(BTreeSet::new).extend(ids);
self
}
/// Remove an event id from the constraint set. If the set becomes
/// empty, it remains as `Some(empty)` — use `clear_ids` to remove
/// the constraint entirely.
pub fn remove_id(mut self, id: &[u8; 32]) -> Self {
if let Some(ids) = &mut self.ids {
ids.remove(id);
}
self
}
/// Remove the ids constraint, matching any event id.
pub fn clear_ids(mut self) -> Self {
self.ids = None;
self
}
/// Add an author to the constraint set.
pub fn add_author(mut self, author: PublicKey) -> Self {
self.authors
.get_or_insert_with(BTreeSet::new)
.insert(author);
self
}
/// Add multiple authors to the constraint set.
pub fn add_authors(mut self, authors: impl IntoIterator<Item = PublicKey>) -> Self {
self.authors
.get_or_insert_with(BTreeSet::new)
.extend(authors);
self
}
/// Remove an author from the constraint set.
pub fn remove_author(mut self, author: &PublicKey) -> Self {
if let Some(authors) = &mut self.authors {
authors.remove(author);
}
self
}
/// Remove the authors constraint, matching any author.
pub fn clear_authors(mut self) -> Self {
self.authors = None;
self
}
/// Add a kind to the constraint set.
pub fn add_kind(mut self, kind: u16) -> Self {
self.kinds.get_or_insert_with(BTreeSet::new).insert(kind);
self
}
/// Add multiple kinds to the constraint set.
pub fn add_kinds(mut self, kinds: impl IntoIterator<Item = u16>) -> Self {
self.kinds.get_or_insert_with(BTreeSet::new).extend(kinds);
self
}
/// Remove a kind from the constraint set.
pub fn remove_kind(mut self, kind: &u16) -> Self {
if let Some(kinds) = &mut self.kinds {
kinds.remove(kind);
}
self
}
/// Remove the kinds constraint, matching any kind.
pub fn clear_kinds(mut self) -> Self {
self.kinds = None;
self
}
/// Add a value to a tag filter: the event must have at least one tag
/// with this name whose value appears in the set.
pub fn add_tag(mut self, name: impl Into<String>, value: impl Into<String>) -> Self {
self.tags
.entry(name.into())
.or_default()
.insert(value.into());
self
}
/// Add multiple values to a tag filter.
pub fn add_tags(
mut self,
name: impl Into<String>,
values: impl IntoIterator<Item = impl Into<String>>,
) -> Self {
self.tags
.entry(name.into())
.or_default()
.extend(values.into_iter().map(Into::into));
self
}
/// Remove a value from a tag filter. If the value set becomes empty,
/// the tag entry is removed from the map.
pub fn remove_tag(mut self, name: &str, value: &str) -> Self {
if let Some(values) = self.tags.get_mut(name) {
values.remove(value);
if values.is_empty() {
self.tags.remove(name);
}
}
self
}
/// Remove an entire tag filter by name.
pub fn clear_tag(mut self, name: &str) -> Self {
self.tags.remove(name);
self
}
/// Remove all tag filters.
pub fn clear_tags(mut self) -> Self {
self.tags.clear();
self
}
/// Set the lower bound on `created_at` (inclusive).
pub fn add_since(mut self, since: u64) -> Self {
self.since = Some(since);
self
}
/// Remove the lower bound on `created_at`.
pub fn clear_since(mut self) -> Self {
self.since = None;
self
}
/// Set the upper bound on `created_at` (inclusive).
pub fn add_until(mut self, until: u64) -> Self {
self.until = Some(until);
self
}
/// Remove the upper bound on `created_at`.
pub fn clear_until(mut self) -> Self {
self.until = None;
self
}
/// Set the result-count limit.
pub fn add_limit(mut self, limit: usize) -> Self {
self.limit = Some(limit);
self
}
/// Remove the result-count limit.
pub fn clear_limit(mut self) -> Self {
self.limit = None;
self
}
}
```
### Address convenience
Filtering for an addressable event by its address is common enough — and
error-prone enough when done by hand — to warrant a dedicated method.
An address carries a kind, an author, and an identifier; the method
translates these into the corresponding filter fields.
```rust {file=coracle-lib/src/filters.rs}
impl Filter {
/// Add constraints that match events at the given address.
///
/// Sets the kind, author, and `d` tag filter from the address's
/// components. If the identifier is empty (plain replaceable events),
/// the `d` tag filter is still set — the event must have a `d` tag
/// with an empty value.
pub fn add_address(self, addr: &Address) -> Self {
self.add_kind(addr.kind)
.add_author(addr.pubkey)
.add_tag("d", &addr.identifier)
}
}
impl Default for Filter {
fn default() -> Self {
Filter::new()
}
}
```
A filter built with `.add_address()` looks like this:
```rust
use coracle_lib::filters::Filter;
use coracle_lib::addresses::Address;
let addr: Address = "30023:ab12...cd34:my-article".parse().unwrap();
let filter = Filter::new().add_address(&addr);
// Equivalent to:
// Filter::new().add_kind(30023).add_author(pubkey).add_tag("d", "my-article")
```
## Serialization
The NIP-01 wire format for a filter is a flat JSON object where tag
filters appear as keys prefixed with `#`:
```json
{
"kinds": [1],
"authors": ["ab12...cd34"],
"#t": ["nostr", "rust"],
"since": 1700000000,
"limit": 10
}
```
The `tags` map in our struct needs to be flattened into the top-level
object during serialization and reconstituted during deserialization.
This rules out `#[derive(Serialize, Deserialize)]` — we need a hand-
written implementation.
```rust {file=coracle-lib/src/filters.rs}
impl Serialize for Filter {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
// Count present fields for the map size hint.
let mut count = self.tags.len();
if self.ids.is_some() {
count += 1;
}
if self.authors.is_some() {
count += 1;
}
if self.kinds.is_some() {
count += 1;
}
if self.since.is_some() {
count += 1;
}
if self.until.is_some() {
count += 1;
}
if self.limit.is_some() {
count += 1;
}
let mut map = serializer.serialize_map(Some(count))?;
if let Some(ids) = &self.ids {
let hex_ids: Vec<String> = ids.iter().map(hex::encode).collect();
map.serialize_entry("ids", &hex_ids)?;
}
if let Some(authors) = &self.authors {
let hex_authors: Vec<String> = authors.iter().map(|pk| pk.to_hex()).collect();
map.serialize_entry("authors", &hex_authors)?;
}
if let Some(kinds) = &self.kinds {
let kinds_vec: Vec<u16> = kinds.iter().copied().collect();
map.serialize_entry("kinds", &kinds_vec)?;
}
for (name, values) in &self.tags {
let key = format!("#{name}");
let vals: Vec<&str> = values.iter().map(String::as_str).collect();
map.serialize_entry(&key, &vals)?;
}
if let Some(since) = self.since {
map.serialize_entry("since", &since)?;
}
if let Some(until) = self.until {
map.serialize_entry("until", &until)?;
}
if let Some(limit) = self.limit {
map.serialize_entry("limit", &limit)?;
}
map.end()
}
}
```
Deserialization collects known keys into their fields and routes any key
starting with `#` into the `tags` map. Unknown keys are silently
ignored for forward compatibility.
```rust {file=coracle-lib/src/filters.rs}
impl<'de> Deserialize<'de> for Filter {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
deserializer.deserialize_map(FilterVisitor)
}
}
struct FilterVisitor;
impl<'de> Visitor<'de> for FilterVisitor {
type Value = Filter;
fn expecting(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("a nostr filter object")
}
fn visit_map<M: MapAccess<'de>>(self, mut map: M) -> Result<Filter, M::Error> {
let mut ids: Option<BTreeSet<[u8; 32]>> = None;
let mut authors: Option<BTreeSet<PublicKey>> = None;
let mut kinds: Option<BTreeSet<u16>> = None;
let mut tags: BTreeMap<String, BTreeSet<String>> = BTreeMap::new();
let mut since: Option<u64> = None;
let mut until: Option<u64> = None;
let mut limit: Option<usize> = None;
while let Some(key) = map.next_key::<String>()? {
match key.as_str() {
"ids" => {
let hex_ids: Vec<String> = map.next_value()?;
let mut set = BTreeSet::new();
for h in hex_ids {
let bytes = hex::decode(&h)
.map_err(|_| de::Error::custom("invalid hex in ids"))?;
let arr: [u8; 32] = bytes
.try_into()
.map_err(|_| de::Error::custom("id must be 32 bytes"))?;
set.insert(arr);
}
ids = Some(set);
}
"authors" => {
let hex_authors: Vec<String> = map.next_value()?;
let mut set = BTreeSet::new();
for h in hex_authors {
let pk = PublicKey::from_hex(&h)
.map_err(|_| de::Error::custom("invalid pubkey in authors"))?;
set.insert(pk);
}
authors = Some(set);
}
"kinds" => {
let kind_vec: Vec<u16> = map.next_value()?;
kinds = Some(kind_vec.into_iter().collect());
}
"since" => since = Some(map.next_value()?),
"until" => until = Some(map.next_value()?),
"limit" => limit = Some(map.next_value()?),
other if other.starts_with('#') => {
let tag_name = other[1..].to_string();
let values: Vec<String> = map.next_value()?;
tags.insert(tag_name, values.into_iter().collect());
}
_ => {
let _: de::IgnoredAny = map.next_value()?;
}
}
}
Ok(Filter {
ids,
authors,
kinds,
tags,
since,
until,
limit,
})
}
}
```
A round-trip through JSON preserves all fields:
```rust
use coracle_lib::filters::Filter;
let filter = Filter::new()
.add_kind(1)
.add_tag("t", "nostr")
.add_since(1_700_000_000)
.add_limit(10);
let json = serde_json::to_string(&filter).unwrap();
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(filter, parsed);
```
## Identity and grouping
Two methods support deduplication and merging at higher layers —
subscription managers, relay pools, and storage engines that need to
detect redundant or combinable filters.
`id` produces a deterministic hash of the entire filter. Two filters
with the same id are structurally identical. Since `Filter` derives
`Hash`, we feed it through a standard `Hasher` — no need to round-trip
through JSON.
```rust {file=coracle-lib/src/filters.rs}
use std::hash::{Hash, Hasher};
use std::collections::hash_map::DefaultHasher;
impl Filter {
/// Compute a deterministic identifier for this filter.
///
/// Two filters with the same id are structurally identical. The id
/// is derived from the `Hash` implementation, which covers every
/// field including `limit`.
pub fn id(&self) -> u64 {
let mut hasher = DefaultHasher::new();
self.hash(&mut hasher);
hasher.finish()
}
}
```
`group` determines which filters can be merged by unioning their
set fields. Two filters can only be merged if they have the same
structural shape (which set fields are present, which tag names appear)
*and* the same scalar constraints (time windows). A filter with a
`limit` can never be merged — combining two limited queries into one
would change the result semantics — so each limited filter gets a
unique group key.
```rust {file=coracle-lib/src/filters.rs}
static GROUP_COUNTER: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
impl Filter {
/// Compute a group key that determines merge compatibility.
///
/// Filters in the same group can be merged by unioning their set
/// fields (`ids`, `authors`, `kinds`, tag values). The group key
/// captures:
///
/// - Which set fields are present and which tag names appear
/// (structural shape)
/// - The exact `since` and `until` values (different time windows
/// cannot be combined)
///
/// A filter with a `limit` always gets a unique group key, because
/// merging limited filters would change result-count semantics.
pub fn group(&self) -> u64 {
let mut hasher = DefaultHasher::new();
self.ids.is_some().hash(&mut hasher);
self.authors.is_some().hash(&mut hasher);
self.kinds.is_some().hash(&mut hasher);
for name in self.tags.keys() {
name.hash(&mut hasher);
}
self.since.hash(&mut hasher);
self.until.hash(&mut hasher);
if self.limit.is_some() {
// Each limited filter gets a unique group — merging two
// limited queries into one would change which events are
// returned.
GROUP_COUNTER
.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
.hash(&mut hasher);
}
hasher.finish()
}
}
```
Including tag names in the group key means that a filter on `#e` tags
and a filter on `#p` tags land in different groups — as they should,
since merging them by union would change the semantics. Likewise, two
filters with different `since` or `until` values land in different
groups, because a union of their sets under one time window would either
over-fetch or under-fetch relative to what was requested.
## Union and intersection
Two operations combine filters in different ways.
`union_filters` takes a list of filters and merges those that share
the same group — same structural shape, same time window, no limit.
Within each group it unions the set fields: ids, authors, kinds, and
tag values. The result is a shorter list that matches the same events
as the original but with fewer individual filters to evaluate.
```rust {file=coracle-lib/src/filters.rs}
/// Merge compatible filters by unioning their set fields.
///
/// Filters with the same [`group`](Filter::group) are combined into a
/// single filter whose set fields are the union of the originals. The
/// result matches the same events as the input but with fewer filters.
pub fn union_filters(filters: &[Filter]) -> Vec<Filter> {
let mut groups: BTreeMap<u64, Filter> = BTreeMap::new();
for filter in filters {
let key = filter.group();
groups
.entry(key)
.and_modify(|existing| {
merge_sets(&mut existing.ids, &filter.ids);
merge_sets(&mut existing.authors, &filter.authors);
merge_sets(&mut existing.kinds, &filter.kinds);
for (name, values) in &filter.tags {
existing
.tags
.entry(name.clone())
.or_default()
.extend(values.iter().cloned());
}
})
.or_insert_with(|| filter.clone());
}
groups.into_values().collect()
}
fn merge_sets<T: Ord + Clone>(
target: &mut Option<BTreeSet<T>>,
source: &Option<BTreeSet<T>>,
) {
match (target.as_mut(), source) {
(Some(t), Some(s)) => t.extend(s.iter().cloned()),
_ => {}
}
}
```
`intersect_filters` takes multiple groups of filters — each group
representing one independent query — and produces the set of filters
that satisfies all groups simultaneously. It does this by computing
the cartesian product across groups, combining each pair by unioning
their set fields and tightening their time windows: the latest `since`,
the earliest `until`. Finally it passes the result through
`union_filters` to collapse any redundancy.
```rust {file=coracle-lib/src/filters.rs}
/// Combine independent filter groups into filters that satisfy all of
/// them.
///
/// Each inner `Vec<Filter>` represents one group of alternatives (OR).
/// The result matches events that satisfy at least one filter from
/// *every* group (AND across groups, OR within each group).
///
/// Set fields are unioned. Time windows are tightened: the latest
/// `since` and earliest `until` win. If both filters have a `limit`,
/// the larger one is kept. The result is simplified with
/// [`union_filters`].
pub fn intersect_filters(groups: &[Vec<Filter>]) -> Vec<Filter> {
let Some(first) = groups.first() else {
return vec![];
};
let mut result: Vec<Filter> = first.clone();
for filters in &groups[1..] {
let mut combined = Vec::with_capacity(result.len() * filters.len());
for f1 in &result {
for f2 in filters {
combined.push(combine_pair(f1, f2));
}
}
result = combined;
}
union_filters(&result)
}
fn combine_pair(a: &Filter, b: &Filter) -> Filter {
let mut f = Filter::new();
f.ids = union_option_sets(&a.ids, &b.ids);
f.authors = union_option_sets(&a.authors, &b.authors);
f.kinds = union_option_sets(&a.kinds, &b.kinds);
for (name, values) in a.tags.iter().chain(b.tags.iter()) {
f.tags
.entry(name.clone())
.or_default()
.extend(values.iter().cloned());
}
f.since = match (a.since, b.since) {
(Some(a), Some(b)) => Some(a.max(b)),
(s, None) | (None, s) => s,
};
f.until = match (a.until, b.until) {
(Some(a), Some(b)) => Some(a.min(b)),
(u, None) | (None, u) => u,
};
f.limit = match (a.limit, b.limit) {
(Some(a), Some(b)) => Some(a.max(b)),
(l, None) | (None, l) => l,
};
f
}
fn union_option_sets<T: Ord + Clone>(
a: &Option<BTreeSet<T>>,
b: &Option<BTreeSet<T>>,
) -> Option<BTreeSet<T>> {
match (a, b) {
(Some(a), Some(b)) => {
let mut merged = a.clone();
merged.extend(b.iter().cloned());
Some(merged)
}
(Some(s), None) | (None, Some(s)) => Some(s.clone()),
(None, None) => None,
}
}
```
## What's next
The next chapter extends filters with NIP-50 full-text search — an
optional `search` field that some relays support for content-based
queries.
book/SUMMARY.md
+39 -35
View File
@@ -11,53 +11,57 @@
- [Kinds](06-kinds.md) - [Kinds](06-kinds.md)
- [Addresses](07-addresses.md) - [Addresses](07-addresses.md)
- [Proof of Work](08-proof-of-work.md) - [Proof of Work](08-proof-of-work.md)
- [Filters](09-filters.md) - [Expiring Events](09-expiring-events.md)
- [Protected Events](10-protected-events.md)
- [Filters](11-filters.md)
- [Search](12-search.md)
## Domain ## Domain
- [Relay Selections](10-relay-selections.md) - [Relay Selections](13-relay-selections.md)
- [Relay Metadata](11-relay-metadata.md) - [Relay Metadata](14-relay-metadata.md)
- [Relay Membership](12-relay-membership.md) - [Relay Membership](15-relay-membership.md)
- [Profiles](13-profiles.md) - [Profiles](16-profiles.md)
- [Follows](14-follows.md) - [Follows](17-follows.md)
- [Microblogging](15-microblogging.md) - [Microblogging](18-microblogging.md)
- [Reactions](16-reactions.md) - [Reactions](19-reactions.md)
- [Reports](17-reports.md) - [Reports](20-reports.md)
- [Emojis](18-emojis.md) - [Emojis](21-emojis.md)
- [Zaps](19-zaps.md) - [Zaps](22-zaps.md)
- [Rooms](20-rooms.md) - [Rooms](23-rooms.md)
- [Open Timestamp Attestations](24-open-timestamp-attestations.md)
## Networking ## Networking
- [Relay Connections](21-relay-connections.md) - [Relay Connections](25-relay-connections.md)
- [Relay Authentication](22-relay-authentication.md) - [Relay Authentication](26-relay-authentication.md)
- [Relay Policies](23-relay-policies.md) - [Relay Policies](27-relay-policies.md)
- [Server Authentication](24-server-authentication.md) - [Server Authentication](28-server-authentication.md)
- [Relay Management API](25-relay-management-api.md) - [Relay Management API](29-relay-management-api.md)
- [Blossom Media Storage](26-blossom-media-storage.md) - [Blossom Media Storage](30-blossom-media-storage.md)
## Signers ## Signers
- [Signer Interface](27-signer-interface.md) - [Signer Interface](31-signer-interface.md)
- [Secret Signers](28-secret-signers.md) - [Secret Signers](32-secret-signers.md)
- [Remote Signers](29-remote-signers.md) - [Remote Signers](33-remote-signers.md)
- [Android Signers](30-android-signers.md) - [Android Signers](34-android-signers.md)
- [Browser Signers](31-browser-signers.md) - [Browser Signers](35-browser-signers.md)
## Content ## Content
- [Entities](32-entities.md) - [Entities](36-entities.md)
- [Relays](33-relays.md) - [Relays](37-relays.md)
- [Rooms](34-rooms.md) - [Rooms](38-rooms.md)
- [Links](35-links.md) - [Links](39-links.md)
- [Lightning](36-lightning.md) - [Lightning](40-lightning.md)
- [Cashu](37-cashu.md) - [Cashu](41-cashu.md)
- [Emojis](38-emojis.md) - [Emojis](42-emojis.md)
- [Topics](39-topics.md) - [Topics](43-topics.md)
- [Code](40-code.md) - [Code](44-code.md)
## Storage ## Storage
- [Event Repository](41-event-repository.md) - [Event Repository](45-event-repository.md)
- [In Memory Backend](42-in-memory-backend.md) - [In Memory Backend](46-in-memory-backend.md)
- [Sqlite Backend](43-sqlite-backend.md) - [Sqlite Backend](47-sqlite-backend.md)
book/plan/filters.md
+207
View File
@@ -0,0 +1,207 @@
# Plan: Filters
## Topic Summary
Filters are the NIP-01 data structure for matching events. They form an elegant primitive for
matching events independent of the client/relay context — not just for REQ messages, but as a
general-purpose event matching and querying abstraction. The chapter covers the filter structure,
matching semantics (AND within a filter, OR across filters), tag filters, timestamp constraints,
limits, construction, hashing, grouping, and cardinality estimation.
## Chapter Outline
1. **Introduction** — Filters as a general-purpose event matching primitive. Not tied to relays;
they're a predicate you can evaluate against any event. Analogy to database WHERE clauses.
2. **The Filter Struct** — Walk through the fields:
- `ids: Option<BTreeSet<[u8; 32]>>` — match event IDs
- `authors: Option<BTreeSet<PublicKey>>` — match event authors
- `kinds: Option<BTreeSet<u16>>` — match event kinds
- `tags: BTreeMap<String, BTreeSet<String>>` — match tag values by tag name
- `since: Option<u64>` — lower bound on `created_at` (inclusive)
- `until: Option<u64>` — upper bound on `created_at` (inclusive)
- `limit: Option<usize>` — result count constraint (not a matching criterion)
Explain `Option` semantics: `None` = no constraint, `Some(empty set)` = matches nothing.
Note that `limit` is metadata for consumers, not part of matching logic.
3. **Matching** — Implement `matches(&self, event: &Event) -> bool`:
- AND semantics: all present fields must match
- Early exit on scalar checks (ids, kinds, authors) before tag matching
- Tag matching: for each tag filter, event must have at least one tag with that name
whose value is in the filter's set (OR within a tag filter, AND across tag filters)
- Timestamp: `since <= created_at <= until`
- `limit` is ignored
- Implement `matches_any(filters: &[Filter], event: &Event) -> bool` as a free function
for OR-across-filters semantics
4. **Construction** — Builder pattern with fluent API:
- `Filter::new()` — empty filter (matches everything)
- `.id(id)` / `.ids(iter)` — add event IDs
- `.author(pk)` / `.authors(iter)` — add authors
- `.kind(k)` / `.kinds(iter)` — add kinds
- `.tag(name, value)` / `.tags(name, iter)` — add arbitrary tag filters
- `.since(ts)` / `.until(ts)` — set timestamp bounds
- `.limit(n)` — set result limit
- `.address(addr)` — convenience: sets kind, author, and `#d` tag from an Address
5. **Serialization** — Custom serde implementation:
- Standard fields serialize normally, skip `None` fields
- `tags` BTreeMap flattened: key `"foo"` becomes JSON key `"#foo"` with array value
- Handle `limit: 0` vs omitted limit (Some(0) serializes as `"limit": 0`)
- Deserialization: any key starting with `#` collected into `tags` map
- Show round-trip example
6. **Identity and Grouping** — Utilities for deduplication and merging:
- `filter_id(filter) -> String` — deterministic hash of filter contents for dedup
- `filter_group(filter) -> String` — hash of structural fields only (ids, kinds, authors,
tag keys) excluding values and temporal fields. Two filters in the same group can be
merged by unioning their value sets.
7. **Cardinality** — `cardinality(&self) -> Option<usize>`:
- Returns `Some(n)` when the maximum number of matching events can be determined
- `ids` present → `ids.len()`
- All kinds are replaceable + `authors` present → `authors.len() * kinds.len()`
- All kinds are addressable + `authors` present + `#d` present →
`authors.len() * kinds.len() * d_values.len()`
- Otherwise → `None` (unbounded)
- If explicit `limit` is set, return `min(limit, computed)` when computed is Some,
or `Some(limit)` when computed is None
- Empty set in any field → `Some(0)`
8. **Recap** — Summarize filter as a composable primitive. Tease usage in relay connections
chapter.
## API Design
```rust
// --- Filter struct ---
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Filter {
pub ids: Option<BTreeSet<[u8; 32]>>,
pub authors: Option<BTreeSet<PublicKey>>,
pub kinds: Option<BTreeSet<u16>>,
pub since: Option<u64>,
pub until: Option<u64>,
pub limit: Option<usize>,
// Flattened in serde as #key -> [values]
pub tags: BTreeMap<String, BTreeSet<String>>,
}
// --- Construction (builder, consuming self) ---
impl Filter {
pub fn new() -> Self
pub fn id(self, id: [u8; 32]) -> Self
pub fn ids(self, ids: impl IntoIterator<Item = [u8; 32]>) -> Self
pub fn author(self, author: PublicKey) -> Self
pub fn authors(self, authors: impl IntoIterator<Item = PublicKey>) -> Self
pub fn kind(self, kind: u16) -> Self
pub fn kinds(self, kinds: impl IntoIterator<Item = u16>) -> Self
pub fn tag(self, name: impl Into<String>, value: impl Into<String>) -> Self
pub fn tags(self, name: impl Into<String>, values: impl IntoIterator<Item = impl Into<String>>) -> Self
pub fn since(self, since: u64) -> Self
pub fn until(self, until: u64) -> Self
pub fn limit(self, limit: usize) -> Self
pub fn address(self, addr: &Address) -> Self
}
// --- Matching ---
impl Filter {
pub fn matches(&self, event: &Event) -> bool
pub fn cardinality(&self) -> Option<usize>
}
pub fn matches_any(filters: &[Filter], event: &Event) -> bool
// --- Identity and grouping ---
pub fn filter_id(filter: &Filter) -> String
pub fn filter_group(filter: &Filter) -> String
```
## Code Organization
All code in `coracle-lib/src/filters.rs`. Single file, single module. Add `pub mod filters;`
to `coracle-lib/src/lib.rs`.
## Dependencies
- `serde` / `serde_json` — already used in the events chapter for serialization
- `std::collections::BTreeSet` / `BTreeMap` — stdlib, no external crate
- `sha2` — already used in events chapter for hashing; reuse for filter_id
No new external dependencies needed.
## Narrative Notes
- Open by framing filters as a standalone primitive. They're a predicate, not a protocol
message. The fact that relays use them in REQ is one application, but they're equally
useful for client-side filtering, local storage queries, and event routing decisions.
- The `Option` semantics deserve careful explanation. Show the difference:
`None` = "I don't care about this field" vs `Some(empty)` = "this field must match
one of these zero values (i.e., nothing matches)". This is the key insight that makes
filters composable.
- When explaining matching, walk through a concrete example: construct a filter, show an
event, trace through the matching logic field by field.
- For tag filters, emphasize that tag keys are arbitrary strings — not restricted to
single letters. The single-letter convention is a relay indexing optimization, not a
protocol constraint.
- `limit` gets a brief note: it's not part of matching. It tells a consumer (relay, storage
engine) how many results to return. Include it in the struct because it's part of the
NIP-01 filter object, but `matches()` ignores it.
- For serialization, the interesting part is the tag flattening. Show the JSON representation
and explain how `tags: {"e": {"abc"}, "p": {"def"}}` becomes `{"#e": ["abc"], "#p": ["def"]}`.
- `filter_id` and `filter_group` are utility functions, not methods, because they serve
infrastructure concerns (dedup, subscription management) rather than core filter semantics.
- `cardinality` leverages kind classification from the kinds chapter. Connect the dots:
replaceable events have at most one per author per kind, addressable events have at most
one per author per kind per identifier.
## Design Decisions
1. **`Option<BTreeSet<T>>` for set fields** — Preserves the None-vs-empty distinction that
NIP-01 requires. BTreeSet gives O(log n) membership checks and deterministic iteration
order for serialization/hashing. (Research: rust-nostr uses this approach.)
2. **Arbitrary string tag keys** — Not restricted to single letters. The protocol allows any
tag name; single-letter indexing is a relay optimization. Consumers can enforce restrictions.
3. **Minimal builder API** — `.id()`, `.author()`, `.kind()`, `.tag()`, `.address()` plus
plural variants. No convenience methods for every common tag (#e, #p, #t, etc.) — the
generic `.tag("e", value)` is clear enough. Keeps the chapter focused.
4. **`limit` in struct but not in matching** — NIP-01 defines it as part of the filter object,
so it belongs in the struct. But it's a result constraint, not a predicate, so `matches()`
ignores it. (Research: NDK, nostr-tools, all implementations agree on this.)
5. **Free functions for identity/grouping** — `filter_id` and `filter_group` are not methods
because they serve infrastructure concerns. Keeps the Filter impl block focused on
construction and matching.
6. **`cardinality` returns `Option<usize>`** — `None` means unbounded. Leverages kind
classification (replaceable, addressable) to compute tight upper bounds when possible.
(Research: nostr-tools' `getFilterLimit`, nostrlib's `GetTheoreticalLimit`.)
7. **Custom serde for tag flattening** — Tags serialize as `#name` keys at the top level of
the JSON object, matching the NIP-01 wire format. This requires custom Serialize/Deserialize
implementations rather than derive macros.
8. **`.address()` convenience** — Translates an Address into the correct combination of kind,
author, and #d tag filter. This is the one domain-aware convenience method because
address-based filtering is extremely common and error-prone to construct manually.
## Open Questions
- Should `filter_group` include tag *names* (keys) in the group hash, or only the set of
field names that are present? Including tag names means `{#e: [...]}` and `{#p: [...]}`
are in different groups (correct for merging). Leaning toward including tag names.
book/research/filters.md
+241
View File
@@ -0,0 +1,241 @@
# Research: Filters
## Topic Summary
Filters are the NIP-01 data structure for matching events. They form an elegant primitive for
matching events independent of the client/relay context — not just for REQ messages, but as a
general-purpose event matching and querying abstraction. The chapter should cover the filter
structure, matching semantics (AND within a filter, OR across filters), tag filters, timestamp
constraints, limits, and programmatic construction/manipulation of filters.
## Philosophy
From `ref/building-nostr`:
**Filters as a conceptual category for tags**: The author identifies "filter tags" as one of
three tag categories (alongside data and behavior tags). Filter tags are "especially useful for
filtering and retrieval" and tend to be single-letter because relays only index single-letter
tags to reduce database overhead.
**Filter structure (NIP-01)**: Standard fields are `ids`, `authors`, `kinds`, `since`, `until`,
`limit`. Tag filters are created by prefixing tag names with `#` (e.g., `#p`, `#e`). Extensions
include prefix matching and NIP-50 `search`. Negative matches were proposed but rejected due to
relay performance concerns.
**Filters as a routing problem**: "Where to send a given filter" is distinct from "where to send
a given event." Filters have less information than events, making routing harder. A routing
heuristic "connects a filter that might be constructed to support a particular use case with the
relay where matching events are stored." This is analogous to database indexes.
**Design principles**:
- **Minimalism**: Filters match discrete criteria without complex negation or boolean logic
- **Decentralization**: Clients understand routing heuristics; relays don't need to understand intent
- **Extensibility**: The `#tag` convention allows arbitrary new filters without protocol changes
- **Trade-offs**: Single-letter indexing limits expressiveness but maintains relay scalability
- **Partition tolerance**: Missing a relay means missing some events, which is acceptable
## Reference Implementation Analysis
### applesauce
**Types**: `Filter` extends nostr-tools' `CoreFilter` with NIP-91 AND operator support (`&`-prefixed
tag names) and NIP-50 `search` field. All standard NIP-01 fields present.
**Matching**: `matchFilter(filter, event)` checks basic fields first for early rejection, then
uses `getIndexableTags(event)` to build a cached `Set<string>` of `"tagName:value"` pairs on the
event (via Symbol key). NIP-91 AND tags processed before OR tags. `matchFilters` implements OR
across filter arrays.
**Utilities**: `mergeFilters(...filters)` unions array fields and deduplicates; takes minimum
`limit`, minimum `since`, maximum `until`. `isFilterEqual` uses `fast-deep-equal` for subscription
deduplication. `createFilterMap` distributes filters across relays by author.
**SQL layer**: Separate `buildFilterConditions` translates filters to SQL WHERE clauses. AND tags
use `GROUP BY`/`HAVING COUNT` subqueries. OR tags use `IN` subqueries.
**Patterns**: Symbol-based memoization of tag indexes on event objects. Early exit optimization.
Only indexes single-letter tags. RxJS integration for streaming filter results.
### ndk
**Types**: `NDKFilter<K extends NDKKind>` — generic parameterized type with all NIP-01 fields plus
`search`. Dynamic tag properties via `[key: #${string}]`.
**Matching**: `matchFilter(filter, event)` uses `indexOf()` for array membership. Special case:
`#t` (hashtag) tags are case-insensitive. Does NOT check `limit` or `search` — these are
submission constraints, not matching criteria. Short-circuits on first mismatch.
**Utilities**:
- `mergeFilters()` — unions arrays, deduplicates via Set. Preserves filters with `limit` separately (limits can't be merged). Returns array.
- `filterFingerprint()` — deterministic hash of filter structure for subscription grouping
- `compareFilter()` — checks if one filter is a subset of another (for cache-hit validation)
- `filterFromId()` — converts bech32 identifiers to filters
- `filterForEventsTaggingId()` — creates tag filters for events referencing a given ID
**Validation**: Three modes (VALIDATE/FIX/IGNORE). Checks for undefined values, type correctness,
hex format, kind range. Guardrails catch common mistakes: empty filters, bech32 in hex arrays,
`since > until`, `#t` with literal `#` prefix.
**Patterns**: Generic type parameterization. Pluggable validation modes. Readable subscription IDs
generated from filter structure.
### nostr-gadgets
Uses `nostr-tools` filter types and functions directly — no custom filter implementation.
**Construction patterns**: Mutable accumulation (`filter.authors?.push(target)`), inline literals,
spread-based composition (`{ ...f, authors: [pubkey], since: newest }`).
**Filter-based deletion**: Converts event tags to filter arrays for batch deletion operations.
**Multi-level filtering**: Filters used at query construction, relay permission checking (purgatory),
and client-side event matching.
### nostrlib
**Types**: Go struct with `IDs []ID`, `Kinds []Kind`, `Authors []PubKey`, `Tags TagMap`,
`Since Timestamp`, `Until Timestamp`, `Limit int`, `Search string`, `LimitZero bool`. Uses
fixed-size byte arrays for IDs/PubKeys.
**Matching**: Two methods:
- `Matches(event)` — full matching including timestamp constraints
- `MatchesIgnoringTimestampConstraints(event)` — `[//go:inline]` optimized, used for live events after EOSE
Tag matching via `tags.ContainsAny(tagName, values)`. Uses `slices.Contains()` for array membership.
**Utilities**: `Clone()` deep copies. `FilterEqual()` order-independent comparison. `GetTheoreticalLimit()`
estimates max results considering replaceability. No merging functions.
**Serialization**: Custom easyjson codec with `xhex` for fast hex encoding. `LimitZero` bool
distinguishes `"limit": 0` from omitted limit.
**Patterns**: Pure data structure with stateless methods. Subscription switches matching function
after EOSE. Query optimizer scores tags by "goodness" for index selection.
### nostr-tools
**Types**: Simple TypeScript type with all NIP-01 fields. Index signature `[key: #${string}]` for
dynamic tag filters. All properties optional.
**Matching**: `matchFilter(filter, event)` — conjunctive (AND) matching. Uses `indexOf()` for
membership. Iterates filter properties for `#`-prefixed tag filters. Both `since` and `until` are
inclusive. `matchFilters` implements OR across array.
**Utilities**:
- `mergeFilters(...filters)` — unions array properties, takes max `limit`, min `since`, max `until`
- `getFilterLimit(filter)` — computes intrinsic limit considering replaceability:
- Empty arrays → 0
- IDs → `ids.length`
- Replaceable kinds → `authors.length * kinds.length`
- Addressable kinds → `authors.length * kinds.length * #d.length`
- Returns minimum across all applicable constraints
**Patterns**: Minimalist, functional, no external dependencies for filter logic. Pure JavaScript.
Early exit on mismatch. Kind classification integration for limit calculation.
**Design**: Self-contained. No validation. `search` field defined but not used in matching logic.
### rust-nostr
**Types**: `Filter` struct with `Option<BTreeSet<T>>` for all set fields. Uses `BTreeSet` for
O(log n) lookups and deterministic serialization. `generic_tags: BTreeMap<SingleLetterTag, BTreeSet<String>>`
for dynamic tag filters.
**Matching**: `match_event(&self, event, opts)` with `MatchEventOptions` controlling which fields
to check (7 boolean flags). Individual match methods are `#[inline]`. Tag matching uses lazy-initialized
`event.tags.indexes()` (OnceCell pattern). NIP-50 search: case-insensitive substring via `.windows()`.
**Builder pattern**: Fluent chainable methods consuming `self`: `Filter::new().kind(k).author(pk)`.
Convenience methods for common tags: `.event()`, `.pubkey()`, `.hashtag()`, `.identifier()`,
`.coordinate()`. Generic `.custom_tag()` for arbitrary tags. Remove methods return `None` if set
becomes empty.
**Option semantics**: `None` = no constraint (matches all). `Some(empty_set)` = matches nothing.
This distinction is explicitly documented (GitHub issue #302).
**Utilities**: `is_empty()`, `extract_public_keys()`. No merging/combining API — multiple filters
handled at protocol layer.
**Patterns**: no_std compatible (uses `alloc`). BTreeSet for deterministic ordering. Custom serde
with `#[serde(flatten)]` for generic tags.
### welshman
**Types**: Standard NIP-01 filter type. `neverFilter = {ids: []}` constant for "matches nothing."
**Matching**: Delegates to nostr-tools for NIP-01 matching. Extends with search: splits by
whitespace, case-insensitive, requires ALL terms match (AND logic).
**Utilities**:
- `getFilterId(filter)` — deterministic hash for deduplication (sort keys, join, hash)
- `calculateFilterGroup(filter)` — groups by matching space (structural fields vs temporal)
- `unionFilters(filters)` — groups by `calculateFilterGroup`, merges arrays within groups
- `intersectFilters(groups)` — Cartesian product across filter groups with intelligent merging (max `since`, min `until`, max `limit`, concatenate `search`)
- `getIdFilters(idsOrAddresses)` — converts mix of IDs and addresses to filters
- `getReplyFilters(events)` — generates filters for replies (#e for regular, #a for replaceable)
- `addRepostFilters(filters)` — adds repost kind variants
- `trimFilter(filter)` — caps array fields at 1000 items with random sampling
- `getFilterGenerality()` — heuristic score 0 (specific) to 1 (general)
**Patterns**: Functional composition. Immutable transformations. Hash-based deduplication.
Domain-driven builders for common query patterns.
## Common Patterns
1. **Type structure**: All implementations use optional fields. Missing = no constraint. The `#tag`
convention for dynamic tag filters is universal.
2. **AND/OR semantics**: Universal agreement — AND within a single filter, OR across an array of
filters. This is fundamental to NIP-01.
3. **Matching order**: Most implementations check scalar fields first (ids, kinds, authors) for
early exit before the more expensive tag matching.
4. **Tag indexing**: Several implementations build cached indexes on events for efficient repeated
matching (applesauce: Symbol-based Set cache; rust-nostr: OnceCell BTreeMap).
5. **No negation**: No implementation supports negative matching (NOT). This aligns with protocol
design — rejected for relay performance reasons.
6. **Limit semantics**: `limit` is not a matching criterion — it's a result count constraint.
Most matching functions ignore it. `getFilterLimit`/`GetTheoreticalLimit` computes intrinsic
upper bounds based on kind replaceability.
7. **Merging**: Most implementations provide union-style filter merging. Array fields are unioned
and deduplicated. Scalar fields use min/max logic.
8. **Option vs empty**: rust-nostr explicitly distinguishes `None` (no constraint) from
`Some(empty)` (matches nothing). Other implementations handle this implicitly.
## Considerations for Our Implementation
1. **Filter as a standalone primitive**: Frame filters independent of REQ messages. They're a
general-purpose matching predicate over events.
2. **Struct design**: Use `Option<BTreeSet<T>>` following rust-nostr's approach — it correctly
models the distinction between "no constraint" and "empty constraint." BTreeSet gives
deterministic serialization and O(log n) lookups.
3. **Matching function**: Implement `matches(&self, event: &Event) -> bool` with early exit on
scalar fields. Tag matching should use event tag indexes.
4. **Builder pattern**: Fluent API for construction: `Filter::new().kind(1).author(pk)`.
Convenience methods for common tags (#e, #p, #d, #a, #t).
5. **Generic tag filters**: Support arbitrary single-letter tag filters via
`BTreeMap<SingleLetterTag, BTreeSet<String>>` or similar.
6. **Serialization**: Custom JSON serialization to flatten generic tags as `#tag` keys. Handle
`limit: 0` vs omitted limit.
7. **No merging in core**: Following rust-nostr, keep the filter primitive simple. Merging and
combining can live in higher-level utilities if needed.
8. **Limit calculation**: Consider `getFilterLimit`-style intrinsic limit computation based on
kind replaceability — useful for query optimization.
9. **Dependencies**: Filter should depend only on existing types (Event, EventId, Pubkey, Kind,
Timestamp, Tags). Self-contained within coracle-lib.
10. **Test strategy**: Test matching logic thoroughly — all field types, AND semantics, tag
matching, timestamp boundaries, empty sets vs None, edge cases.
coracle-lib/tests/filters.rs
+568
View File
@@ -0,0 +1,568 @@
use std::collections::BTreeSet;
use coracle_lib::events::{Event, EventContent};
use coracle_lib::filters::{intersect_filters, matches_any, union_filters, Filter};
use coracle_lib::keys::SecretKey;
use coracle_lib::tags::Tag;
fn fixed_secret() -> SecretKey {
let bytes: [u8; 32] = [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
25, 26, 27, 28, 29, 30, 31, 32,
];
SecretKey::from_hex(&hex::encode(bytes)).unwrap()
}
fn make_event(kind: u16, timestamp: u64, tags: Vec<Tag>) -> Event {
let sk = fixed_secret();
let hashed = EventContent::new("hello", tags)
.kind(kind)
.stamp(timestamp)
.own(sk.public_key())
.hash();
hashed.clone().sign(sk.sign(&hashed.id))
}
// --- Matching ---
#[test]
fn empty_filter_matches_everything() {
let event = make_event(1, 1_700_000_000, vec![]);
assert!(Filter::new().matches(&event));
}
#[test]
fn filter_matches_by_id() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter::new().add_id(event.id);
assert!(filter.matches(&event));
}
#[test]
fn filter_rejects_wrong_id() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter::new().add_id([0u8; 32]);
assert!(!filter.matches(&event));
}
#[test]
fn filter_matches_by_kind() {
let event = make_event(1, 1_700_000_000, vec![]);
assert!(Filter::new().add_kind(1).matches(&event));
assert!(!Filter::new().add_kind(2).matches(&event));
}
#[test]
fn filter_matches_by_author() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter::new().add_author(fixed_secret().public_key());
assert!(filter.matches(&event));
}
#[test]
fn filter_rejects_wrong_author() {
let event = make_event(1, 1_700_000_000, vec![]);
let other_sk = SecretKey::generate();
let filter = Filter::new().add_author(other_sk.public_key());
assert!(!filter.matches(&event));
}
#[test]
fn filter_matches_by_tag() {
let event = make_event(
1,
1_700_000_000,
vec![Tag::new("t", ["nostr"]), Tag::new("p", ["abc123"])],
);
assert!(Filter::new().add_tag("t", "nostr").matches(&event));
assert!(!Filter::new().add_tag("e", "something").matches(&event));
assert!(!Filter::new().add_tag("t", "bitcoin").matches(&event));
}
#[test]
fn filter_tag_or_within_same_name() {
let event = make_event(1, 1_700_000_000, vec![Tag::new("t", ["nostr"])]);
let filter = Filter::new().add_tags("t", ["nostr", "bitcoin"]);
assert!(filter.matches(&event));
}
#[test]
fn filter_tag_and_across_names() {
let event = make_event(
1,
1_700_000_000,
vec![Tag::new("t", ["nostr"]), Tag::new("p", ["abc"])],
);
let filter = Filter::new().add_tag("t", "nostr").add_tag("p", "abc");
assert!(filter.matches(&event));
let filter = Filter::new().add_tag("t", "nostr").add_tag("e", "missing");
assert!(!filter.matches(&event));
}
#[test]
fn filter_since_inclusive() {
let event = make_event(1, 1000, vec![]);
assert!(Filter::new().add_since(1000).matches(&event));
assert!(Filter::new().add_since(999).matches(&event));
assert!(!Filter::new().add_since(1001).matches(&event));
}
#[test]
fn filter_until_inclusive() {
let event = make_event(1, 1000, vec![]);
assert!(Filter::new().add_until(1000).matches(&event));
assert!(Filter::new().add_until(1001).matches(&event));
assert!(!Filter::new().add_until(999).matches(&event));
}
#[test]
fn filter_limit_ignored_in_matching() {
let event = make_event(1, 1_700_000_000, vec![]);
assert!(Filter::new().add_limit(0).matches(&event));
}
#[test]
fn filter_and_semantics() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter::new()
.add_kind(1)
.add_author(fixed_secret().public_key());
assert!(filter.matches(&event));
let filter = Filter::new()
.add_kind(2)
.add_author(fixed_secret().public_key());
assert!(!filter.matches(&event));
}
#[test]
fn empty_set_matches_nothing() {
let event = make_event(1, 1_700_000_000, vec![]);
let filter = Filter {
ids: Some(BTreeSet::new()),
..Filter::new()
};
assert!(!filter.matches(&event));
let filter = Filter {
kinds: Some(BTreeSet::new()),
..Filter::new()
};
assert!(!filter.matches(&event));
}
// --- matches_any ---
#[test]
fn matches_any_or_semantics() {
let event = make_event(1, 1_700_000_000, vec![]);
let filters = vec![Filter::new().add_kind(2), Filter::new().add_kind(1)];
assert!(matches_any(&filters, &event));
}
#[test]
fn matches_any_empty_slice() {
let event = make_event(1, 1_700_000_000, vec![]);
assert!(!matches_any(&[], &event));
}
// --- Construction: add ---
#[test]
fn builder_chaining() {
let pk = fixed_secret().public_key();
let filter = Filter::new()
.add_kind(1)
.add_kinds([2, 3])
.add_author(pk)
.add_tag("t", "nostr")
.add_since(1000)
.add_until(2000)
.add_limit(10);
assert_eq!(filter.kinds.as_ref().unwrap().len(), 3);
assert!(filter.kinds.as_ref().unwrap().contains(&1));
assert!(filter.kinds.as_ref().unwrap().contains(&2));
assert!(filter.kinds.as_ref().unwrap().contains(&3));
assert_eq!(filter.since, Some(1000));
assert_eq!(filter.until, Some(2000));
assert_eq!(filter.limit, Some(10));
}
#[test]
fn address_convenience() {
let pk = fixed_secret().public_key();
let addr = coracle_lib::addresses::Address::new(30023, pk, "my-article");
let filter = Filter::new().add_address(&addr);
assert!(filter.kinds.as_ref().unwrap().contains(&30023));
assert!(filter.authors.as_ref().unwrap().contains(&pk));
assert!(filter.tags.get("d").unwrap().contains("my-article"));
}
// --- Construction: remove ---
#[test]
fn remove_id() {
let id = [1u8; 32];
let filter = Filter::new().add_id(id).add_id([2u8; 32]).remove_id(&id);
assert!(!filter.ids.as_ref().unwrap().contains(&id));
assert!(filter.ids.as_ref().unwrap().contains(&[2u8; 32]));
}
#[test]
fn remove_author() {
let pk = fixed_secret().public_key();
let filter = Filter::new().add_author(pk).remove_author(&pk);
// Set exists but is empty — matches nothing
assert!(filter.authors.as_ref().unwrap().is_empty());
}
#[test]
fn remove_kind() {
let filter = Filter::new()
.add_kinds([1, 2, 3])
.remove_kind(&2);
let kinds = filter.kinds.as_ref().unwrap();
assert!(kinds.contains(&1));
assert!(!kinds.contains(&2));
assert!(kinds.contains(&3));
}
#[test]
fn remove_tag_value() {
let filter = Filter::new()
.add_tags("t", ["nostr", "bitcoin"])
.remove_tag("t", "bitcoin");
let values = filter.tags.get("t").unwrap();
assert!(values.contains("nostr"));
assert!(!values.contains("bitcoin"));
}
#[test]
fn remove_tag_last_value_removes_entry() {
let filter = Filter::new()
.add_tag("t", "nostr")
.remove_tag("t", "nostr");
assert!(!filter.tags.contains_key("t"));
}
#[test]
fn remove_on_none_is_noop() {
// Removing from a field that was never set should not panic
let filter = Filter::new()
.remove_id(&[0u8; 32])
.remove_kind(&1)
.remove_author(&fixed_secret().public_key())
.remove_tag("t", "nostr");
assert!(filter.ids.is_none());
assert!(filter.kinds.is_none());
assert!(filter.authors.is_none());
}
// --- Construction: clear ---
#[test]
fn clear_ids() {
let filter = Filter::new().add_id([1u8; 32]).clear_ids();
assert!(filter.ids.is_none());
}
#[test]
fn clear_authors() {
let filter = Filter::new()
.add_author(fixed_secret().public_key())
.clear_authors();
assert!(filter.authors.is_none());
}
#[test]
fn clear_kinds() {
let filter = Filter::new().add_kind(1).clear_kinds();
assert!(filter.kinds.is_none());
}
#[test]
fn clear_tag() {
let filter = Filter::new()
.add_tag("t", "nostr")
.add_tag("e", "abc")
.clear_tag("t");
assert!(!filter.tags.contains_key("t"));
assert!(filter.tags.contains_key("e"));
}
#[test]
fn clear_tags() {
let filter = Filter::new()
.add_tag("t", "nostr")
.add_tag("e", "abc")
.clear_tags();
assert!(filter.tags.is_empty());
}
#[test]
fn clear_since() {
let filter = Filter::new().add_since(1000).clear_since();
assert!(filter.since.is_none());
}
#[test]
fn clear_until() {
let filter = Filter::new().add_until(2000).clear_until();
assert!(filter.until.is_none());
}
#[test]
fn clear_limit() {
let filter = Filter::new().add_limit(10).clear_limit();
assert!(filter.limit.is_none());
}
// --- Serialization ---
#[test]
fn json_round_trip() {
let pk = fixed_secret().public_key();
let filter = Filter::new()
.add_kind(1)
.add_author(pk)
.add_tag("t", "nostr")
.add_tag("e", "abc123")
.add_since(1000)
.add_until(2000)
.add_limit(10);
let json = serde_json::to_string(&filter).unwrap();
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(filter, parsed);
}
#[test]
fn json_tag_flattening() {
let filter = Filter::new().add_tag("t", "nostr");
let json = serde_json::to_string(&filter).unwrap();
assert!(json.contains("\"#t\""));
assert!(json.contains("\"nostr\""));
}
#[test]
fn json_empty_filter() {
let filter = Filter::new();
let json = serde_json::to_string(&filter).unwrap();
assert_eq!(json, "{}");
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(filter, parsed);
}
#[test]
fn json_limit_zero() {
let filter = Filter::new().add_limit(0);
let json = serde_json::to_string(&filter).unwrap();
assert!(json.contains("\"limit\":0"));
let parsed: Filter = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.limit, Some(0));
}
#[test]
fn json_ids_as_hex() {
let filter = Filter::new().add_id([0xab; 32]);
let json = serde_json::to_string(&filter).unwrap();
assert!(json.contains(&hex::encode([0xab; 32])));
}
#[test]
fn json_ignores_unknown_keys() {
let json = r#"{"kinds": [1], "unknown_field": true}"#;
let filter: Filter = serde_json::from_str(json).unwrap();
assert_eq!(filter.kinds, Some(BTreeSet::from([1])));
}
// --- Identity and grouping ---
#[test]
fn filter_id_deterministic() {
let f1 = Filter::new().add_kind(1).add_tag("t", "nostr");
let f2 = Filter::new().add_kind(1).add_tag("t", "nostr");
assert_eq!(f1.id(), f2.id());
}
#[test]
fn filter_id_differs_for_different_filters() {
let f1 = Filter::new().add_kind(1);
let f2 = Filter::new().add_kind(2);
assert_ne!(f1.id(), f2.id());
}
#[test]
fn filter_group_same_shape() {
let f1 = Filter::new().add_kind(1).add_tag("t", "nostr");
let f2 = Filter::new().add_kind(2).add_tag("t", "bitcoin");
assert_eq!(f1.group(), f2.group());
}
#[test]
fn filter_group_different_shape() {
let f1 = Filter::new().add_kind(1).add_tag("t", "nostr");
let f2 = Filter::new().add_kind(1).add_tag("e", "abc");
assert_ne!(f1.group(), f2.group());
}
#[test]
fn group_key_same_time_window_mergeable() {
let f1 = Filter::new().add_kind(1).add_since(1000).add_until(2000);
let f2 = Filter::new().add_kind(2).add_since(1000).add_until(2000);
assert_eq!(f1.group(), f2.group());
}
#[test]
fn group_key_different_time_window_not_mergeable() {
let f1 = Filter::new().add_kind(1).add_since(1000);
let f2 = Filter::new().add_kind(1).add_since(2000);
assert_ne!(f1.group(), f2.group());
}
#[test]
fn group_key_limit_always_unique() {
let f1 = Filter::new().add_kind(1).add_limit(10);
let f2 = Filter::new().add_kind(1).add_limit(10);
// Even identical limits produce different group keys
assert_ne!(f1.group(), f2.group());
}
#[test]
fn group_key_no_limit_is_mergeable() {
let f1 = Filter::new().add_kind(1);
let f2 = Filter::new().add_kind(2);
assert_eq!(f1.group(), f2.group());
}
// --- union_filters ---
#[test]
fn union_merges_same_shape() {
let pk1 = fixed_secret().public_key();
let pk2 = SecretKey::generate().public_key();
let filters = vec![
Filter::new().add_kind(1).add_author(pk1),
Filter::new().add_kind(1).add_author(pk2),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 1);
let merged = &result[0];
assert!(merged.authors.as_ref().unwrap().contains(&pk1));
assert!(merged.authors.as_ref().unwrap().contains(&pk2));
assert!(merged.kinds.as_ref().unwrap().contains(&1));
}
#[test]
fn union_keeps_different_shapes_separate() {
let filters = vec![
Filter::new().add_kind(1),
Filter::new().add_kind(1).add_tag("t", "nostr"),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 2);
}
#[test]
fn union_keeps_different_time_windows_separate() {
let filters = vec![
Filter::new().add_kind(1).add_since(1000),
Filter::new().add_kind(2).add_since(2000),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 2);
}
#[test]
fn union_keeps_limited_filters_separate() {
let filters = vec![
Filter::new().add_kind(1).add_limit(10),
Filter::new().add_kind(2).add_limit(10),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 2);
}
#[test]
fn union_empty_input() {
let result = union_filters(&[]);
assert!(result.is_empty());
}
#[test]
fn union_merges_tag_values() {
let filters = vec![
Filter::new().add_tag("t", "nostr"),
Filter::new().add_tag("t", "bitcoin"),
];
let result = union_filters(&filters);
assert_eq!(result.len(), 1);
let values = result[0].tags.get("t").unwrap();
assert!(values.contains("nostr"));
assert!(values.contains("bitcoin"));
}
// --- intersect_filters ---
#[test]
fn intersect_cartesian_product() {
let groups = vec![
vec![Filter::new().add_kind(1), Filter::new().add_kind(2)],
vec![Filter::new().add_tag("t", "nostr")],
];
let result = intersect_filters(&groups);
// Two combinations merged by union into one: kinds(1,2)+tag(t)
assert_eq!(result.len(), 1);
assert!(result[0].kinds.as_ref().unwrap().contains(&1));
assert!(result[0].kinds.as_ref().unwrap().contains(&2));
assert!(result[0].tags.contains_key("t"));
}
#[test]
fn intersect_incompatible_shapes_stay_separate() {
let groups = vec![
vec![Filter::new().add_kind(1)],
vec![
Filter::new().add_tag("t", "nostr"),
Filter::new().add_tag("e", "abc"),
],
];
let result = intersect_filters(&groups);
// kind+#t and kind+#e are different shapes, can't merge
assert_eq!(result.len(), 2);
}
#[test]
fn intersect_tightens_time_window() {
let groups = vec![
vec![Filter::new().add_since(1000).add_until(3000)],
vec![Filter::new().add_since(2000).add_until(4000)],
];
let result = intersect_filters(&groups);
assert_eq!(result.len(), 1);
assert_eq!(result[0].since, Some(2000)); // max
assert_eq!(result[0].until, Some(3000)); // min
}
#[test]
fn intersect_empty_groups() {
let result = intersect_filters(&[]);
assert!(result.is_empty());
}
#[test]
fn intersect_single_group_passthrough() {
let groups = vec![vec![
Filter::new().add_kind(1),
Filter::new().add_kind(2),
]];
let result = intersect_filters(&groups);
// Should union the two compatible filters
assert_eq!(result.len(), 1);
assert!(result[0].kinds.as_ref().unwrap().contains(&1));
assert!(result[0].kinds.as_ref().unwrap().contains(&2));
}