From 2e0740910cf1230ff8ee5219e73b933e7d0d051e Mon Sep 17 00:00:00 2001 From: Jon Staab Date: Tue, 24 Mar 2026 14:04:49 -0700 Subject: [PATCH] Create backend spec --- backend/BILLING.md | 94 ------------------------------- backend/spec/api.md | 120 ++++++++++++++++++++++++++++++++++++++++ backend/spec/billing.md | 50 +++++++++++++++++ backend/spec/infra.md | 24 ++++++++ backend/spec/main.md | 7 +++ backend/spec/models.md | 88 +++++++++++++++++++++++++++++ backend/spec/repo.md | 110 ++++++++++++++++++++++++++++++++++++ backend/spec/robot.md | 26 +++++++++ 8 files changed, 425 insertions(+), 94 deletions(-) delete mode 100644 backend/BILLING.md create mode 100644 backend/spec/api.md create mode 100644 backend/spec/billing.md create mode 100644 backend/spec/infra.md create mode 100644 backend/spec/main.md create mode 100644 backend/spec/models.md create mode 100644 backend/spec/repo.md create mode 100644 backend/spec/robot.md diff --git a/backend/BILLING.md b/backend/BILLING.md deleted file mode 100644 index 11c6d16..0000000 --- a/backend/BILLING.md +++ /dev/null @@ -1,94 +0,0 @@ -# Billing Architecture (Agreed) - -This document summarizes the agreed billing architecture for Caravel backend. - -## Billing model - -- Usage-based billing: **sats/hour** for relay operation. -- A relay is billable when it is provisioned/active in lifecycle terms. -- Billing is **monthly**, with a **rolling cycle anchored to tenant signup**. -- One **consolidated invoice per tenant** per billing period. - -## Metering and lifecycle - -- Add an append-only lifecycle event table in the backend database. -- Events are the source for usage computation. -- Canonical event timestamp field name: `created_at` (UTC). -- Lifecycle behavior is treated as a state machine for billing math (idempotent outcomes for repeated/no-op transitions). -- Transition validation is permissive (any transition can be recorded); billing logic interprets sequences. -- Billable time behavior: - - Start on `provisioned` - - Pause on `suspended` - - Stop on `deactivated` - - Resume immediately on unsuspend - -## Pricing - -- Price is per relay plan/tier in **sats/hour**. -- Rates are stored in a mutable `plans` table (current rate only). -- Mid-cycle plan changes are billed by time spent in each plan. -- Plan rate changes are retroactive for un-invoiced usage in the current open period. - -## Rounding and minimums - -- Round usage up to the next full hour. -- Minimum charge: **1 billable hour per relay per month**. - -## Invoice generation - -- A periodic worker creates invoices at billing boundaries. -- Existing relays at launch start billing from launch timestamp only (no historical backfill). -- Avoid duplicate invoices with a DB unique constraint on: - - `(tenant, period_start, period_end)` - -## Invoice status and attempts - -- `invoice_attempts` is the canonical history/state source. -- `invoices.status` is a synchronous projection updated in the same transaction as attempt writes. -- Each payment method attempt is its own row in `invoice_attempts`. -- Attempts in a single retry pass share a `run_id` UUID. - -## Collection order and fallback - -For each invoice collection run: - -1. Try **NWC** auto-pay -2. If not paid, try **Stripe** auto-pay -3. If still unpaid/unavailable, create Lightning invoice and show QR in-app -4. If neither NWC nor Stripe is configured, send a one-time **NIP-17 DM** with invoice/subscription status - -Notes: - -- Retry cadence: every 24 hours (NWC/Stripe retries). -- Do **not** resend DMs on retries. -- Lightning invoice refresh is in-app only when prior invoice expires. -- DM send is recorded as an `invoice_attempts` row (same `run_id` as triggering run). - -## Due dates, grace, and enforcement - -- Invoice due time is derived as: `invoice.created_at + 7 days`. -- Grace period: 7 days, relay service remains fully active during grace. -- If still unpaid after grace, billing flow marks tenant/account past due and performs billing-side handling. -- Full outstanding balance must be paid before billing status is considered clear. - -## Tenant and integration storage - -- Store billing cycle anchor on `tenants` (e.g., `billing_anchor_at`). -- Anchor can be reset when tenant goes from no non-free relays to having one again. -- Determine “has billable relays” by querying relays on demand (no counter cache). -- Keep NWC config in `tenants.nwc_url`. -- Store Stripe IDs directly on `tenants`. - -## Worker and runtime model - -- Scheduler runs inside backend service process. -- Multiple instances may run; correctness relies on DB idempotency and unique constraints. - -## Repository impact - -- Add migration(s) for lifecycle events and billing-related schema changes. -- Add repository methods in `backend/src/repo.rs` for: - - writing lifecycle events - - reading lifecycle events by relay/tenant/time window - - creating/fetching invoices with period boundaries - - writing invoice attempts and projecting invoice status atomically diff --git a/backend/spec/api.md b/backend/spec/api.md new file mode 100644 index 0000000..cfaf4e6 --- /dev/null +++ b/backend/spec/api.md @@ -0,0 +1,120 @@ +# `pub struct Api` + +Api manages the HTTP interface for the application + +Members: + +- `host: String` - the hostname of the service for checking NIP 98 auth, from `HOST` +- `port: u16` - a port to run the server on from `PORT` +- `admins: Vec` - a list of admin pubkeys from `ADMINS` +- `origins: Vec` - to be used in CORS headers, from `ALLOW_ORIGINS` +- `repo: Repo` + +Notes: + +- Authentication is done using NIP 98 +- Each route is responsible for authorization using `self.is_admin(pubkey)` or `self.is_tenant(authorized_pubkey, tenant_pubkey)` +- Successful API responses should be of the form `{data, code: "ok"}` with an appropriate http status code. +- Unsuccessful API responses should be of the form `{error, code}` with an appropriate http status code. `code` is a short error code (e.g. `duplicate-subdomain`) and `error` is a human-readable error message. + +## `pub fn new() -> Self` + +- Reads environment and populates members + +## `pub fn serve(&self) -> Result<()>` + +- Initializes an `axum::Router` +- Adds CORS middleware based on `origins` +- Calls `axum::serve` with a listener + +--- Tenant routes + +## `async fn list_tenants(...) -> Response` + +- Serves `GET /tenants` +- Authorizes admin only +- Return `data` is a list of tenant structs from `repo.list_tenants` + +## `async fn get_tenant(...) -> Response` + +- Serves `GET /tenants/:pubkey` +- Authorizes admin or matching tenant +- Return `data` is a single tenant struct from `repo.get_tenant` + +## `async fn create_tenant(...) -> Response` + +- Serves `POST /tenants` +- Authorizes anyone, but must be authorized +- Creates a new tenant using `repo.create_tenant` based on the authorized pubkey +- If tenant is a duplicate, return a `422` with `code=pubkey-exists` +- Return `data` is a single tenant struct. Use HTTP `201`. + +--- Relay routes + +## `async fn list_relays(...) -> Response` + +- Serves `GET /relays?tenant=` +- Authorizes admin or existing tenants +- If user is admin, `tenant` query parameter is optional +- If user is a tenant, `tenant` query parameter is not ok; authenticated `pubkey` is used +- Return `data` is a list of relay structs from `repo.list_relays` + +## `async fn get_relay(...) -> Response` + +- Serves `GET /relays/:id` +- Authorizes admin or relay owner +- Return `data` is a single relay struct from `repo.get_relay` + +## `async fn create_relay(...) -> Response` + +- Serves `POST /relays` +- Authorizes admin or matching tenant pubkey in request body +- Validates/prepares the relay data to be saved using `prepare_relay` +- Creates a new relay using `repo.create_relay` +- If relay is a duplicate by subdomain, return a `422` with `code=subdomain-exists` +- Return `data` is a single relay struct. Use HTTP `201`. + +## `async fn update_relay(...) -> Response` + +- Serves `PUT /relays/:id` +- Authorizes admin or relay owner +- Validates/prepares the relay data to be saved using `prepare_relay` +- Updates the given relay using `repo.update_relay` +- If relay is a duplicate by subdomain, return a `422` with `code=subdomain-exists` +- Return `data` is a single relay struct. + +## `async fn deactivate_relay(...) -> Response` + +- Serves `POST /relays/:id/deactivate` +- Authorizes admin or relay owner +- Deactivates relay using `repo.deactivate_relay` +- Return `data` is empty + +--- Billing routes + +## `async fn list_invoices(...) -> Response` + +- Serves `GET /invoices?tenant=` +- Authorizes admin or existing tenants +- If user is admin, `tenant` query parameter is optional +- If user is a tenant, `tenant` query parameter is not ok; authenticated `pubkey` is used +- Return `data` is a list of invoice structs from `repo.list_invoices` + +# Utility functions + +## `extract_auth_pubkey(headers: &HeaderMap, method: &Method, uri: &Uri) -> Result` + +- Parses `Authorization` header +- Validates event kind and signature using `nostr_sdk` +- Validates event `u` and `method` tags against parameters +- Returns pubkey if header is valid + +Refer to https://github.com/nostr-protocol/nips/blob/master/98.md for details. Use `nostr_sdk` functionality where possible. + +## `prepare_relay(relay: Relay) -> anyhow::Result` + +- Validate `subdomain` +- If `plan` is free and `blossom` is enabled, return `premium-feature` +- If `plan` is free and `livekit` is enabled, return `premium-feature` +- Populate `schema` if not already set +- Populate missing fields using reasonable defaults diff --git a/backend/spec/billing.md b/backend/spec/billing.md new file mode 100644 index 0000000..777b889 --- /dev/null +++ b/backend/spec/billing.md @@ -0,0 +1,50 @@ +# `pub struct Billing` + +Billing is a service which polls the database, creates invoices, and attempts to collect payment for invoices. + +Members: + +- `nwc_url: String` - a nostr wallet connect URL used to **create** invoices +- `repo: Repo` +- `robot: Robot` + +## `pub fn new(repo: Repo, robot: Robot) -> Self` + +- Reads environment and populates members + +## `pub async fn start(self)` + +Calls `self.tick` in a loop every hour. + +## `pub async fn tick(self)` + +Iterates over `repo.list_activity` since last run and does the following: + +- For any `relay_created|relay_updated` activity if this is the first non-free relay for the tenant, update tenant's billing anchor to the time the relay was created. + +Also iterates over `repo.list_tenants()` and for each tenant calls `self.generate_invoice_if_due(tenant)` and `self.collect_outstanding(tenant)`. + +## `async fn generate_invoice_if_due(&self, tenant: &Tenant)` + +- Skip tenants that have a `pending` invoice or have no active non-free relays. +- Compute current billing period from `tenant.billing_anchor` as rolling monthly windows: `[period_start, period_end)`. +- Only generate an invoice once the period has closed (`now >= period_end`). +- Load activity needed to compute usage for the tenant using `repo.list_activity`. +- Calculate how many hours (rounded up) each relay was active during the window per paid plan. +- If total invoice amount is 0, return. +- Generate a `bolt11` invoice. If this fails, log the error and return. +- Generate an invoice for the tenant and an invoice item for each relay. +- Persist invoice + items atomically using `repo.create_invoice`. + +## `async fn collect_outstanding(&self, tenant: &Tenant)` + +- Load `pending` tenant invoices and attempt to collect each one. +- If `attempted_at` is less than 24 hours ago, skip it. +- If the `bolt11` invoice has been paid out of band, call `repo.mark_invoice_paid` and return. +- If the tenant has a `nwc_url`, attempt to pay the invoice with nwc. +- If collection succeeds, call `repo.mark_invoice_paid`. +- If collection fails, populate `repo.mark_invoice_attempted`. +- If nwc isn't set up or fails and `sent_at` is not set: + - Send a NIP 17 DM to the user with the invoice included. + - Call `repo.mark_invoice_sent`. +- If the invoice is 7 days past `created_at`, call `repo.mark_invoice_closed`. diff --git a/backend/spec/infra.md b/backend/spec/infra.md new file mode 100644 index 0000000..d65d70f --- /dev/null +++ b/backend/spec/infra.md @@ -0,0 +1,24 @@ +# `pub struct Infra` + +Infra is a service which polls the database and synchronizes updates to relays to a remote zooid instance via `api_url`. + +Members: + +- `api_url: String` - the URL of the zooid instance to be managed, from `ZOOID_API_URL` +- `repo: Repo` + +## `pub fn new(repo: Repo) -> Self` + +- Reads environment and populates members + +## `pub async fn start(self)` + +Calls `self.tick` in a loop every 10 seconds. + +## `pub async fn tick(self)` + +Iterates over `repo.list_activity` since last run and does the following: + +- For any `relay_created|relay_updated` activity, sync relay config to zooid. +- For any `relay_deactivated` activity, sync relay config to zooid. +- If unsuccessful, call `repo.fail_relay_sync`. diff --git a/backend/spec/main.md b/backend/spec/main.md new file mode 100644 index 0000000..21aa9c6 --- /dev/null +++ b/backend/spec/main.md @@ -0,0 +1,7 @@ +# `async fn main() -> Result<()>` + +- Configures logging +- Creates instances of `Repo`, `Robot`, `Billing`, `Api`, and `Infra` +- Spawns `billing.start()` +- Spawns `infra.start()` +- Calls `api.serve()` diff --git a/backend/spec/models.md b/backend/spec/models.md new file mode 100644 index 0000000..462cc30 --- /dev/null +++ b/backend/spec/models.md @@ -0,0 +1,88 @@ +This file describes the domain model. This description should be translated into standard structs and sqlite schemas in a way that makes sense. + +# Activity + +Activity is an audit log of all actions performed by a user or a worker process. This allows us to trace history to create invoices, synchronize actions to external services, and debug system behavior. + +- `id` - a random activity ID +- `created_at` - unix timestamp when the activity was created +- `activity_type` is one of: + - `tenant_created` + - `tenant_billing_anchor_updated` + - `relay_created` + - `relay_updated` + - `relay_activated` + - `relay_deactivated` + - `relay_sync_failed` + - `invoice_created` + - `invoice_paid` + - `invoice_attempted` + - `invoice_sent` + - `invoice_closed` +- `identifier` is a string identifying the resource being modified. This id in interpreted depending on what the `activity_type` is. + +# Tenant + +Tenants are customers of the service, identified by a nostr `pubkey`. Public metadata like name etc are pulled from the nostr network. They also have associated billing information. + +- `pubkey` is the nostr public key identifying the tenant +- `nwc_url` a nostr wallet connect URL used for **paying** invoices generated by the system +- `created_at` unix timestamp identifying tenant creation time +- `billing_anchor` unix timestamp identifying billing cycle anchor. This gets reset when the tenant has no paid relays and adds (or reactivates) one. + +# Relay + +A relay is a nostr relay owned by a `tenant` and hosted by the attached zooid instance. Relay subdomains MUST be unique. + +- `id` - a random ID identifying the relay +- `tenant` - the tenant's pubkey +- `schema` - the relay's db schema (read_only, calculated based on `subdomain` + `id`) +- `subdomain` - the relay's subdomain +- `plan` - the relay's plan +- `status` - `new|active|inactive`. Only `active` relays count toward billing. +- `sync_error` - a string indicating any errors encountered when synchronizing. +- `info_name` - the relay's name +- `info_icon` - the relay's icon image URL +- `info_description` - the relay's description +- `policy_public_join` - whether to allow non-members to join the relay without an invite code +- `policy_strip_signatures` - whether to remove signatures when serving events to non-admins +- `groups_enabled` - whether NIP 29 groups are enabled +- `management_enabled` - whether NIP 86 management API is enabled +- `blossom_enabled` - whether blossom file storage is enabled +- `livekit_enabled` - whether livekit calls are enabled +- `push_enabled` - whether relay push is enabled + +Some attributes persisted to zooid via API have special handling: + +- The relay's `secret` is generated once and persisted to the zooid configuration but isn't stored in the database. +- The relay's `host` is calculated based on `subdomain` + `RELAY_DOMAIN` +- The value of `inactive` is calculated based on `status` +- The relay's `livekit_*` configuration is inferred based on environment variables and `livekit_enabled`. +- The relay's `roles` are hard-coded for now. + +# Invoice + +Invoices are generated at the end of a tenant's monthly billing period. The billing module is responsible for creating them, collecting them, and dunning them. + +- `id` - random invoice ID +- `tenant` - tenant pubkey +- `status` - `pending|paid|closed` +- `amount` is derived as the sum of associated invoice item `sats` values (not stored as a separate source of truth) +- `created_at` - unix timestamp for when the invoice was created +- `attempted_at` - unix timestamp for when collection was last attempted +- `error` - optional human-readable error from the last failed collection attempt +- `closed_at` - unix timestamp for when the invoice was closed +- `sent_at` - unix timestamp for when the invoice was sent via DM +- `paid_at` - unix timestamp for when the invoice was paid +- `bolt11` - a BOLT 11 lightning invoice that can be used to pay the invoice +- `period_start` - unix timestamp for period start +- `period_end` - unix timestamp for period end + +# Invoice Item + +Invoice items are attached to an invoice and represent charges for a given relay. + +- `id` - random invoice item ID +- `invoice` - invoice ID +- `relay` - relay ID +- `sats` - amount in satoshis diff --git a/backend/spec/repo.md b/backend/spec/repo.md new file mode 100644 index 0000000..a9428fd --- /dev/null +++ b/backend/spec/repo.md @@ -0,0 +1,110 @@ +# `pub struct Repo` + +Repo is a wrapper around a sqlite pool which implements methods related to database access. + +Members: + +- `database_url: String` - the location of the sqlite database, from `DATABASE_URL` +- `pool: sqlx::SqlitePool` - a sqlite connection pool + +Notes: + +- All public methods should be run in a transaction so they're atomic +- All writes should be accompanied by an activity log entry of `(activity_type, identifier)` + +## `pub fn new() -> Self` + +- Reads environment and populates members +- Ensures that any directories referred to in `self.database_url` exist +- Initializes its sqlx `pool` + +## `pub fn list_tenants(&self) -> Result>` + +- Returns all tenants + +## `pub fn get_tenant(&self, pubkey: &str) -> Result` + +- Returns matching tenant + +## `pub fn create_tenant(&self, tenant: &Tenant) -> Result<()>` + +- Creates tenant, may throw sqlite uniqueness error on pubkey +- Logs activity as `(tenant_created, tenant_id)` + +## `pub fn update_tenant_billing_anchor(&self, pubkey: &str, billing_anchor: i64) -> Result<()>` + +- Updates the tenant's `billing_anchor` +- Logs activity as `(tenant_billing_anchor_updated, tenant_id)` + +## `pub fn list_relays(&self, tenant_id: Option<&str>) -> Result>` + +- Returns all matching relays + +## `pub fn get_relay(&self, id: &str) -> Result` + +- Returns matching relay + +## `pub fn create_relay(&self, relay: &Relay) -> Result<()>` + +- Creates relay, may throw sqlite uniqueness error on subdomain +- Sets relay status to `new` +- Logs activity as `(relay_created, relay_id)` + +## `pub fn update_relay(&self, relay: &Relay) -> Result<()>` + +- Updates relay, may throw sqlite uniqueness error on subdomain +- Logs activity as `(relay_updated, relay_id)` + +## `pub fn deactivate_relay(&self, relay: &Relay) -> Result<()>` + +- Sets relay status to `inactive` +- Logs activity as `(relay_deactivated, relay_id)` + +## `pub fn activate_relay(&self, relay: &Relay) -> Result<()>` + +- Sets relay status to `active` +- Logs activity as `(relay_activated, relay_id)` + +## `pub fn fail_relay_sync(&self, relay: &Relay, sync_error: String) -> Result<()>` + +- Sets relay status to `inactive`, sets `sync_error` +- Logs activity as `(relay_sync_failed, relay_id)` + +## `pub fn create_invoice(&self, invoice: &Invoice, invoice_items: [&InvoiceItem]) -> Result<()>` + +- Saves invoice and invoice_items +- Logs activity as `(invoice_created, invoice_id)` + +## `pub fn list_invoices(tenant_id: Option<&str>) -> Result>` + +- Returns all matching invoices + +## `pub fn mark_invoice_paid(&self, invoice_id: &str) -> Result<()>` + +- Sets invoice status to `paid` +- Sets `paid_at` to now +- Clears `error` if set +- Logs activity as `(invoice_paid, invoice_id)` + +## `pub fn mark_invoice_attempted(&self, invoice_id: &str, error: Option<&str>) -> Result<()>` + +- Sets `attempted_at` to now +- Updates `error` if provided +- Leaves status as `pending` +- Logs activity as `(invoice_attempted, invoice_id)` + +## `pub fn mark_invoice_sent(&self, invoice_id: &str) -> Result<()>` + +- Sets `sent_at` to now +- Leaves status as `pending` +- Logs activity as `(invoice_sent, invoice_id)` + +## `pub fn mark_invoice_closed(&self, invoice_id: &str) -> Result<()>` + +- Sets invoice status to `closed` +- Sets `closed_at` to now +- Logs activity as `(invoice_closed, invoice_id)` + +## `pub fn list_activity(&self, since: &i64, tenant: Option<&str>) -> Result>` + +- Returns all activity occuring after `since` matching `tenant` diff --git a/backend/spec/robot.md b/backend/spec/robot.md new file mode 100644 index 0000000..dee0895 --- /dev/null +++ b/backend/spec/robot.md @@ -0,0 +1,26 @@ +# `pub struct Robot` + +Robot is a nostr identity which acts on behalf of the application. + +Members: + +- `secret: String` - a nostr secret key, from `ROBOT_SECRET` +- `name: String` - the name of the bot, from `ROBOT_NAME` +- `description: String` - the description of the bot, from `ROBOT_DESCRIPTION` +- `picture: String` - the picture URL for the bot, from `ROBOT_PICTURE` +- `outbox_relays: Vec` - outbox relay URLs, from `ROBOT_OUTBOX_RELAYS` +- `indexer_relays: Vec` - indexer relay URLs, from `ROBOT_INDEXER_RELAYS` +- `messaging_relays: Vec` - messaging relay URLs, from `ROBOT_MESSAGING_RELAYS` +- `client: nostr_sdk::Client` + +## `pub fn new() -> Self` + +- Reads environment and populates members +- Publishes a `kind 0` nostr profile, a `kind 10002` relay list, and `kind 10050` relay selections using `client` + +## `pub async fn send_dm(&self, recipient: &str, message: &str) -> Result<()>` + +- Fetches recipient's outbox relays from `indexer_relays` (cached) +- Fetches recipient's messaging relays from their outbox relays (cached) +- Sends DM to recipient via their messaging relays +- If no outbox/messaging relays are found, return an error