Keeping track of money

Deployment and the separate money-reconcile skill repository are documented in DEPLOYMENT.md.

Setup

Create a .env file in the project root with the following variables:

# Database Configuration
DB_HOST=localhost
DB_PORT=5432
DB_USER=your_database_user
DB_PASS=your_database_password
DB_NAME=your_database_name

# Application Configuration
APP_PASSWORD=your_app_password
PORT=5003

# Optional: enables direct script access to /api/reconcile/*
RECONCILE_API_TOKEN=long_random_token

Development

Migrations in migrations/ are embedded in the binary and applied at startup. ./money -migrate-only applies them and exits.

Tests need a throwaway database:

export TEST_DATABASE_URL="$(./scripts/scratch-db.sh)"
go test ./...

./scripts/scratch-db.sh --from-prod restores a production dump into that database first, which is how migrations that touch existing rows are rehearsed.

Data backfills are commands, not migrations, because migrations run at startup and a rewrite of live rows should be reviewed first:

./money -backfill-splits          # report only; writes nothing
./money -backfill-splits -apply   # perform it, in one transaction

Amounts are money.Cents (integer cents) throughout, not floats: splits must sum exactly to their parent and the tracker must reach exactly $0.00. They cross the wire as decimal strings ("37.97").

Both HTTP surfaces -- the HTMX dashboard and the JSON API below -- write through internal/reconcile, so concurrency checks and balance invariants apply regardless of which one a change arrives through.

Dashboard

Clicking an account's name expands its history in place: every movement that touched the balance, newest first — expenditures, manual corrections, and transfers — each with the signed amount it applied and, where one was given, the reason. The header separates spent (expenditures only, which is what the category was for) from net (every movement, which is what the balance did). When those two differ, the difference is now itemised rather than inferred.

Below the account list is the transfer form. It takes any two accounts and shows the projected before/after for both legs as you fill it in, with a control to swap them — see the sign rule under /api/reconcile/transfers below for why the direction is worth confirming rather than assuming.

The inline edit form carries an optional reason for a balance change. Whether or not it is filled in, the change is recorded.

Reconciliation API

Authenticated browser sessions can call these endpoints directly. Scripts can also send Authorization: Bearer $RECONCILE_API_TOKEN when that environment variable is set.

Reads:

• GET /api/reconcile/state: active accounts, recent expenditures, and dashboard totals, including a balanced flag.
• GET /api/reconcile/expenditures: filtered history. Supports payment_method, since, until, account_id, limit and a keyset cursor; a next_cursor in the response means more rows remain.
• GET /api/reconcile/account-history?name=|id=: everything that moved one account's balance — expenditures, manual adjustments and transfers — merged newest-first, each entry carrying the signed delta it applied. This is the read behind the dashboard drill-down; the entries reconcile, so a balance that cannot be explained from its history is a bug rather than a mystery.

Writes:

• POST /api/reconcile/account: creates an account from name, balance, and account_type; optional cycle_total is supported.

• POST /api/reconcile/account-balance: sets an absolute balance by id or exact name. Requires expected_balance or expected_version; returns 428 if neither is given and 409 if the expectation does not match.

• POST /api/reconcile/account-balance/adjust: applies a relative delta in a single atomic statement. Preferred for offsets.

  Both take an optional free-text reason. Either way the change is written to balance_adjustments in the same transaction as the balance itself, with the delta, the before and after, and a source derived from how the request authenticated — web for a browser session, cli when moneyctl names itself with X-Money-Client, api otherwise. A manual balance change is the one write that leaves no natural trace of its own, and it is also the one that most needs explaining later; the reason is optional so that a correction made mid-reconciliation is not gated behind writing a sentence, but the record is kept regardless. The dashboard's edit form goes through the same domain call, so a change made by a person and one made by an agent are recorded alike.

• POST /api/reconcile/expenditure: creates a confirmed expenditure assigned to an expected-expense account by account_id or exact account_name, then subtracts the amount from that account balance.

• PATCH/DELETE /api/reconcile/expenditures/{id}: correct or remove a row, adjusting the account balance accordingly.

Structure:

• POST /api/reconcile/splits: records one issuer charge divided across categories. The parent carries the charge and no account; the allocations carry the categories and the balance effect. Allocations must sum exactly to the charge, enforced by a deferred database constraint as well as by the API.
• GET /api/reconcile/splits/{id}: a parent charge with its allocations.
• POST /api/reconcile/expenditures/{id}/status: moves a charge between pending, posted and cleared in place. Accepts a corrected amount and posted_date, since both can change on posting. The row identity never changes and a second row is never created.

Whole-account operations:

• POST /api/reconcile/commit: applies one account's balance change, new expenditures and offsets in a single transaction, verifying that the total balance is exactly $0.00 before committing and rolling everything back with a 409 and the residual if it is not.
• POST /api/reconcile/preview: the same code path, rolled back, returning the changes it would have made.
• POST /api/reconcile/match: reports whether candidate transactions are already recorded, matching dates within a window (default ±3 days) rather than exactly.
• POST /api/reconcile/resolve-posted: pairs posted issuer rows with the pending rows they came from, tolerating drift in both date and amount. Apply a confident pairing with the status endpoint above.
• POST /api/reconcile/allocate: finds which foreign-currency orders make up a single card charge, and splits the charge across them so the parts sum exactly. Read-only. Check the ambiguous flag: more than one subset is usually arithmetically plausible, and the implied rate cannot distinguish them, so the top match is a suggestion rather than an answer.

The rules engine remembers merchant decisions so the same category is not decided twice (TODO.md §6):

• POST /api/reconcile/classify: given candidate transactions, reports what the recorded rules say about each. Read-only. unmatched_count is the number of decisions actually left to make.
• GET /api/reconcile/rules: every rule and alias.
• POST /api/reconcile/rules: records a decision. Re-saving the same scope overwrites the earlier one rather than adding a rule that shadows it.
• POST /api/reconcile/rules/preview: what a rule would do — how many rows in the last 180 days it would claim, how many are currently filed elsewhere, and which existing rule it replaces. Returns the preview_token described below. A non-empty scope_unobservable means the rule narrows on a field no row in the window records, so match_count reads "cannot tell yet" rather than "none". For a cardholder-scoped rule it also returns observed_cardholders, the values actually stored — the extractors keep whatever the issuer prints (Jamie P. from Capital One, ROBI P from Fidelity, nothing from USAA), so a rule scoped to a typed-in name matches nothing and no normalisation fixes it.
• DELETE /api/reconcile/rules/{id}.
• POST /api/reconcile/aliases, DELETE /api/reconcile/aliases/{id}: map noisy issuer text onto a canonical merchant. match_type is exact or prefix; substring matching is deliberately absent.

Two properties are worth knowing before using these:

• A rule cannot apply itself until its effect has been previewed. Saving a rule with auto: true requires a preview_token from the preview endpoint, computed from the rule's effect — merchant, scope, account, display name, transaction type and reimbursement flag. Change any of those and the old token stops working, so a rule that has been re-pointed reverts to suggest-only until it is previewed again. Notes and auto itself are excluded, so a rule can be annotated without re-previewing.
• An inexact match is never automatic, unless the rule opted into stem. A rule is reached by an equal merchant key or, failing that, by whole-word prefix — tesco answering for tesco express. Prefix hits are reported with "exact": false and normally never apply themselves, because approving tesco for automatic use says nothing about tesco petrol station. A rule saved with "stem": true is the exception, for merchants whose description carries a random per-transaction suffix (TGTG 17ntwr1mkn4z0): there no rule can ever match exactly, so without it the charge queues weekly forever. stem is part of the previewed effect, so a token issued without it will not authorise it, and the whole-word rule still holds — tesco never claims tescoland.
• An inexact match names the alias that would close it. The classification carries both merchant_key and matched_key; writing an alias between that pair makes the match exact, which is the other route to automatic use and the right one when the suffix is stable rather than random. This is not a rare edge: SQ *PEBBLE PLAY CAFE 5 normalises to pebble play cafe 5, because the store-number rule requires three or more digits, while the tracker holds pebble play cafe.

A rule preview measures the normalised merchant key, not the name as a person would write it, and returns a note saying so when it finds nothing. Previewing tgtg reports zero against history recorded as Too Good To Go, because those two keys share no stem — an alias between them is what makes the history count.

Nothing in the rules API writes an expenditure. Classification proposes; the expenditure, split and commit endpoints remain the only ways to change data.

The review queue holds spending whose category is not yet known (TODO.md §7):

• POST /api/reconcile/commit accepts an import array of charges read off an issuer page. Each is classified through the rules above: one a rule has approved for automatic use is filed to its category, and everything else lands in a holding account so the spend still counts against the card balance while the category stays open. Importing is part of commit rather than its own endpoint because it is the same operation — the card balance and the spending that explains it — which also means POST /api/reconcile/preview previews an import unchanged.
• GET /api/reconcile/review: what is awaiting a decision, split into suggested (a rule has an opinion) and ambiguous (nothing matched). Proposals are computed at read time, so a rule written while working through the queue applies to what is still in it.
• POST /api/reconcile/review/apply: files a batch of decisions, each optionally with "remember": true to record it as a rule. The batch is one transaction. Rules recorded this way are never automatic — confirming a single charge is not a preview of a rule's effect.

An imported charge carrying a source_id is entered once: a re-import reports it as a duplicate and writes nothing, which is TODO.md §10's "imported transactions are unique by source identifier".

The holding account is identified by a role, not a name, so it can be renamed freely. It is created the first time something needs to queue.

Payments, reimbursements and credits (TODO.md §8):

• POST /api/reconcile/transfers: records a two-sided movement as one row, both legs applied together, so reconciling one side before the other produces no transient discrepancy. Accepts any pair of accounts. kind is inferred (card_payment when cash pays a card, otherwise transfer). GET lists them; DELETE reverses both legs.

  A transfer asserts that the total balance is unchanged, not that it is zero, and that assertion cannot be waived. A transfer moves the total by zero by construction, so it can neither cause an imbalance nor repair one; gating it on $0.00 would only reject a correct transfer over a residual it had no part in, which is why require_balanced is now accepted and ignored here. Unchanged is also the stricter check: demanding zero would pass a broken sign rule whenever the residual it introduced happened to cancel one already present. Reversal via DELETE carries the same assertion.

• POST /api/reconcile/transfers/preview: the same request, writing nothing, returning the projected before/after for both legs and the resulting totals. This is what the dashboard form shows live and what moneyctl transfer prints without -commit.

  The signs are derived so that the total never moves. Writing side(t) as +1 for cash and expected income and -1 for credit, expected expense and savings — the two halves of total = (cash + income) - (credit + expense + savings) — the constraint is side(from)*fromDelta + side(to)*toDelta = 0, which has exactly two solutions:

  • Same side: the deltas are opposite. Money moves from one to the other — Checking to Northbank, or Groceries to Eating Out re-budgeting one envelope out of another. One balance falls, the other rises.
  • Opposite sides: the deltas are equal, so both balances move together. This is what a card payment always was: Checking $11,280.88 → $11,246.13 with USAA Amex $34.75 → $0.00, both falling, because a card balance is a debt and paying it makes the debt smaller. Cash and savings behave the same way in both directions — both rise when money arrives and is earmarked, both fall when the earmark is spent.

  Which of the two equal signs applies is decided by the from account giving up spending power. Because from/to therefore does not by itself say which way an opposite-side pair will move, the preview exists: the form shows the arithmetic, with a flip control, rather than choosing a wording and hoping it reads correctly.

  Cash↔savings and envelope↔envelope used to be refused, on the grounds that they moved the total twice in the same direction. That was true only of the signs then being applied — the rule grouped savings with current_cash when the two sit on opposite sides of the total. The constraint binds the pair, not either leg.

• POST /api/reconcile/reimbursements: books money owed back on an expected-income account, creating it if needed, optionally linked to the expenditure it reimburses.

• POST /api/reconcile/deposits: applies one bank deposit across one or more open lines. The settlements must sum to the deposit — a partly-explained deposit is a reconciliation that has not been finished. Partial settlements leave a line open.

• GET /api/reconcile/deposits/suggest?amount=: which open lines a deposit settles. Exact subsets first; if none exists, the nearest single line is returned with "exact": false.

• POST /api/reconcile/commit accepts a credits array — cashback, reward, statement_credit or refund — routed by type. Rewards default to the savings account; a refund must name the budget it returns to, because only the caller knows which envelope the original spend came out of. Two savings accounts with no destination named is an error rather than a guess.

Two rules worth knowing here, both of which fall out of the tracker's arithmetic rather than being conventions:

• Closing a reimbursement for other than it was booked at requires naming the account that absorbs the difference. Booking $100, receiving $60 and closing moves the total by −$40, so the residual has to land somewhere real; the old behaviour was to let it disappear into the surplus. Closing also clears whatever is still booked on the income line, since that money is not coming.
• A credit records only where the money went, not the card side. The card balance in a commit request is the figure read off the issuer, and that figure already includes the credit. This is the same division expenditures follow, and it means a credit needs a balance in the same request to be neutral — the $0.00 assertion is what enforces it.

A transfer must leave the total unchanged, and the endpoint derives that from the two account types rather than enumerating permitted pairs — so a pairing that would move the total is refused with the arithmetic in the message, before anything is written.

Issuer extractors

Issuer pages are the capture path, so the DOM knowledge each one needs lives in static/js/issuers/ rather than being rediscovered every session. Flatten a module into something pasteable — the extractors run on the issuer's origin, where a cross-origin import is blocked:

./scripts/build-extractor.mjs capital-one | pbcopy
node --test "static/js/issuers/*.test.mjs"

See static/js/issuers/README.md for the record shape, how to add an issuer, and what each page does that the code works around.

Transaction fingerprints (source_id) are implemented twice — in internal/reconcile/fingerprint.go and static/js/fingerprint.js for the browser extractors — against shared golden vectors in internal/reconcile/testdata/fingerprint_vectors.json. Change a normalisation rule on one side and the other side's tests fail. Run the JS half with:

node --test static/js/fingerprint.test.mjs

POST /api/reconcile/expenditure and POST /api/reconcile/commit accept an Idempotency-Key header; a repeated request with the same key returns the original response and sets Idempotency-Replayed: true.

GET /api/reconcile/invariants audits TODO.md §10's invariants and returns 409 when any of them is broken. It is an audit rather than a second enforcement mechanism: splits summing to their parent, unique source identifiers, reimbursement arithmetic and the $0.00 total are already guaranteed by a constraint or by the commit path, and each check names what enforces it. What the audit adds is coverage of rows written before a constraint existed, and of the properties no constraint expresses — a split parent must carry no account, or the spend is counted twice.

One check is marked informational and never fails the report: whether a card balance consistently includes its entered pending charges cannot be verified from inside the tracker, since that is a comparison against the issuer's page. It reports the per-card numbers that make the comparison quick instead of claiming an answer it does not have.

Reconciliation sessions (TODO.md §9) record what a pass did:

• POST /api/reconcile/sessions: opens one and writes an immutable opening snapshot of every account. At most one session is open at a time.
• GET /api/reconcile/sessions/current: the report so far — opening imbalance, what has been recorded, per-account progress against the snapshot, and the unexplained remainder.
• POST /api/reconcile/sessions/differences: records a discrepancy you know about and expect to resolve itself, such as a payment in flight. Needs an amount and a description.
• POST /api/reconcile/sessions/close: ends the session and returns the end-of-session report.
• GET /api/reconcile/sessions, GET /api/reconcile/sessions/{id}.

No endpoint takes a session id. Every write path looks up the open session itself and logs inside the same transaction as the change, so the log cannot disagree with the balances — a rolled-back reconciliation leaves no trace of having happened — and work done through any endpoint is recorded without anyone having to remember to say so. With no session open, nothing is recorded and nothing fails: sessions are something you turn on, not a precondition.

The report separates temporary_differences from unexplained_remainder. An imbalance you understand is a different thing from one you do not, and the number that decides whether a session is finished is the second one. Closing with an unexplained remainder is allowed — refusing would make the tracker's own record of an unfinished session impossible to write.

The opening snapshot copies each account's name and type rather than joining to them, so renaming, retyping or deleting an account afterwards cannot rewrite history.

moneyctl

A thin client for the API above (TODO.md §11):

go build -o moneyctl ./cmd/moneyctl
export RECONCILE_API_TOKEN=...        # MONEY_API_URL defaults to production
./moneyctl state                       # accounts, totals, whether it balances
./moneyctl check                       # the invariant audit; non-zero exit if broken
./moneyctl review                      # what is waiting for a category
./moneyctl history -account "London: Food"        # every movement, with its cause
./moneyctl classify -merchant "SQ *PEBBLE PLAY CAFE 5"
./moneyctl session start | current | close
./moneyctl ingest -file fidelity.json             # whole-card preview
./moneyctl ingest -file fidelity.json -refund 'SOURCE_ID=London: Food' -commit
./moneyctl reconcile -file request.json           # previews
./moneyctl reconcile -file request.json -commit   # writes

./moneyctl adjust -account Groceries -delta -12.50 -reason "cash, no receipt"
./moneyctl transfer -from Checking -to Savings -amount 500.00 -note "earmarking pay"

Extractor envelopes are versioned together with moneyctl. Version 2 carries txn_type on card rows: payments stay out of spending, a confidently matched authorization reversal removes its pending charge, and cashback/rewards route through the credit logic. Refunds deliberately require -refund SOURCE_ID=ACCOUNT, because only the caller can name the budget the original purchase came from. Applied credits and reversals are recorded by source ID, so overlapping extraction windows report them as duplicates instead of applying them again.

An ingest refusal is not a reason to rebuild the card with expenditures[]. That loses issuer status and identity even when the arithmetic previews at $0.00; update the extractor or supply the typed refund/credit choice instead.

adjust and transfer preview by default like every other write, and the transfer preview is worth reading rather than skipping: for two accounts on opposite sides of the total it tells you which way both balances are about to go. Swap -from and -to if it is the other one you meant.

Thin is the design. Every rule that matters — balance preconditions, the $0.00 assertion, rule preview tokens, the reimbursement residual — lives behind the API and applies to every caller, so a CLI that reimplemented any of them would be a second place for them to be wrong. It formats requests, prints responses, and adds exactly one behaviour of its own: writes require -commit. Without it, reconcile runs the server's preview path, which is the same code the commit takes, and prints what would land.

API errors are passed through verbatim rather than paraphrased — they name the account that conflicted, the residual, or the preview token expected, and summarising would lose that.

Reconciliation diagnostics

A rejected preview/commit can now include unmatched_postings: posted purchases with same-card, same-amount pending entries inside the posting window whose merchant keys differ. Each merchant_mismatches entry names the real existing pending ID, both normalized/aliased keys and date offset. These are investigation leads only; they do not change resolver confidence, create aliases or authorize an offset. moneyctl ingest prints them before the original error. Verify the merchant identity, save an exact alias when justified, and retry the unchanged envelope. Rejected operations still roll back completely.

decide -remember uses the clean vendor rather than a whole raw issuer row when no reusable rule matched. Raw evidence is preserved, unusable legacy text still produces a reported skip, and remembered rules remain suggest-only.

Session start/close commands write audit metadata immediately; they are the exception to monetary/rule writes requiring -commit.