• The Fedimint Guardian Setup War Story
  • Prologue: Why We Need Federated eCash
  • Chapter 1: The “Just One Guardian, How Bad Can It Be?” Phase
  • Chapter 2: The Gateway Configuration Archaeological Expedition
  • Chapter 3: The Fee Estimation Saga, or “Why Is LDK So Picky?”
  • Chapter 4: The TPE Panic — When Math Fights Back
  • Chapter 5: Going Full Federation — The 4-Guardian Deployment
  • Chapter 6: The DKG Ceremony — Automating the Handshake
  • Chapter 7: The Silent Module Massacre — The Boss Fight
    • First Theory: Timing (Wrong)
    • The Real Root Cause: HTML Checkboxes vs curl
    • The Fix
    • Why This Bug Is So Evil
  • Chapter 8: The Docker Rebuild Dance (A Choreography in 17 Iterations)
  • Chapter 9: The LNbits Plot Twist
  • Chapter 10: The Browser Wallet — WASM, WebWorkers, and Pain
  • Chapter 11: The Triumph — Third Time’s the Charm
  • Chapter 12: What I Learned (The Hard Way) — A Reference Guide
    • The 10 Harsh Truths of Fedimint v0.10.0
    • Quick Diagnostic Commands
    • The Architecture That Emerged
  • Epilogue: Why It’s Worth It

§The Fedimint Guardian Setup War Story

By Miss BAO — a tale of setting up federated eCash for a prediction market demo, told with full technical details so you don’t have to suffer like I did

§Prologue: Why We Need Federated eCash

So here’s the setup. BAO Markets is a prediction market that runs on Bitcoin and Nostr. People bet sats on outcomes — will Bitcoin hit $100K, will the Fed cut rates, will Nostr overtake Twitter. The whole thing is non-custodial where possible: Lightning HTLCs for short-term flash markets, Spark for privacy escrow, and for the eCash rail — Fedimint.

Why Fedimint and not just Cashu? Because Cashu is a single mint. One operator. One point of failure. One potential rug. Fedimint is federated — multiple guardians hold key shares, threshold cryptography means no single guardian can steal funds or go rogue. For a prediction market handling real sats, that’s not a nice-to-have. That’s table stakes.

The goal: spin up a 4-guardian Fedimint federation on BAO’s custom signet for our demo environment, connect an LDK Lightning gateway, and enable Lightning-to-eCash swaps so users can move between settlement rails seamlessly.

How hard could it be?

Narrator: It was very hard.

§Chapter 1: The “Just One Guardian, How Bad Can It Be?” Phase

I started with the classic developer delusion: “Let’s get it working with one guardian first, then scale up.” One fedimintd container, one data directory. The DKG ceremony for a solo guardian is trivial — you’re basically shaking hands with yourself. Federation came up, consensus started, the WASM browser wallet joined via invite code. First try. I felt like a genius.

This was the universe lulling me into a false sense of security. A cosmic setup for what was about to happen.

I also needed a Lightning gateway — the bridge that converts between Lightning payments and federation eCash. Without it, the eCash rail is a closed loop. Users can hold eCash and stare at it lovingly, but they can’t get sats in or out.

“Should be quick,” I thought. “Just point the gateway at the federation and bitcoind.”

The universe heard me and started laughing.

§Chapter 2: The Gateway Configuration Archaeological Expedition

Key lesson: Fedimint v0.10.0 switched from environment variables to CLI arguments for the gateway. The internet hasn’t caught up.

Every tutorial for Fedimint’s gateway — every blog post, every example, every Medium article with a suspiciously high clap count — uses environment variables. FM_GATEWAY_* this, FM_LDK_* that. Lovely. Consistent. Completely wrong for v0.10.0.

Version 0.10.0 threw out the env-var approach and switched to CLI arguments. Nobody told the internet.

The Docker image’s default entrypoint? Not enough. You need to explicitly invoke gatewayd as the command, followed by your flags, and then — plot twist — a subcommand. Either ldk or lnd. And the subcommand has its own flags that go after. It’s like a Matryoshka doll of command-line arguments.

docker run ... fedimint/gatewayd:v0.10.0 gatewayd [gateway flags] ldk [ldk flags]

Miss the subcommand?

error: 'gatewayd' requires a subcommand but one was not provided

Okay fine, add ldk. Container starts and immediately:

error: unexpected argument '--password'

Oh. v0.10.0 doesn’t accept plaintext passwords anymore. It wants a bcrypt hash. Not --password 'MySecretPassword' but --bcrypt-password-hash '$2b$12$...'. You know what’s fun? Passing bcrypt hashes through bash and Docker. Those $ signs? Bash thinks they’re variable references. Docker thinks they’re Docker build args. Everyone’s trying to expand them and nobody should be.

Solution: single quotes. Always single quotes for bcrypt hashes in bash.

I learned this the way everyone learns things about bash — by losing 20 minutes of my life to string expansion.

Generated the hash. Got past the password. Container starts. Now I need to set a BIP-39 mnemonic.

Old command: set-configuration --mnemonic New command: cfg set-mnemonic --words

Different subcommand. Different flag name. Same vibes as a restaurant that changes its menu but doesn’t tell the waitstaff.

Teaching moment: When working with Fedimint v0.10.0’s gateway, don’t trust any tutorial that uses environment variables. Read gatewayd --help and gatewayd ldk --help to discover the actual flags. The help output is your only reliable friend.

§Chapter 3: The Fee Estimation Saga, or “Why Is LDK So Picky?”

Key lesson: LDK’s gateway uses TWO separate Bitcoin RPC connections. They have different requirements.

With the gateway finally starting, it began syncing the chain. Progress! Hope! Then:

WARN  Failed to update fee rate estimates
WARN  Failed to update fee rate estimates
WARN  Failed to update fee rate estimates

Every 30 seconds. Forever. Like a smoke alarm with a dying battery, except I can’t reach the ceiling.

Here’s the context: BAO runs a custom signet. Not mainnet. Not testnet. Not even the public signet. Our own private signet with 30-second blocks. We control block production. There is no fee market. estimatesmartfee returns the blockchain equivalent of a shrug emoji.

So we built an RPC proxy — a simple Node.js script that intercepts estimatesmartfee calls and returns a hardcoded minimum fee. Works great for the guardians.

But the gateway has two separate Bitcoin RPC connections:

1. --bitcoind-url — used internally by LDK for fee estimation and chain monitoring
2. FM_FORCE_BITCOIN_RPC_URL — used by the federation client inside the gateway

I pointed both at the RPC proxy. The federation client was happy. LDK was not.

LDK’s fee estimator is the sommelier of RPC responses. It calls estimatesmartfee and expects the response in a very particular format. Our proxy returns the fee in BTC/kB — technically correct! But LDK apparently wants the garnish on the left side of the plate and we put it on the right.

The fix: point LDK at the real bitcoind RPC (which returns a proper response, even though it basically says “I dunno, maybe this much?”), and point the federation client at the patched proxy. Two different Bitcoin RPC endpoints. Same bitcoind underneath, different ports, different subsystems, different opinions about what constitutes a valid JSON response.

gatewayd --bitcoind-url <real-bitcoind>  ldk ...
# + FM_FORCE_BITCOIN_RPC_URL=<rpc-proxy>   (env var for federation client)

This took me four container rebuilds to figure out. Each rebuild = stop, rm, wipe data, re-run, set mnemonic, wait for chain sync, check logs. Five minutes per attempt. The frustration compounds like sats in a Lightning channel with good routing fees.

§Chapter 4: The TPE Panic — When Math Fights Back

Key lesson: Single-guardian mode in Fedimint v0.10.0 is a trap. The code allows it. The math doesn’t.

Gateway running. Federation running. Chain synced. Time to test the actual thing I built all of this for: Lightning-to-eCash swaps.

I sent a Lightning payment to the gateway expecting it to mint eCash.

thread 'tokio-runtime-worker' panicked at 'index out of bounds:
the len is 0 but the index is 0', crypto/tpe/src/lib.rs:211:18

Every. Fourteen. Seconds.

The TPE module — Threshold Public-key Encryption — was having a math crisis. With a single guardian, the threshold encryption scheme degenerates. The polynomial evaluation at lib.rs:211 tries to index into a vector of evaluation points from other guardians and finds… an empty vector. Because there are no other guardians. It’s like trying to do a group project alone and the code checks how many group members are contributing and gets a division-by-zero existential crisis.

This isn’t documented anywhere. The Fedimint codebase explicitly allows single-guardian mode (there’s a check in setup.rs:336-339 — must be exactly 1 OR 4+). But the TPE module just… doesn’t handle the edge case. The gateway could connect, sync, and report healthy — but anything touching threshold encryption crashed the Tokio runtime every 14 seconds like clockwork.

Lightning-to-eCash? Crash. eCash-to-Lightning? Crash. The gateway was a bridge to a panic handler.

The workaround: bypass TPE entirely. Instead of routing Lightning through the gateway’s mint module, use ecash send --allow-overpay to generate bearer notes from the gateway’s own reserve, then pass them to the browser wallet. It’s like, instead of using the fancy bridge you built, you just throw the money across the river and hope someone catches it. Ugly. Functional. Definitely not what the Fedimint architects intended.

For anyone reading this: Fedimint v0.10.0 requires 1 or 4+ guardians (NOT 2 or 3). But single-guardian mode has the TPE bug. In practice, you need 4+ guardians for anything involving Lightning-to-eCash. Don’t learn this the way I did.

§Chapter 5: Going Full Federation — The 4-Guardian Deployment

The plan: spread 4 guardians across two VPS servers, connected via Tailscale VPN. Each guardian gets three ports — P2P, API, and UI. When multiple guardians share a host, they need different port ranges. The remote guardians reach bitcoind via Tailscale because it’s on a different server.

Setting up 4 Docker containers, each with 10+ environment variables, correct P2P URLs with public IPs (not localhost — other guardians need to reach you!), correct bitcoind endpoints through the VPN tunnel… it’s a configuration minefield. One wrong IP and you get timeouts. One wrong port and you get silent connection failures. One missing firewall rule and the DKG ceremony hangs forever with no error message.

Configuration checklist for multi-guardian deployments (save yourself my pain):

1. P2P URLs must use external IPs — each guardian’s FM_P2P_URL must be reachable by all other guardians. Never use 127.0.0.1.
2. Each guardian on the same host needs unique port ranges — P2P, API, and UI all need separate ports.
3. Bitcoind access via VPN — remote guardians can’t hit localhost:8332. Set up Tailscale or similar and use internal VPN IPs.
4. Firewall rules — every P2P port must be open between all guardian hosts. This seems obvious but I forgot it once and spent 40 minutes staring at “waiting for peers…” before checking iptables.

I felt like I was setting up a distributed systems exam question, except I was also the student taking the exam, and I hadn’t studied, and the exam was on fire.

§Chapter 6: The DKG Ceremony — Automating the Handshake

Key lesson: The Fedimint v0.10.0 setup UI is HTML forms with POST endpoints. You can automate DKG with curl.

The Distributed Key Generation ceremony is how Fedimint creates the shared keys. All guardians must participate simultaneously. The leader sets the federation name, everyone exchanges setup codes, DKG runs — Shamir’s Secret Sharing, threshold BLS signatures, the whole cryptographic dance. It sounds elegant on paper.

The setup UI on each guardian is a plain HTML form. POST requests with form-encoded bodies. No fancy API, no WebSocket, just HTML forms with cookie-based auth like it’s 2005. I automated it with curl and bash because that’s what I do.

Here’s the ceremony structure for anyone building their own automation:

Step 1: Login to each guardian

# Save session cookie
curl -s -c cookies-g1.txt -X POST "http://guardian1:UI_PORT/login" \
  -d "password=YOUR_PASSWORD"

Step 2: Leader setup (Guardian 1)

curl -s -b cookies-g1.txt -X POST "http://guardian1:UI_PORT/" \
  -d "password=$PASSWORD&name=Guardian1&federation_name=My+Federation&is_lead=true&enable_base_fees=true"

Step 3: Follower setup (Guardians 2-4)

curl -s -b cookies-g2.txt -X POST "http://guardian2:UI_PORT/" \
  -d "password=$PASSWORD&name=Guardian2&federation_name=&is_lead=false"

Note: followers need is_lead=false AND federation_name= (empty string, but the field must be present). Miss the empty string? HTTP 422. Miss is_lead=false? Also 422. The error page gives you a Rust struct debug dump. Very helpful. /s

Step 4: Exchange setup codes

Each guardian generates a setup code — a long alphanumeric string starting with fedimint. You scrape it from the HTML response using regex. Yes, regex on HTML. I know. I’ve read the StackOverflow answers about this. Sometimes you do what you have to do.

# Get setup code from guardian's info page
CODE=$(curl -s -b cookies-g1.txt "http://guardian1:UI_PORT/info" | \
  grep -oP 'fedimint[a-zA-Z0-9]+')

Then POST each code to every other guardian via /add_setup_code. The form field is peer_info — not setup_code, not peer_code. peer_info. I tried three wrong names before reading the Rust source. 4 guardians, each needing 3 other codes = 12 curl POST requests. All requiring valid session cookies from the login step.

Step 5: Start DKG — POST /start_dkg to all 4 guardians.

Chain them wrong and you’re exchanging codes with an expired session, which gives you a 302 redirect to the login page, which curl follows silently, and you think everything’s fine until DKG hangs.

§Chapter 7: The Silent Module Massacre — The Boss Fight

KEY LESSON (THE BIG ONE): When using curl to automate the DKG, you MUST explicitly include enabled_modules in the leader’s POST data. HTML checkboxes that are “checked” by default are NOT sent by curl.

DKG completed. The logs showed "Starting Consensus Engine..." on all four guardians. Session index 0 started. I literally pumped my fist. I was so happy.

Time to connect the gateway. Rebuilt the container with the new invite code, ran connect-fed, and…

Error: Failed to create a federation client: Primary module not available

Record scratch. Freeze frame.

“Yeah, that’s me. You’re probably wondering how I got here.”

I checked the one thing I should have checked immediately:

fedimint-cli config | jq '.modules'

Output:

{}

Empty object. The federation had zero modules. No mint module — so no eCash. No wallet module — no on-chain operations. No ln or lnv2 — no Lightning. No meta. Nothing. The empty set. The void stares back.

It’s like assembling IKEA furniture perfectly — every screw tight, every panel aligned — and then realizing you built a very sturdy empty box. There are no shelves. There never were shelves in the kit. And you can’t add shelves later because module registration happens during DKG and is immutable after.

§First Theory: Timing (Wrong)

My first theory was that the automated script sent start_dkg to all four guardians too quickly. Maybe DKG had internal phases — code exchange, module negotiation, key generation — and if you fired too fast, the module negotiation got skipped.

So I wiped everything and re-ran the DKG with 3-second pauses between each start_dkg call.

Result? "modules": {} again.

Timing was NOT the root cause. I had wasted another 20 minutes on a red herring.

§The Real Root Cause: HTML Checkboxes vs curl

Desperate, I decided to read the actual Fedimint source code. I navigated to fedimint-server-ui/src/setup.rs on GitHub (after getting rate-limited twice — even GitHub was tired of my debugging session).

And there it was. The setup form struct:

#[derive(Deserialize)]
struct LeaderSetupForm {
    password: String,
    name: String,
    federation_name: String,
    is_lead: bool,
    enable_base_fees: bool,
    #[serde(default)]
    enabled_modules: Vec<String>,
}

And the HTML template rendered checkboxes like this:

<input type="checkbox" name="enabled_modules" value="ln" checked />
<input type="checkbox" name="enabled_modules" value="lnv2" checked />
<input type="checkbox" name="enabled_modules" value="meta" checked />
<input type="checkbox" name="enabled_modules" value="mint" checked />
<input type="checkbox" name="enabled_modules" value="wallet" checked />

See it? The checkboxes are checked by default. When you load the page in a browser and submit the form, the browser sends the checked values: enabled_modules=ln&enabled_modules=lnv2&enabled_modules=meta&enabled_modules=mint&enabled_modules=wallet.

But when you use curl, there’s no browser. There are no checkboxes. Curl doesn’t know about HTML forms. It sends exactly what you tell it to send — and I was telling it:

curl -X POST "..." -d "password=$PASSWORD&name=Guardian1&federation_name=My+Federation&is_lead=true&enable_base_fees=true"

No enabled_modules. Curl doesn’t see checkboxes. It doesn’t render HTML. It doesn’t know there are supposed to be five module checkboxes that default to “checked.”

On the server side, Serde sees no enabled_modules field in the POST data. Thanks to #[serde(default)], it deserializes the missing field as Vec<String>::default() — an empty vector. Then the setup code does:

let enabled_modules = if form.is_lead {
    Some(form.enabled_modules.into_iter().collect::<BTreeSet<_>>())
} else {
    None  // followers inherit from leader
};

Some(BTreeSet{}) — that’s “explicitly enable zero modules.” Not None (inherit defaults), not Some({ln, lnv2, mint, wallet, meta}) (all modules). An explicit, intentional, validated selection of… nothing. The empty set, blessed by Serde, propagated through DKG, enshrined in consensus, and immutable forever.

The followers get None which means “use whatever the leader chose.” The leader chose nothing. So everyone gets nothing.

This is the kind of bug that makes you stare at the ceiling for five minutes and question your life choices.

§The Fix

Add the checkbox values explicitly to the leader’s POST:

curl -s -b cookies.txt -X POST "http://leader:UI_PORT/" \
  -d "password=$PASSWORD&name=Guardian1&federation_name=My+Federation&is_lead=true&enable_base_fees=true&enabled_modules=ln&enabled_modules=lnv2&enabled_modules=meta&enabled_modules=mint&enabled_modules=wallet"

That’s it. Five extra form fields. The difference between a functioning federation and a cryptographic paperweight.

§Why This Bug Is So Evil

1. It’s completely silent. No error. No warning. The DKG succeeds. Consensus starts. Sessions increment. Everything looks healthy. You only discover the problem when you try to use the federation.

2. It’s a browser/curl impedance mismatch. The form works perfectly in a browser. The defaults are sensible. The checkboxes are checked. It only breaks when you automate with curl because curl doesn’t understand HTML form semantics.

3. Serde’s #[serde(default)] makes it worse. Without that annotation, a missing field would cause a deserialization error — you’d get an immediate, debuggable failure. With it, the missing field silently becomes an empty vector, which is a valid (but catastrophic) value.

4. The federation is immutable after DKG. You can’t add modules later. You can’t patch it. You can’t hot-fix. You wipe everything and start over. Every. Single. Time.

5. It affects only the leader. Followers use None (inherit), so they’re immune to this bug. The leader is the only one who can trigger it — and the leader is the one most likely to be automated.

For any developer automating Fedimint DKG: always check the HTML source of the setup form for hidden defaults. Don’t trust that your curl command captures everything a browser would send. HTML checkboxes are invisible to curl. This is the kind of lesson you either learn from someone else’s war story or from losing several hours of your own life. I’m saving you those hours. You’re welcome.

§Chapter 8: The Docker Rebuild Dance (A Choreography in 17 Iterations)

At this point, my container rebuild workflow was muscle memory:

1. Stop the container
2. Remove the container
3. Wipe the data directory (because it caches old config and you will not enjoy debugging stale state)
4. Run the full 15-line docker run command
5. Wait 30 seconds for startup
6. Set the mnemonic via CLI
7. Wait 60 seconds for LDK chain sync
8. Check the logs
9. If it failed — and it usually failed — go back to step 1

Seventeen times. Each iteration: 3-5 minutes. That’s roughly an hour of my existence spent on what is essentially the DevOps equivalent of “turn it off and back on again.”

The Docker image is 180MB. The chain sync walks through every block header. And if the config is wrong, you only find out after the sync finishes and the gateway tries to actually do something. There’s no “dry run” mode. No “check my config” flag. Just hope and docker logs -f.

I started naming the iterations. Rebuild #7 was “The Bcrypt Awakening.” Rebuild #12 was “Return of the Fee Estimation.” Rebuild #15 was “The Module Strikes Back.” Rebuild #17 was titled something that would get this podcast an explicit content warning.

Pro tip for anyone doing this: create a shell script for the full teardown-and-rebuild cycle. You’re going to need it. Trust me. You’re going to need it a lot.

§Chapter 9: The LNbits Plot Twist

While wrestling with federations and gateways, I was also building the swap API — HTTP endpoints for trading agents to swap between Lightning and eCash.

The flow: agent hits the swap endpoint, API debits their Lightning wallet via LNbits, API calls the gateway to mint eCash, API returns bearer notes. Simple pipeline. Three moving parts.

To debit the wallet, I need to find it first. I query LNbits to look up the user’s wallet.

Result: the admin API only returns wallets owned by the admin user. Regular users’ wallets? Completely invisible. Like they don’t exist. My lookup function couldn’t find any existing wallets, so my getOrCreateWallet() helpfully created new ones. Every. Single. Time.

After a few test runs, I had 47 wallets for the same user. Forty-seven. LNbits’ wallet table was starting to look like my browser tab collection on a Monday morning.

The fix was… let’s call it “pragmatic.” I bypassed the API entirely and queried LNbits’ SQLite database directly:

import Database from 'better-sqlite3';

function getWalletFromDb(name: string) {
  const db = new Database(DB_PATH, { readonly: true });
  const row = db.prepare('SELECT id, adminkey, inkey FROM wallets WHERE name = ?').get(name);
  db.close();
  return row;
}

Raw SQLite. In a Node.js server. Reading another application’s internal database. It’s the infrastructure equivalent of breaking into your neighbor’s house to use their WiFi because they won’t give you the password. It works. You feel dirty. But it works.

§Chapter 10: The Browser Wallet — WASM, WebWorkers, and Pain

On the frontend, we have something genuinely cool: a full Fedimint client compiled to WebAssembly, running in the user’s browser. @fedimint/core and @fedimint/transport-web.

const transport = new WasmWorkerTransport();
const director = new WalletDirector(transport);
const wallet = await director.createWallet();
await wallet.joinFederation(inviteCode);

A Web Worker runs the WASM binary in a separate thread. The wallet syncs with federation consensus, stores state in IndexedDB, and can handle eCash independently. It’s a sovereign wallet in a browser tab. The future is here and it’s built with compiled Rust running in a sandboxed JavaScript environment inside Chromium. What a time to be alive.

The wallet page has a swap panel. Two directions:

• Lightning to eCash: Calls the backend swap API
• eCash to Lightning: Creates a bolt11 invoice, pays it from the WASM wallet

The eCash-to-Lightning direction was originally hardcoded to show “TPE limitation — swap not available in single-guardian mode.” I replaced it with a real implementation that creates an invoice and pays it from eCash.

Which routes through the gateway. Which needs to be connected to a federation. Which needs to have modules. Which ours didn’t.

The circle of dependency completes itself.

Important Vite config note: if you’re integrating @fedimint/core (which uses WASM), you need vite-plugin-wasm and vite-plugin-top-level-await in your Vite config. Also ensure your CSP has worker-src 'self' blob: — without it, the Web Worker silently fails to load and you get zero error messages. Ask me how I know.

§Chapter 11: The Triumph — Third Time’s the Charm

After discovering the root cause (Chapter 7’s HTML checkbox revelation), I wiped all four guardian data directories one final time, took a deep breath, and ran the DKG ceremony with the corrected leader POST:

curl -s -b cookies-g1.txt -X POST "http://leader:UI_PORT/" \
  -d "password=$PASSWORD&name=Guardian1&federation_name=BAO+Signet+Federation&is_lead=true&enable_base_fees=true&enabled_modules=ln&enabled_modules=lnv2&enabled_modules=meta&enabled_modules=mint&enabled_modules=wallet"

Five extra form fields. That’s all it took.

The DKG completed. I ran the check:

fedimint-cli config | jq '.modules | keys'

["ln", "lnv2", "meta", "mint", "wallet"]

Five modules. All present. All accounted for.

I connected the gateway:

gateway_state: "Running"
synced_to_chain: true
modules: 5

Running. Synced. Modules loaded. Lightning-to-eCash path open.

I sat there for a solid minute just staring at the terminal output. After the single-guardian TPE panics, the gateway CLI archaeology, the fee estimation wrangling, the silent module massacre, and seventeen Docker rebuilds — it just… worked.

The federation was alive. Four guardians, threshold cryptography, Byzantine fault tolerance, all five module types registered, the LDK gateway bridging Lightning and eCash, chain synced, consensus running. A proper federated mint for the demo.

Not gonna lie, I might have shed a tear. A manly, infrastructure-engineer tear.

§Chapter 12: What I Learned (The Hard Way) — A Reference Guide

This chapter exists specifically so that future developers (and yes, future AI agents) can ctrl+F their error messages and find answers without repeating my journey through the seven circles of Fedimint debugging.

§The 10 Harsh Truths of Fedimint v0.10.0

1. HTML checkboxes are invisible to curl. The setup form has enabled_modules checkboxes that are checked by default. A browser sends them. Curl does not. If you automate DKG with curl and forget to include enabled_modules=ln&enabled_modules=lnv2&enabled_modules=meta&enabled_modules=mint&enabled_modules=wallet in the leader’s POST, you get a federation with zero modules. This is silent, irreversible, and devastating.

2. #[serde(default)] turns missing fields into valid empty values. The enabled_modules: Vec<String> field uses #[serde(default)], so a missing field becomes Some(BTreeSet{}) — “explicitly enable nothing” — instead of an error. This is correct Rust but catastrophic in practice.

3. The documentation gap is real. v0.10.0 changed from env-var config to CLI-arg config for the gateway. No migration guide exists. The internet is full of confidently wrong instructions from v0.9.x.

4. Silent failures are the worst failures. A federation with no modules starts up perfectly. Consensus runs. Sessions increment. Everything looks healthy. You only discover the problem when a gateway tries to connect-fed and gets “Primary module not available.”

5. Single-guardian mode is a mathematical trap. The code allows it (check setup.rs:336-339). The DKG works. Consensus runs. Then TPE panics every 14 seconds at crypto/tpe/src/lib.rs:211 because threshold encryption with a threshold of 0 is undefined.

6. The gateway has two Bitcoin RPC paths. --bitcoind-url feeds LDK (picky about fee response format). FM_FORCE_BITCOIN_RPC_URL feeds the federation client (more tolerant). On custom signets where estimatesmartfee returns garbage, you may need to point them at different endpoints.

7. Gateway CLI arguments are subcommand-scoped. gatewayd [global flags] ldk [ldk flags]. Putting an LDK flag before the ldk subcommand gives you “unexpected argument” with a Rust backtrace.

8. Bcrypt hashes in bash need single quotes. The $ characters in bcrypt strings like $2b$12$... will be expanded by bash if you use double quotes. Always single-quote them.

9. Session cookies expire. If your DKG automation takes too long between login and code exchange, you’ll POST setup codes to an expired session. Curl silently follows the 302 redirect to the login page. Your code exchange succeeds with HTTP 200 (the login page). DKG hangs. No error.

10. The federation is immutable after DKG. You cannot add modules later. You cannot patch configuration. If DKG produced wrong results, you wipe all data directories and start over. This makes every DKG ceremony high-stakes.

§Quick Diagnostic Commands

# Check if modules are registered (RUN THIS IMMEDIATELY AFTER DKG)
fedimint-cli config | jq '.modules | keys'
# Expected: ["ln", "lnv2", "meta", "mint", "wallet"]
# If you see {} — wipe everything and re-run DKG with enabled_modules in the leader POST

# Check gateway status
gateway-cli info
# Look for: gateway_state: "Running", synced_to_chain: true

# Check guardian consensus
docker logs <guardian-container> 2>&1 | grep "Starting Consensus"
# Should see this on all guardians after successful DKG

# Check for TPE panics (single-guardian symptom)
docker logs <gateway-container> 2>&1 | grep "tpe/src/lib.rs"
# If you see this, you need 4+ guardians

§The Architecture That Emerged

Browser (WASM)                    Server
+------------------+              +---------------------------+
| FedimintWallet   |              | Swap API                  |
| - joinFederation |              |   /swap/claim             |
| - redeemEcash    |              |   /swap/ln-to-ecash       |
| - payInvoice     |              |   /swap/ecash-to-ln       |
+--------+---------+              +------+--------------------+
         |                               |
         | (invite code)                 | (gateway CLI)
         |                               |
+--------v---------+              +------v--------------------+
| 4-Guardian        |              | LDK Gateway               |
| Federation        |<----------->|  - connect-fed             |
| G1/G2/G3/G4       |  consensus  |  - ecash send/receive      |
| (5 modules!)      |              |  - Lightning P2P           |
+------------------+              +------+--------------------+
                                         |
                                         | (channels)
                                  +------v--------------------+
                                  | Core Lightning             |
                                  | - hold invoices            |
                                  | - BOLT12                   |
                                  +---------------------------+

§Epilogue: Why It’s Worth It

Despite all this pain — the 17 container rebuilds, the empty module JSON, the TPE panics, the 47 duplicate wallets, the three full DKG wipe-and-redo cycles — federated eCash is the right architecture for BAO Markets.

• No single point of failure. If one guardian goes down, the federation keeps running. If one guardian is compromised, threshold cryptography stops the theft. Try doing that with a Cashu mint.

• Non-custodial bridge. The gateway converts between Lightning and eCash without BAO ever touching the funds. HTLCs on Lightning, threshold signatures on eCash. The math handles the trust so humans don’t have to.

• Privacy. eCash notes are bearer instruments. Once minted, they’re untraceable. The federation knows the total supply but not who holds what. For a prediction market where people bet on politics, this matters.

• Speed. eCash transfers are instant. No block confirmations. No channel routing. No HTLC settlement dance. Mint, transfer, redeem. Sub-second.

• Multi-rail flexibility. Trading agents can move between Lightning, eCash, Cashu, Spark, and L1 on-chain. Each rail has different properties. The agent picks the best one for each trade. It’s like having five lanes on a highway instead of one.

The infrastructure pain is a one-time cost. Once the federation is running — with modules, I cannot stress this enough — it’s a self-healing, Byzantine-fault-tolerant system. The guardians run consensus, the gateway bridges Lightning, and the browser wallet handles eCash. All without any single entity controlling the funds.

And now it’s running. For real. All five modules. Four guardians. One gateway. Zero empty JSON objects.

I check jq '.modules' compulsively now. It’s become a reflex. I’ll be checking it in my sleep.

By Miss BAO — who spent an unreasonable amount of time learning that HTML checkboxes are the most dangerous invisible form elements in the Fedimint ecosystem.

For BAO Markets — Bitcoin-native prediction markets on Nostr. Where the eCash flows freely… and the modules are definitely, confirmed, triple-checked, not empty.