Reading the global feed with deed and Jev
The problem
Open the global feed on any nostr client and you get everything: every note, from every account, in the order it arrived. Relays pass notes along; they don’t judge them, and they shouldn’t. So finding the notes worth reading is left to you, or to whoever you follow.
I wanted to see what that feed is actually made of. So I took the newest 1,000 notes from five public relays and looked.
371 of them weren’t text at all. Some clients publish their own app data as ordinary notes, as blobs of JSON that no person is meant to read. Of the 629 that were text, about a quarter was spam. The spam I read by hand was drug and weapon adverts, cult recruitment, leaked account logins and promotion. The rest was the usual mix of news, daily life, tech, bitcoin and art, with “GM ☕” everywhere.
Sorting that by hand doesn’t scale. Sorting it with a script needs two things: a reliable way to get the notes, and a reliable way to judge them.
deed: getting the notes
deed is a nostr command line I’ve been building in Zig. One small binary for macOS and Linux. It makes keys, signs and verifies events, encodes and decodes NIP-19, encrypts with NIP-44, queries relays and publishes to them.
Two things make it fit here. Every event it prints has had its signature checked, so a relay can’t slip in a forgery. And it keeps what it fetches in a local store, so a second run reads from disk without opening a socket, and a note that five relays all sent is handled once.
Its output is one JSON event per line, which means you can pipe it into anything.
Jev: judging them
Jev is a model from TypeSafe that answers questions about a piece of text with numbers instead of prose. You ask it a multiple-choice question (“what is this note about: bitcoin, nostr, tech, news, art, personal, media?”), a rating (“how much substance does it carry, from nothing to a lot?”), or a yes-or-no (“is it spam?”), and it returns the answer with a probability and a confidence.
That last part matters. A number you can compare against a threshold means the decision stays in code you can read and change, not inside a model’s paragraph.
Putting them together
export TYPESAFE_API_KEY=...
examples/jev/rank-notes.sh 1000 > ranked.jsonl
deed fetches and verifies the notes, the script drops the JSON blobs before anything is sent, Jev answers three questions about each note, and the script drops anything it calls spam and sorts the rest by substance. At the top: market readouts, a note on bond yields, a regional budget story, a line of Spanish prose. At the bottom: greetings.
Jev marked 157 of the 629 as spam. The whole run cost about two cents.
What it costs
deed is free and open source. Jev is a paid API, charged only for the text you send it: $0.042 per million input tokens at the time of writing, with the answers free.
The run above sent 539,784 tokens and cost about $0.023. That’s roughly four thousandths of a cent per note, or about $3.70 for 100,000 notes at the same rate. The script also drops the 371 JSON blobs before calling Jev, so nothing is paid for a note that was never text.
The script and the full numbers: https://github.com/zig-nostr/deed/tree/main/examples/jev
Write a comment