fanfic-archivist

Multi-platform companion bot for FicHub, a self-hosted fanfiction archive with 107-site scraper parity, pluggable recommendations, full-text body search, an LLM "Ask the Archive", a Fic Requests board, and a Roadmap consensus arena.

The bot is a thin client over the FicHub REST API: it never touches the archive's database directly, and it shares FicHub's Redis for a transient pagination cache so !next/!prev don't hammer the DB. The same platform-neutral core drives every chat front-end — Discord, Matrix, Telegram, Slack, IRC, Mastodon/Bluesky/Piefed, and a terminal CLI.

Architecture

┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌────────────┐ ┌──────────┐
│   Discord   │ │   Matrix    │ │  Telegram   │ │    Slack   │ │    IRC   │
│  (Discord)  │ │ (matrix)    │ │ (teloxide)  │ │ (socket    │ │ (irc-rs) │
│   poise     │ │  SDK        │ │             │ │   mode)    │ │          │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └─────┬──────┘ └────┬─────┘
       │               │               │              │             │
       │  ┌────────────┴───────────────┴──────────────┴─────────────┴───┐
       │  │            archivist-core  (platform-neutral)              │
       │  │  commands · intent/LLM · pagination cache · token store    │
       │  │  do_* → PlatformMessage (render matrix per platform)        │
       │  └───────────────┬───────────────────────────┬────────────────┘
       │                  │                           │
       │   ┌──────────────▼──────────┐    ┌───────────▼──────────┐
       │   │   FicHub REST API       │    │  Redis (shared with  │
       └──▶│   (HTTP, base_url)      │    │  FicHub: cache,       │
           └─────────────────────────┘    │  tokens, link codes)  │
                                          └──────────────────────┘

Two more front-ends exist: fanfic-archivist-fediverse (Mastodon, Bluesky, Piefed listeners in one binary) and fic-archivist (terminal + a WebSocket module for a future FicHub web-chat route).

crates/
├── archivist-core/            # all business logic (the only "real" crate)
│   ├── config.rs               # FANFIC_ARCHIVIST_* env config
│   ├── api.rs                  # typed reqwest client for the REST API
│   ├── dispatch.rs             # do_* commands → PlatformMessage
│   ├── intent.rs               # free-form mention intent parsing
│   ├── cache.rs                # Redis pagination cache
│   ├── store.rs                # token store + link codes
│   └── lemmy.rs, debug.rs, ... # community monitor, LLM failure diagnosis
├── fanfic-archivist/           # Discord adapter (poise; the original bot)
├── fanfic-archivist-matrix/    # Matrix adapter
├── fanfic-archivist-telegram/  # Telegram adapter (teloxide)
├── fanfic-archivist-slack/     # Slack adapter (Socket Mode)
├── fanfic-archivist-irc/       # IRC adapter (irc-rs, TLS by default)
├── fanfic-archivist-fediverse/ # Mastodon + Bluesky + Piefed listeners
└── fic-archivist/                 # terminal CLI (+ ws module for web chat)

Prerequisites

• Rust (stable; workspace resolver = "2", edition 2024) — https://rustup.rs
• Redis — used for the pagination cache, token store, and link codes:

  redis-server   # or your distro package / existing FicHub Redis
  

• A FicHub instance — the bot talks only to its REST API. Point FANFIC_ARCHIVIST_BASE_URL at it (default http://localhost:8000).
• Ollama — optional, only if you enable FANFIC_ARCHIVIST_LLM_ENABLED for intent classification.

Quickstart

git clone <this repo> && cd fanfic-archivist

cp .env.example .env        # then edit .env: tokens, base URL, ...
redis-server &              # or ensure Redis is already up

cargo build --workspace     # or --release
cargo test --workspace      # unit + integration tests (no services needed)

Run one adapter (each binary loads .env from its working directory):

# Discord (needs DISCORD_TOKEN)
cargo run -p fanfic-archivist

# Matrix (needs MATRIX_USERNAME/PASSWORD or MATRIX_ACCESS_TOKEN)
cargo run -p fanfic-archivist-matrix

# Telegram (needs TELEGRAM_BOT_TOKEN)
cargo run -p fanfic-archivist-telegram

# Slack (needs SLACK_BOT_TOKEN + SLACK_APP_TOKEN)
cargo run -p fanfic-archivist-slack

# IRC (sensible defaults; set FANFIC_ARCHIVIST_IRC_CHANNELS)
cargo run -p fanfic-archivist-irc

# Mastodon + Bluesky + Piefed (at least one listener must be configured)
cargo run -p fanfic-archivist-fediverse

# Terminal CLI
cargo run -p fic-archivist -- search "slow burn dramione"
cargo run -p fic-archivist -- tui                 # interactive three-pane TUI
cargo run -p fic-archivist -- --help              # full command list

For release builds add --release (or build once and run the binaries in target/release/). Adapters that are missing a required credential exit at startup with a message naming the variable.

Configuration

All env vars, grouped by crate, with defaults and REQUIRED markers, live in .env.example — copy it to .env and fill in.

• Shared core: FANFIC_ARCHIVIST_BASE_URL, FANFIC_ARCHIVIST_REDIS_URL, page sizes, upload caps, freeform/LLM toggles, Lemmy monitor.
• Per platform: DISCORD_TOKEN, MATRIX_*, TELEGRAM_*, SLACK_*, FANFIC_ARCHIVIST_IRC_*, MASTODON_*, BSKY_*, PIEFED_*, FANFIC_ARCHIVIST_API_TOKEN (CLI).

Documentation

Doc	Contents
ADAPTERS.md	Per-platform setup & operations: status table, credentials, systemd units, Redis namespaces, render matrix
PLATFORM-SETUP.md	How to create bot accounts on each platform, step by step
docs/CLI-TUI.md	fic-archivist subcommands, batch download, forum, TUI keybindings, REPL
docs/QUICKSTART.md	10-step run-your-own-bot walkthrough
ROADMAP.md	Backlog and direction

Testing

cargo test --workspace

Unit tests cover config parsing, URL joining, credential detection, and dispatch logic — they run with no services (Redis/FicHub/Ollama) required. Start an adapter and smoke-test manually: send it a known URL or !recs in your test channel and confirm the metadata reply and pagination buttons.

Deploy

deploy/install.sh installs built binaries + systemd units to /opt/fanfic-archivist (see deploy/systemd/*.service); ADAPTERS.md has the manual systemd recipe.

License

AGPL-3.0-or-later (matches FicHub).