archivist-core

Platform-neutral core for the FicHub companion bot ("the Archivist").

All business logic shared by every platform adapter — Discord, Telegram, Matrix, Slack, IRC, fediverse (Lemmy/Piefed), CLI and web — lives in this crate. It contains zero platform SDKs (no discord-rs, no teloxide, …): adapters depend on this crate, call the do_* functions, and render the returned PlatformMessage intermediate representation (IR) natively.

What's inside

Module	Purpose
api	Typed FicHub REST API client (FichubClient) — search, export/download, recommendations, bookmarks, feed, requests, roadmap, auth, forum (forum_* methods: categories, cursor-paginated topics, topic detail + after=, create, reply, follow, mark-read, search, moderation, metamod)
model	Typed request/response structs matching the FicHub REST API (incl. forum models ForumCategory, ForumTopic, ForumPost, …)
dispatch	CoreCtx + every do_* command: do_search, do_ask, do_quote, do_body, do_recs, do_roll, do_download, do_bookmark, do_metadata, do_help, do_kudos, and do_forum_* (categories, topics, topic, create, reply, follow, mark-read, search)
core	The platform-neutral PlatformMessage IR: Text / Rich (header + RichItems + action rows) / File / Ephemeral
intent	Free-form mention classification (Intent enum) — optional LLM (Ollama) with a heuristic fallback ladder; everything the LLM can produce is whitelist-checked before it runs
cache	Redis-backed pagination cache shared with FicHub (session-based paging, response cache keyed by sha256 of the normalized query)
store	Platform-scoped token store (/link flow): archivist:token:<platform>:<ext_id>
ratelimit	Cross-platform rate limiting via shared Redis buckets
config	Environment-driven BotConfig
error	BotError — no platform-specific variants
util	Supported-site URL detection, word formatting, truncation
lemmy	Lemmy/Piefed community monitor (poll-based)
debug	LLM-assisted diagnosis of failed searches

How commands flow

platform event ──► adapter ──► dispatch::do_*(&CoreCtx, ...) ──► PlatformMessage ──► adapter renders natively

The adapter owns platform plumbing (events, message handles, attachments, editing) and calls one do_* function. The core never touches the platform, so there is exactly one implementation of every command's behavior, shared by all adapters — and new platforms get every command for free.

CoreCtx bundles the shared pieces: a FichubClient, the PageCache, and an Arc<BotConfig>.

Adding a new platform adapter

1. Implement BotConfig::default() loading (or reuse BotConfig::default() for local dev).
2. On a platform event, translate it into a PlatformMessage and return it — that's the whole adapter contract.

use std::sync::Arc;
use archivist_core::api::FichubClient;
use archivist_core::cache::PageCache;
use archivist_core::config::BotConfig;
use archivist_core::core::PlatformMessage;
use archivist_core::dispatch::{do_search, CoreCtx};
use archivist_core::{BotError, Result};

// One adapter function: turn a platform event into a PlatformMessage.
async fn on_message(cfg: &BotConfig, user_id: u64, text: &str) -> Result<PlatformMessage> {
    let client = FichubClient::new(Arc::new(cfg.clone()))?;
    let cache = PageCache::connect(cfg).await?;
    let ctx = CoreCtx::new(client, cache, Arc::new(cfg.clone()));

    match text.trim() {
        "/search" => {
            // do_* functions are the single source of truth for behavior.
            let params = archivist_core::api::SearchParams::default();
            do_search(&ctx, user_id, &params, None).await
        }
        other => Err(BotError::Command(format!("unknown command: {other}"))),
    }
}

Configuration

Everything is read from environment variables at startup (see BotConfig for the full list and defaults):

Env var	Default	Purpose
FANFIC_ARCHIVIST_BASE_URL	http://localhost:8000	FicHub REST API root (no trailing slash)
FANFIC_ARCHIVIST_REDIS_URL	redis://localhost:6379	Redis for pagination cache + token store
FANFIC_ARCHIVIST_LLM_ENABLED	0	Optional Ollama intent classification / debug
FANFIC_ARCHIVIST_OLLAMA_URL	http://localhost:11434	Ollama base URL
FANFIC_ARCHIVIST_INTENT_MODEL	lfm2.5:8b	Model for intent classification
FANFIC_ARCHIVIST_PAGE_SIZE	5	Results per rendered page
FANFIC_ARCHIVIST_API_PAGE_SIZE	20	Results fetched from the API per page
FANFIC_ARCHIVIST_ALLOW_UPLOAD	1	Allow direct EPUB upload on /download
FANFIC_ARCHIVIST_MAX_UPLOAD_BYTES	26214400	Max direct-upload bytes (25 MiB)
FANFIC_ARCHIVIST_FREEFORM	1	Enable free-form @mention handling
FANFIC_ARCHIVIST_LINK_TTL	600	/link code TTL in seconds
FANFIC_ARCHIVIST_INTENT_TIMEOUT_SECS	15	Intent-classification timeout
FANFIC_ARCHIVIST_INTENT_CACHE_TTL	3600	Intent-cache TTL in seconds
FANFIC_ARCHIVIST_LEMMY_*	—	Lemmy monitor (URL, account, community, poll interval)

All LLM features are optional: with FANFIC_ARCHIVIST_LLM_ENABLED=0 (default) the free-form path still handles fanfiction URLs, docs questions, and natural-language /ask fallback without any Ollama dependency.

Testing

cargo test -p archivist-core

Unit tests cover URL detection/normalization, pagination-row rendering, the PlatformMessage IR (describe, RichItem builder), config URL joining, and cache logic. Integration against a live FicHub instance requires FANFIC_ARCHIVIST_BASE_URL + FANFIC_ARCHIVIST_REDIS_URL pointing at real services.

License

AGPL-3.0-or-later. See the LICENSE file in the repository.