add integrated chatbot with streaming + tools

README.md

@@ -48,6 +48,7 @@ docker run --rm -p 3000:3000 tbd
 | `POST` | `/oauth/register` | Dynamic client registration (RFC 7591) for MCP clients |
 | `GET`, `POST` | `/oauth/authorize` | Where an MCP client sends the user's browser: Gitea login if needed, then a consent page |
 | `POST` | `/oauth/token` | Trades an authorization code (with its PKCE verifier) or a refresh token for tokens |
+| `POST` | `/api/chat` | **Login required.** Body `{"messages": [{"role": "user", "content": "hi"}]}` (the conversation so far, 1 to 40 messages of up to 8000 characters, roles `user`, `assistant` or `tool`; and optionally `"page"`, the id of the UI page the user is on (`"games"`, `"ai-setup"`: a lowercase slug of up to 64 characters), which the LLM is told in a system message, and `"tools"`, the page's tools as MCP describes them: see [the chatbot's tools](#the-chatbots-tools)) sends it to the [chatbot's](#chatbot) LLM and **streams the reply back** as server-sent events (`text/event-stream`): `data: {"thinking":"Let me"}` for each piece of the model's reasoning, `data: {"content":"Hel"}` for each piece of the reply, `data: {"tool-calls":[{"id":"call_1","name":"add","arguments":"{\"x\":1,\"y\":2}"}]}` if the model asks for tools to be called, and last `data: {"done":true}` or `data: {"error":"..."}`. Failures before the reply starts are ordinary errors: 400 for an invalid conversation, 502 if the LLM server can't be reached or refuses, 503 if `LMSTUDIO_URL` isn't set |
 | `GET` | `/api-docs` | Swagger UI |
 | `GET` | `/swagger.json` | OpenAPI/Swagger spec |
 | `GET` | `/auth/login` | Starts the Gitea OAuth login (redirects to Gitea) |
@@ -184,6 +185,7 @@ login, **403** for a logged-in non-admin. With no allowlist, nobody is an admin.
 | `SESSION_SECRET` | Signs and encrypts session cookies. Unset = a random key per start, so logins don't survive a restart |
 | `TODO_DB_PATH` | SQLite file for the todo group (default `data/todo.db`) |
 | `OAUTH_DB_PATH` | SQLite file for the MCP OAuth server: registered clients, codes and tokens (default `data/oauth.db`). Mount a volume for it in Docker too, or every MCP client has to be approved again after each restart |
+| `LMSTUDIO_URL`, `LMSTUDIO_MODEL`, `LMSTUDIO_API_KEY`, `LMSTUDIO_TIMEOUT_SECONDS` | The LM Studio server behind the [chatbot](#chatbot). Only the URL is needed |
 | `NREPL_PORT`, `NREPL_BIND` | Where the debug nREPL listens (default `7888` on `127.0.0.1`) |
 
 Sessions live in an encrypted, signed, `HttpOnly`, `SameSite=Lax` cookie and expire after 8 hours
@@ -220,6 +222,9 @@ It has three pages, as tabs across the top. Each has its own address in the URL'
 
 A "Log in" / "Log out" link sits in the footer row next to the API docs links.
 
+A green square in the bottom left corner (on every page, opposite the blue WebMCP one) opens the
+[chatbot](#chatbot).
+
 shadow-cljs compiles it into `resources/public/js/` (git-ignored). The server serves that
 directory at `/js`, and serves `resources/public/index.html`, the shell that mounts the app, at `/`.
 
@@ -231,6 +236,89 @@ directory at `/js`, and serves `resources/public/index.html`, the shell that mou
 The Docker image builds the UI in its own stage, so `docker build` needs no local Node. The UI's tests run
 in another stage of the same Dockerfile (`--target ui-test`), which the server image doesn't include.
 
+### Chatbot
+
+The green square opens a small chat panel. What you type goes to `POST /api/chat`, and the backend
+forwards the conversation to an [LM Studio](https://lmstudio.ai) server (its OpenAI-compatible
+`/v1/chat/completions`) and returns the reply. The browser never talks to the LLM directly, so the
+server's address (and key) stay on the backend. The reply is streamed: the panel shows "Thinking…" until the
+model's first word (a reasoning model can take a while), then the reply appears as it is generated. What
+the model reasons is never shown, only that it is thinking; the tool calls (see below) show as small grey
+lines between the replies, in full, however long, and the conversation scrolls. The conversation lives in the page only (a reload or logging out clears it), and the UI sends
+the last 40 messages of it. Chat needs a login, like the todos: every message costs the LLM server's time.
+
+The conversation scrolls. It follows the reply while you are at its end; scroll up to read back and it
+stays put until you send a message or scroll down again. Drag the panel's top right corner to make it
+wider or taller (the size is kept while the page is open, through Clear and closing the panel).
+
+`curl -N` shows the stream (`-N` turns off curl's own buffering; use your session cookie):
+
+```bash
+curl -N -H "Cookie: $COOKIE" -H 'Content-Type: application/json' \
+  -d '{"messages":[{"role":"user","content":"hi"}]}' http://localhost:3000/api/chat
+```
+
+#### The chatbot's tools
+
+The chatbot can use the page. The [browser tools](#browser-mcp-tools-webmcp) (the calculator, the todos,
+tic-tac-toe) are registered in the page's *model context*, `ui/src/tbd/ui/model_context.cljs`, which is
+what WebMCP's `navigator.modelContext` is to an agent outside the page. The chat is that agent, embedded:
+
+1. Each `POST /api/chat` carries `"page"`, the page the user is on, and `"tools"`: the tools' metadata, exactly what an MCP server answers
+   `tools/list` with (`{"name", "description", "inputSchema"}`, the schema being JSON Schema).
+   The backend gives it to the LLM as OpenAI-style `tools` (LM Studio's own tool use), and stays stateless.
+   Only the tools of the page being shown are listed: a tool registered with `:pages #{:games}` is
+   offered on the Games page and nowhere else (the calculator and tic-tac-toe are games' tools, the
+   todos' are the Todos page's), and one with no `:pages` (`ping`, `whoami`, `navigate`) on every page. The
+   list follows the page the user is on each time the conversation is sent (so does `"page"`, which the
+   backend puts in the system message ahead of the conversation for the LLM to know where the user is,
+   with the names of the tools it has),
+   and a call for a tool the page doesn't offer is refused with an error the LLM reads. The LLM
+   can move between pages with `navigate`; its next request then carries the new page and that
+   page's tools. An agent outside the page, through WebMCP, is still offered all of them.
+2. The LLM never runs anything. When it wants a tool, it says so: the reply ends with a
+   `{"tool-calls": [...]}` event (the calls are put together from the pieces the model writes, the
+   `arguments` being the JSON text it wrote). The backend only relays them.
+3. The page intercepts them and calls the tools right there (one after the other), through the same
+   handlers WebMCP uses, so they act on the page as a user would and you see it happen. The chat panel
+   shows each call and its result in small print between the messages.
+4. The results go back as messages with the role `tool` (`{"role": "tool", "tool-call-id": "call_1",
+   "content": "..."}`, after the assistant message that asked, which carries `"tool-calls"`), in
+   another request with the conversation so far, for the LLM to go on from, until it answers with words.
+
+A call that can't be made (a tool that isn't there, arguments that aren't a JSON object, a tool that
+fails) is a result with the reason, for the model to read and try again from. To keep a model that never
+stops calling tools from acting on your page for ever, one message of yours gets it at most 12 calls;
+then the chat stops, and says so. Results are cut to the 8000 characters the server accepts.
+
+The chatbot is told to be an agent, not a conversationalist: the system message (`default-system-prompt` in
+`src/tbd/chat/core.clj`) says that what it does shows on your screen at once, so it should call a tool in
+the same turn rather than offer to, chain calls (navigate, list, then act), take obvious defaults instead
+of asking, and answer in one short sentence, since the call and its result are already shown. If a model
+still answers requests with offers, that prompt (and its four examples, which name tools) is what to
+tune for it; the tool descriptions in `webmcp.cljs` are the other lever.
+
+To try it, use a model that supports tool use in LM Studio, log in, and ask the chat something the page
+can do: *"add 2 and 3 with the calculator"*, *"add a todo to buy milk"*, *"play tic-tac-toe with me, you're O"*.
+
+A reverse proxy in front of tbd must not buffer responses, or the reply arrives all at once at the end
+(the server sends `X-Accel-Buffering: no`, which nginx obeys; for others, turn buffering off for `/api/chat`).
+The `thinking` events (the reasoning) also keep the connection from looking idle to a proxy while the model reasons; the UI drops them.
+Anything that goes wrong is logged on the server's stdout, with a `[tbd] chat:` prefix.
+
+Point the backend at your instance with environment variables (see [.env-template](.env-template)):
+
+```bash
+LMSTUDIO_URL=http://192.168.1.20:1234 script/server
+```
+
+Only `LMSTUDIO_URL` is required (the address of LM Studio's server, without `/v1`; for a remote instance
+turn on "Serve on Local Network" there). `LMSTUDIO_MODEL` picks a model by id (default: the one LM Studio
+has loaded), `LMSTUDIO_API_KEY` is sent as a bearer token if the server wants one, and
+`LMSTUDIO_TIMEOUT_SECONDS` (default 120) is how long the LLM server may stay silent, before it starts
+answering or between two pieces of it, before the server gives up. Without `LMSTUDIO_URL` the server
+still starts (it says so), and the chat panel shows "chat is not configured".
+
 ### Browser MCP tools (WebMCP)
 
 The page also offers tools of its own to your MCP client (Claude), through [WebMCP](https://webmcp.dev):
@@ -242,6 +330,7 @@ registered with `registerTool` on a `WebMCP` instance (the `@jason.today/webmcp`
 | Tool | What it does |
 |------|--------------|
 | `ping`, `whoami` | Health check; who is logged in on the page, if anyone |
+| `navigate` | Opens a page (`games`, `todos` or `ai-setup`) as a click on its tab would; `todos` needs a login, like its tab. The chatbot is then offered the new page's tools |
 | `get_calculator` | What the calculator form holds: its two fields and the result it shows |
 | `add` | Types `x` and `y` into the calculator, presses Add, and returns the sum the page shows |
 | `list_todos`, `get_todo` | The todos the page shows (login required, like the todo list itself) |
@@ -320,9 +409,10 @@ src/tbd/
   auth/               Gitea OAuth (core), session + admin middleware (middleware), /auth routes
   mcp/                MCP server (PluMCP) mounted at `/mcp`: tools (tools.clj), Malli-to-tool glue (schema.clj) and the Ring handler (routes.clj)
   oauth/              tbd as the OAuth authorization server for MCP clients: rules (core), SQLite storage (db), `/oauth` + discovery routes, and the bearer-token middleware guarding `/mcp` (middleware)
+  chat/               The chatbot: request to the LM Studio server, tool calls included (core), the request's Malli schema (schema), `/api/chat` route
   nrepl/              On-demand nREPL server (core) and the admin-only `/nrepl` routes
   todo/               Per-user todos: pure rules (core), Malli schemas of the input (schema), SQLite storage (db), `/api/todos` routes
-ui/                   ClojureScript re-frame app (shadow-cljs project); ui/src/tbd/ui/{db,events,subs,views,webmcp,core}.cljs, tests in ui/test/
+ui/                   ClojureScript re-frame app (shadow-cljs project); ui/src/tbd/ui/{db,events,subs,views,webmcp,model_context,core}.cljs, tests in ui/test/
 .env-template         Settings the server reads from the environment; copy to .env (git-ignored)
 resources/public/     index.html (the shell served at /) and js/ (compiled UI output, git-ignored)
 test/                 Unit tests