# llms
## Tool selection
Which tool functions to offer the LLM. `_LOCAL_TOOLS` /
`DEFAULT_TOOL_SELECTION` define the available sources and the default
set;
[`get_tool_list`](https://drscotthawley.github.io/boopiter/llms.html#get_tool_list)
assembles the actual function list from whichever sources are selected.
------------------------------------------------------------------------
source
### get_tool_list
``` python
def get_tool_list(
selection:dict=None, # which tool sources to include -- keys 'boopiter'/'slmn-nbtools'/'slmn-misc'/'slmn-remote', bool values (see the wrench-icon Tools menu in the GUI). Defaults to DEFAULT_TOOL_SELECTION if omitted. 'slmn-remote', even when selected, only ever contributes its safe/read-only subset (_SLMN_REMOTE_SAFE) -- remote_launch/remote_status/remote_smoke_test (arbitrary remote command execution over ssh) are never included here, by design, regardless of selection.
)->list:
```
*Assemble the list of tool functions to offer an LLM, from whichever
sources are selected. Returns actual callables, not names – pass
straight to prompt_llm(tools=…). Per-notebook ad-hoc tools (see
add_tool()) are layered on top of this by the caller, not included
here.*
------------------------------------------------------------------------
source
### sync_ollama_loaded
``` python
def sync_ollama_loaded(
keep:list
)->list:
```
*Enforce the invariant that* only\* the models in `keep` stay resident
in Ollama: unload every currently-loaded model that isn’t one of them.
Returns the names actually unloaded. `keep` accepts bare model names or
‘ollama/’-prefixed ids interchangeably, and ignores both None and
non-Ollama ids like ‘deaddrop/…’, so a caller can pass
nb.standard_model/nb.reasoning_model straight through. Sweeping the live
/api/ps list – instead of remembering which model we last switched away
from – is the point: it also clears models left resident by a previous
boopiter process, by another client of the same Ollama server, or by an
earlier unload that quietly failed. It’s idempotent, so calling it on
every model-menu change costs one cheap GET when there’s nothing to
do.\*
------------------------------------------------------------------------
source
### unload_ollama_model
``` python
def unload_ollama_model(
name:str
)->bool:
```
*Evict one model from Ollama’s memory, given its bare name (‘qwen3:14b’
– NOT the ‘ollama/’-prefixed id used everywhere else, see
get_ollama_list()). Done over HTTP, as a generate request with
keep_alive=0 (Ollama’s documented way to drop a model immediately),
rather than by shelling out to `ollama stop`: the CLI has to be on the*
server process’s\* PATH – which it often isn’t when boopiter is launched
from a desktop launcher or systemd unit rather than a login shell – and
it always talks to localhost, whereas this honours OLLAMA_URL like the
rest of this module. Returns whether it worked, and warns (rather than
failing silently) if not, since a silent no-op here looks exactly like a
memory leak.\*
------------------------------------------------------------------------
source
### ollama_loaded
``` python
def ollama_loaded()->list:
```
*Names of the models Ollama is currently holding in memory, per its
/api/ps endpoint – not to be confused with get_ollama_list()‘s
/api/tags, which lists every model on disk whether loaded or not.
Returns \[\] rather than raising if Ollama is unreachable, so callers
can treat ’no server’ and ‘nothing loaded’ the same way.*
## Listing available models
[`get_ollama_list`](https://drscotthawley.github.io/boopiter/llms.html#get_ollama_list)
reads every locally-available Ollama model from `/api/tags`;
[`get_model_list`](https://drscotthawley.github.io/boopiter/llms.html#get_model_list)
wraps it (and any other sources) into a uniform list of model-info
dicts.
------------------------------------------------------------------------
source
### get_ollama_list
``` python
def get_ollama_list(
strict:bool=False
)->list:
```
\*Get info on every locally-available Ollama model: the raw /api/tags
entry (name, details incl. parameter_size/family) plus an added ‘id’ key
(‘ollama/
’, the string used elsewhere as the model identifier –
see nb.model) and a ‘capabilities’ list
(e.g. ‘vision’/‘tools’/‘thinking’). Capabilities are NOT in /api/tags –
only the per-model /api/show endpoint reports them (see \_ollama_caps,
which caches them and is why calling this repeatedly is cheap). Those
probes are independent per model and each is almost entirely network
wait, so cache misses are fetched on a small thread pool rather than in
series – with a cold cache that turns N sequential round trips into
roughly one. Returns \[\] and warns if Ollama is unavailable – unless
`strict`, which raises instead, so a caller that already has a good
listing (refresh_models(), re-checking on every brain-menu hover) can
tell ‘Ollama is down’ apart from ‘Ollama has no models’ and keep showing
what it had rather than blanking the menu on a transient hiccup. A model
whose individual capability probe fails just gets an empty capability
list, not a failed listing.\*
------------------------------------------------------------------------
source
### get_model_list
``` python
def get_model_list(
strict:bool=False
)->list:
```
\*Wrapper routine to get info dicts (see get_ollama_list()) for all
available models from all sources. Live dead-drop sessions (see
\_deaddrop_sessions) are listed as models too, one
‘deaddrop/
’ entry each: selecting one in the brain-menu
dropdown is what binds this notebook’s Prompt cells to that session’s
inbox/outbox. Modelling the binding as a model choice – rather than as a
global set once per process – means it’s visible in the UI, saved with
the notebook, and per-notebook, so several notebooks on different topics
can each talk to their own session without crossing wires, and a restart
can’t silently reroute one of them. ‘deaddrop/claude’ remains the
original single-file prompts/ + responses/ route.\*
## Forcing streaming responses
A monkeypatch (`_call`, over the saved `_orig_chat_call`) makes the chat
client always yield streamed responses, working around a limitation in
the underlying library.
## Reply metadata
[`_details_block`](https://drscotthawley.github.io/boopiter/llms.html#_details_block)
builds the collapsible `` panel of call metadata (model, finish
reason, token counts, tool calls);
[`_reply_details_html`](https://drscotthawley.github.io/boopiter/llms.html#_reply_details_html)
fills it in for a real API response. Rendered with escaped FT
components, not raw HTML.
### Why tools aren’t passed as native API `tools=`
Early on, enabling any tool source made local models (tested worst-case:
`qwen2.5-coder:latest`, but reproduced up through `qwen3.6:27b`)
reflexively try to call a tool on *every* prompt, including plain “write
me some code” requests that had nothing to do with any tool – producing
garbled, empty, or JSON-as-prose replies instead of a normal answer.
The root cause: Ollama/Qwen’s chat template wraps any non-empty `tools=`
API parameter into a `...` special-token block. That
structural block biases these models into feeling obligated to fill a
`` slot, regardless of `tool_choice` (`'auto'` vs. the
default `None` made no measurable difference) and regardless of a system
prompt saying “only use a tool if you actually need one” (that improved
*presentation* – coherent prose vs. raw JSON – but didn’t stop the
reflex). Bigger models weren’t immune either, just failed differently
(e.g. the tool call landing in `reasoning_content` while `content` came
back empty).
The fix, found by reading how SolveIt’s `dialoghelper`/`pyskills` solve
the same problem: never populate the native `tools=` parameter. Instead,
describe the available functions in the system prompt as plain Python
callables already present in the execution namespace (see
[`tools_system_prompt()`](https://drscotthawley.github.io/boopiter/llms.html#tools_system_prompt)
below), and let the model express “I need a tool” by writing an ordinary
\`\``python code block as part of its reply -- the same code-generation skill these models are already reliably good at, rather than their much less reliable native function-calling judgment. [`stream_llm_reply()`](https://drscotthawley.github.io/boopiter/llms.html#stream_llm_reply) (and [`\_push_tools()`](https://drscotthawley.github.io/boopiter/cells.html#_push_tools) in`cells.py\`,
which keeps those functions live in the shared kernel namespace)
implement that pattern end to end.
------------------------------------------------------------------------
source
### stream_llm_reply
``` python
def stream_llm_reply(
context:str, model:str, tools:list | None=None, think:str | None=None, images:list | None=None,
keep_alive_min:int | None=None, num_ctx:int | None=None
):
```
\*Generator streaming a model reply token-by-token, using the
code-callable tool strategy (see tools_system_prompt) instead of native
tool-calling. `images` is an optional list of raw image bytes to send
alongside the context – pass them only for models that advertise the
vision capability (the caller checks; see cells.py’s
\_start_prompt_run), as non-vision models error or ignore them. They
ride as extra message parts via lisette’s mk_msg (bytes -\> base64 data
URL), with the context text last. The dead-drop route ignores them for
now – file-based image handoff is its own upcoming feature. Yields
(‘delta’, text) chunks as they arrive, then a final (‘final’, content,
details_html) tuple once the response completes. Callers
(e.g. cells.py’s \_run_prompt_bg) drive this into their own
background-thread/UI state – that part isn’t LLM-specific, so it stays
out of this module. `think` (‘l’/‘m’/‘h’ or None) – see prompt_llm(). A
`model` starting with ‘deaddrop/’ is routed through
\_deaddrop_stream_reply() instead of a real API call – everything else
about this function’s contract (the yielded tuple shapes) is identical
either way, so no caller needs to know or care which one answered.\*
------------------------------------------------------------------------
source
### prompt_llm
``` python
def prompt_llm(
context:str, model:str='ollama_chat/qwen2.5-coder:latest', tools:list | None=None, think:str | None=None,
keep_alive_min:int | None=None, num_ctx:int | None=None
)->tuple:
```
\*Send a prompt to the LLM; returns (content, details_html) – the reply
text itself, and a separate collapsible
block of call metadata (model/tokens/finish reason/reasoning) meant to
be stored apart from the reply (see Cell.details), not mixed into it.
`tools`, if given, are described via tools_system_prompt()
(code-callable, not native tool-calling) – see there for why. `think`,
if given (‘l’/‘m’/‘h’), is lisette’s own reasoning-effort control –
passed straight through to litellm, which maps it to Ollama’s ‘think’
request field; only meaningful for a model that actually supports
thinking (see the brain-icon reasoning-model picker).\*
------------------------------------------------------------------------
source
### tools_system_prompt
``` python
def tools_system_prompt(
tools:list | None
)->str:
```
*Build a system-prompt block describing `tools` as plain Python
functions already available in the model’s execution environment – NOT
passed as native API tool schemas. This sidesteps a real reliability
problem found via testing: Ollama/Qwen’s chat template wraps any
non-empty `tools=` into a `
` special-token block that biases even
capable local models into reflexive, often-wrong tool calls, regardless
of tool_choice or how carefully the system prompt says ‘only if needed’
(both were tried and didn’t help). Routing tool use through the model’s
own code-writing judgment instead – which local models are much more
reliably good at – fixed it in practice. Same approach
dialoghelper/pyskills use for SolveIt (not a dependency here, just the
inspiration). Returns ’’ if `tools` is empty, so no system prompt is
added at all when there’s nothing to describe.*
### Tool Use
Example tool from lisette docs:
``` python
def add_numbers(
a: int, # First number to add
b: int # Second number to add
) -> int:
"Add two numbers together"
return a + b
```
``` python
res = prompt_llm("What's 47 + 23? Use the tool.", tools=[add_numbers])
print(res[0])
```
The sum of 47 and 23 is 70. I have completed my task as requested. If you need further assistance or have additional questions, feel free to ask!
``` python
# Confirms get_tool_list() assembles real, callable tools according to a selection dict.
from boopiter.llms import get_tool_list, DEFAULT_TOOL_SELECTION
print(len(get_tool_list()), "tools with defaults:", [t.__name__ for t in get_tool_list()])
print(len(get_tool_list({'boopiter': True, 'slmn-nbtools': False, 'slmn-misc': False, 'slmn-remote': True})), "tools with only boopiter+remote")
```