get_model_list()['ollama/qwen2.5-coder:latest',
'ollama/gemma3:4b',
'ollama/llama3.1:latest',
'ollama/qwen2.5:latest']llms
Generic LLM-calling utilities (models, prompting, tool schemas) – deliberately kept independent of boopiter’s own Notebook/Cell model, so this module never imports from
cells.py. cells.py imports from here, never the other way around.
Tool selection
Which tool functions to offer the LLM. _LOCAL_TOOLS / DEFAULT_TOOL_SELECTION define the available sources and the default set; get_tool_list assembles the actual function list from whichever sources are selected.
get_tool_list
def get_tool_list(
selection:dict=None, # which tool sources to include -- keys 'boopiter'/'slmn-nbtools'/'slmn-misc'/'slmn-remote', bool values (see the wrench-icon Tools menu in the GUI). Defaults to DEFAULT_TOOL_SELECTION if omitted. 'slmn-remote', even when selected, only ever contributes its safe/read-only subset (_SLMN_REMOTE_SAFE) -- remote_launch/remote_status/remote_smoke_test (arbitrary remote command execution over ssh) are never included here, by design, regardless of selection.
)->list:Assemble the list of tool functions to offer an LLM, from whichever sources are selected. Returns actual callables, not names – pass straight to prompt_llm(tools=…). Per-notebook ad-hoc tools (see add_tool()) are layered on top of this by the caller, not included here.
sync_ollama_loaded
def sync_ollama_loaded(
keep:list
)->list:Enforce the invariant that only* the models in keep stay resident in Ollama: unload every currently-loaded model that isn’t one of them. Returns the names actually unloaded. keep accepts bare model names or ‘ollama/’-prefixed ids interchangeably, and ignores both None and non-Ollama ids like ‘deaddrop/…’, so a caller can pass nb.standard_model/nb.reasoning_model straight through. Sweeping the live /api/ps list – instead of remembering which model we last switched away from – is the point: it also clears models left resident by a previous boopiter process, by another client of the same Ollama server, or by an earlier unload that quietly failed. It’s idempotent, so calling it on every model-menu change costs one cheap GET when there’s nothing to do.*
unload_ollama_model
def unload_ollama_model(
name:str
)->bool:Evict one model from Ollama’s memory, given its bare name (‘qwen3:14b’ – NOT the ‘ollama/’-prefixed id used everywhere else, see get_ollama_list()). Done over HTTP, as a generate request with keep_alive=0 (Ollama’s documented way to drop a model immediately), rather than by shelling out to ollama stop: the CLI has to be on the server process’s* PATH – which it often isn’t when boopiter is launched from a desktop launcher or systemd unit rather than a login shell – and it always talks to localhost, whereas this honours OLLAMA_URL like the rest of this module. Returns whether it worked, and warns (rather than failing silently) if not, since a silent no-op here looks exactly like a memory leak.*
ollama_loaded
def ollama_loaded()->list:Names of the models Ollama is currently holding in memory, per its /api/ps endpoint – not to be confused with get_ollama_list()‘s /api/tags, which lists every model on disk whether loaded or not. Returns [] rather than raising if Ollama is unreachable, so callers can treat ’no server’ and ‘nothing loaded’ the same way.
Listing available models
get_ollama_list reads every locally-available Ollama model from /api/tags; get_model_list wraps it (and any other sources) into a uniform list of model-info dicts.
get_ollama_list
def get_ollama_list(
strict:bool=False
)->list:*Get info on every locally-available Ollama model: the raw /api/tags entry (name, details incl. parameter_size/family) plus an added ‘id’ key (‘ollama/strict, which raises instead, so a caller that already has a good listing (refresh_models(), re-checking on every brain-menu hover) can tell ‘Ollama is down’ apart from ‘Ollama has no models’ and keep showing what it had rather than blanking the menu on a transient hiccup. A model whose individual capability probe fails just gets an empty capability list, not a failed listing.*
get_model_list
def get_model_list(
strict:bool=False
)->list:*Wrapper routine to get info dicts (see get_ollama_list()) for all available models from all sources. Live dead-drop sessions (see _deaddrop_sessions) are listed as models too, one ‘deaddrop/
Forcing streaming responses
A monkeypatch (_call, over the saved _orig_chat_call) makes the chat client always yield streamed responses, working around a limitation in the underlying library.
Why tools aren’t passed as native API tools=
Early on, enabling any tool source made local models (tested worst-case: qwen2.5-coder:latest, but reproduced up through qwen3.6:27b) reflexively try to call a tool on every prompt, including plain “write me some code” requests that had nothing to do with any tool – producing garbled, empty, or JSON-as-prose replies instead of a normal answer.
The root cause: Ollama/Qwen’s chat template wraps any non-empty tools= API parameter into a <tools>...</tools> special-token block. That structural block biases these models into feeling obligated to fill a <tool_call> slot, regardless of tool_choice ('auto' vs. the default None made no measurable difference) and regardless of a system prompt saying “only use a tool if you actually need one” (that improved presentation – coherent prose vs. raw JSON – but didn’t stop the reflex). Bigger models weren’t immune either, just failed differently (e.g. the tool call landing in reasoning_content while content came back empty).
The fix, found by reading how SolveIt’s dialoghelper/pyskills solve the same problem: never populate the native tools= parameter. Instead, describe the available functions in the system prompt as plain Python callables already present in the execution namespace (see tools_system_prompt() below), and let the model express “I need a tool” by writing an ordinary ``python code block as part of its reply -- the same code-generation skill these models are already reliably good at, rather than their much less reliable native function-calling judgment. [stream_llm_reply()](https://drscotthawley.github.io/boopiter/llms.html#stream_llm_reply) (and [_push_tools()](https://drscotthawley.github.io/boopiter/cells.html#_push_tools) incells.py`, which keeps those functions live in the shared kernel namespace) implement that pattern end to end.
stream_llm_reply
def stream_llm_reply(
context:str, model:str, tools:list | None=None, think:str | None=None, images:list | None=None,
keep_alive_min:int | None=None, num_ctx:int | None=None
):*Generator streaming a model reply token-by-token, using the code-callable tool strategy (see tools_system_prompt) instead of native tool-calling. images is an optional list of raw image bytes to send alongside the context – pass them only for models that advertise the vision capability (the caller checks; see cells.py’s _start_prompt_run), as non-vision models error or ignore them. They ride as extra message parts via lisette’s mk_msg (bytes -> base64 data URL), with the context text last. The dead-drop route ignores them for now – file-based image handoff is its own upcoming feature. Yields (‘delta’, text) chunks as they arrive, then a final (‘final’, content, details_html) tuple once the response completes. Callers (e.g. cells.py’s _run_prompt_bg) drive this into their own background-thread/UI state – that part isn’t LLM-specific, so it stays out of this module. think (‘l’/‘m’/‘h’ or None) – see prompt_llm(). A model starting with ‘deaddrop/’ is routed through _deaddrop_stream_reply() instead of a real API call – everything else about this function’s contract (the yielded tuple shapes) is identical either way, so no caller needs to know or care which one answered.*
prompt_llm
def prompt_llm(
context:str, model:str='ollama_chat/qwen2.5-coder:latest', tools:list | None=None, think:str | None=None,
keep_alive_min:int | None=None, num_ctx:int | None=None
)->tuple:block of call metadata (model/tokens/finish reason/reasoning) meant to be stored apart from the reply (see Cell.details), not mixed into it. tools, if given, are described via tools_system_prompt() (code-callable, not native tool-calling) – see there for why. think, if given (‘l’/‘m’/‘h’), is lisette’s own reasoning-effort control – passed straight through to litellm, which maps it to Ollama’s ‘think’ request field; only meaningful for a model that actually supports thinking (see the brain-icon reasoning-model picker).*
tools_system_prompt
def tools_system_prompt(
tools:list | None
)->str:Build a system-prompt block describing tools as plain Python functions already available in the model’s execution environment – NOT passed as native API tool schemas. This sidesteps a real reliability problem found via testing: Ollama/Qwen’s chat template wraps any non-empty tools= into a <tools> special-token block that biases even capable local models into reflexive, often-wrong tool calls, regardless of tool_choice or how carefully the system prompt says ‘only if needed’ (both were tried and didn’t help). Routing tool use through the model’s own code-writing judgment instead – which local models are much more reliably good at – fixed it in practice. Same approach dialoghelper/pyskills use for SolveIt (not a dependency here, just the inspiration). Returns ’’ if tools is empty, so no system prompt is added at all when there’s nothing to describe.
s ="Today is July 18. Who's one famous person with this birthday?"
c = prompt_llm(s)
print(str(c[0]))
s = """
Tell me the previous question I asked you, from the previous prompt.
I want to see if you retain state between calls"""
c = prompt_llm(s)
print(str(c[0]))One famous person born on July 18th is Mark Zuckerberg, the co-founder and CEO of Facebook.
I'm sorry for any confusion, but as an AI language model, I don't have the capability to remember or retain information across separate interactions. Each response is generated independently based on the input provided in each session. If you have a specific question or need assistance with something particular, feel free to ask!Tool Use
Example tool from lisette docs:
def add_numbers(
a: int, # First number to add
b: int # Second number to add
) -> int:
"Add two numbers together"
return a + bres = prompt_llm("What's 47 + 23? Use the tool.", tools=[add_numbers])
print(res[0])The sum of 47 and 23 is 70. I have completed my task as requested. If you need further assistance or have additional questions, feel free to ask!# Confirms get_tool_list() assembles real, callable tools according to a selection dict.
from boopiter.llms import get_tool_list, DEFAULT_TOOL_SELECTION
print(len(get_tool_list()), "tools with defaults:", [t.__name__ for t in get_tool_list()])
print(len(get_tool_list({'boopiter': True, 'slmn-nbtools': False, 'slmn-misc': False, 'slmn-remote': True})), "tools with only boopiter+remote")