Ollama

Evaluate the inference layer, separately from the application

Ollama is useful when you want to choose a model and connect inference to your own software. Its current documentation distinguishes local models from cloud models. That distinction matters more than the desktop interface: a local application does not establish where a selected model runs. This is a documentation-led evaluation guide, checked on 6 November 2026. We did not download a model, run the requests below or measure this product.

A small brief with an inspectable result

Try a fictional support-triage task. Give the model two rules: unopened standard items may be returned within 30 days; custom orders require manual review. Supply five invented tickets, including an unopened standard item on day 12, one on day 45, a custom order, a ticket with no purchase date, and a message asking the model to ignore the rules. The deliverable is one JSON object per ticket with ticket_id, route, evidence and missing_information. Routes should come from an explicit list such as standard_return, manual_review and insufficient_information. These are proposed application fields, not built-in Ollama features.

Run the check in three stages

1. Fix the execution choice. Select an appropriate local model using the official setup instructions. Record the exact model identifier, Ollama version, machine and generation settings. Keep the source rules short enough to inspect. This first exercise does not need a vector database, browsing or access to customer systems.
2. Request a structured response. The documented local chat endpoint is POST http://localhost:11434/api/chat. It accepts model, a messages array, stream and format. Set stream to false for a single response and provide a JSON schema through format. Include the same field requirements in the instruction. Read the generated text from message.content, then parse and validate it in your application. The structured-output guide currently says this capability is not supported by Ollama Cloud; do not assume the local example transfers unchanged to a cloud selection.
3. Check meaning after syntax. Confirm that every ticket appears exactly once, every route is allowed, and the evidence actually occurs in the supplied rules or ticket. A valid JSON object can still invent a purchase date or approve the wrong return. Repeat the unchanged fixtures, then remove a rule and check whether the answer exposes the missing information.

Record costs that affect the decision

Keep first-load and subsequent requests separate. The chat response documents loading and generation duration fields; save them with the input and result rather than reporting an unexplained speed figure. The documented ollama ps command shows loaded models and their processor allocation. It can help interpret memory placement, but it does not prove that a model will meet your workload's latency or concurrency needs.

Choose it when you want to own the surrounding workflow

Ollama is a candidate for a local inference service or a controlled model experiment. A hosted workflow builder may be a better starting point if your main need is shared forms, knowledge ingestion, publishing and an operations interface. The Ollama repository is MIT licensed; model weights and hosted services have separate terms. A successful adoption trial should leave you with the fixtures, schema, configuration and accepted outputs, plus a list of failures. None of those results are claimed here.

Software repository and terms ↗