Dev Update #1 - From Specification to Routing Core

What was Milestone 1 about

This milestone focused on the core layer of smista.ai, which defines the types common to the CLI and the router, with particular emphasis on the REST API they use to communicate.

Also, I have already implemented the CLI, router configuration model, and validation system.

The model picker is a lie

One of the biggest pain points in today's developer workflow when using LLMs is model selection.

Some providers offer better models for planning, while others perform better at code writing. Also, the same providers offer different solutions based on the needs.

The pain comes when you have to switch between them. It's not always a natural action, and losing context is very annoying.

It's true that, for instance, Claude, when executing plans, usually dispatches to different models based on complexity. Unfortunately, though, this doesn't always come with lucky choices. Sometimes the chosen model performs well, while others waste money and time due to poorly executed tasks.

For this reason, when designing smista.ai, the primary focus was on providing deterministic, functional, yet human-readable routing policies that let the user choose the correct model.

Why routing must never touch an LLM

This first milestone was particularly focused on implementing the types used by smista.ai to communicate between the CLI and the router. This means defining the interface of policies and routing rules.

One could argue that deterministic routing policies make things more annoying because one has to define many complicated rules to achieve that; delegating this job to the LLM would simplify things a lot. This actually is not true. LLMs are useful for many things, but routing decisions should be reproducible, auditable, and predictable. This approach would be non-deterministic. You may think it's okay if, for once, the wrong model was chosen for executing a task. But it really depends on the case. Did I waste a lot of money because of that? Did I waste time with a poor execution outcome? Did this leak information to an unauthorised model? All these concerns are not acceptable in most enterprise ecosystems.

Encoding a decision as data

Due to issues caused by non-deterministic task dispatching, building an accurate model of routing rules is essential. This model must allow us to make conscious choices, and that's exactly what was done for smista.ai.

[[routing.rules]]
name = "plan with strongest reasoning model"
priority = 10
intent = "plan"
model = "openai/gpt-5.5-thinking"
fallbacks = ["anthropic/claude-sonnet"]
 
[[routing.rules]]
name = "use local model for changelog skill"
priority = 20
skill = "changelog"
model = "ollama/qwen2.5-coder"
fallbacks = ["openai/gpt-5.5-mini"]
 
[[routing.rules]]
name = "auth code uses Claude"
priority = 30
intent = "edit"
paths = ["src/auth/**"]
model = "anthropic/claude-sonnet"
fallbacks = ["openai/gpt-5.5-thinking"]
 
[[routing.rules]]
name = "review security-sensitive code locally"
priority = 5
effort = "low"
intent = "review"
paths = ["src/crypto/**", "src/auth/**"]
local_only = true
model = "ollama/qwen2.5-coder"
 
[[routing.rules]]
name = "deep remote review of crypto"
priority = 5
intent = "review"
paths = ["src/crypto/**"]
requires_capabilities = { reasoning = true, tools = true }
required_permissions = { permissions = { shell = "deny", network = "ask" } }
cost_limit = "50.0"
model = "anthropic/claude-sonnet"
fallbacks = ["openai/gpt-5.5-thinking"]

The routing rules define which model executes a task. I had to provide simplicity in usage and granularity at the same time.

For that reason, it was decided to classify a routing rule by:

• Intent: what does this task want to do? Write, review, plan, etc.
• File paths involved: which files is this task working on?
• Skills: Which skills are involved in this execution?

At the same time, routing rules allow the user to apply different granularities of restrictions to each rule.

One user may just want to specify the intent, the file paths involved, a cost limit, or whether capabilities are supported by the models.

Understanding user intents is complicated

One of the biggest challenges in deterministic intent detection is understanding user intent.

Since LLMs can't be used in deterministic models, it's important to find a way to infer the user's intentions.

While some systems implement natural-language dictionaries, smista.ai did not take this into account.

The reason is that we don't have the requirements for opting for such an approach:

• smista.ai can be used by anyone and in any language
• Users may use terminology specific to a certain project or company.
• The work surface is too big: there is no enclosed space for possible actions.

For this reason, we have decided to give the users the ability to specify their intents. This is done through the policy configuration as follows:

[[classification.rules]]
intent = "review"
priority = 10
keywords = ["review", "audit", "check", "inspect"]
requires_any_context = ["git_diff", "pull_request"]
 
[[classification.rules]]
intent = "edit"
priority = 20
keywords = ["change", "modify", "refactor", "fix", "implement"]
 
[classification]
default_intent = "chat"

This approach allows the user to define their intents as many as they want, in the language they prefer. Also, the fact that I decided to unify the configuration into a single file lets the user define the intent dictionary while they define their routing policy.

The unglamorous half: config, validation, errors

Providing users with reliable routing policy configuration is not just about defining a good and flexible model, but also implementing a trustworthy validation system.

smista.ai provides a validation system which doesn't just make syntax checks, but also

• rejects unknown providers, models, intents or skills.
• Rejects invalid globs, duplicate rule names, and a missing default route.
• Rejects invalid fallback references; ambiguous rules; invalid permission values
• Rejects secrets stored inline where forbidden.
• Reports to the user conflicting rules.

Invalid configuration always produces an actionable error.

An open frontend

While smista.ai provides a CLI as one of its main products for daily use, both Rust and TypeScript SDKs are being developed.

The two SDKs provide all API types and clients for interacting with the smista-router.

This will allow users to implement their own frontends for smista.ai.

Also, since the TypeScript SDK is generated from the Rust SDK, and the Rust SDK is a direct dependency of smista-cli, full compatibility between the official and the custom implementations is always guaranteed.

What's next

The next step is to implement smista-storage, which is the smista-router library for storing users' sessions.

For this implementation, I have decided to opt for SurrealDB, a young but excellent multi-model database implemented in Rust, which gives smista.ai the option to either embed it or connect to an existing instance.

This will give smista.ai the ability to run on your own network without the hassle of service setup and scale as a SaaS on a clustered infrastructure on the internet.

Stay tuned for the next Dev Update next week. See you soon!