Dev Update #5 - From the router API to an interactive CLI

What was milestone 5 about

At the end of the previous dev update, we had completed the most important component of smista.ai: the router.

The router could already classify tasks, select models, orchestrate runs, mediate tools, store sessions and expose all of this through its REST API.

But an API alone doesn't make a product pleasant to use.

Milestone 5 was about implementing the clients for that API: the smista-router-client crate, the Rust SDK facade and the TypeScript SDK. Once that milestone was completed, we immediately started working on milestone 6, which is the actual smista CLI.

In the last two weeks, we have completed the entire router client layer and most of the CLI. smista.ai is finally starting to look and behave like the developer tool we designed at the beginning of the project.

One interface for the entire router API

The smista-router-client crate provides a typed Rust interface for every endpoint exposed by smista-router.

This includes:

• router status and authentication;
• session creation, listing, retrieval, update and deletion;
• task execution and continuation;
• buffered and streamed responses;
• routing previews;
• traces and usage;
• provider and model catalogues.

The important design choice was to keep the interface independent of the HTTP library used underneath.

pub trait Client: Send + Sync + 'static {
    fn status(
        &self,
    ) -> impl Future<Output = Result<StatusResponse>> + Send;
 
    fn execute(
        &self,
        id: Uuid,
        req: ExecuteRequest,
    ) -> impl Future<Output = Result<TurnResponse>> + Send;
 
    fn stream_execute(
        &self,
        id: Uuid,
        req: ExecuteRequest,
    ) -> impl Future<
        Output = Result<BoxStream<'static, Result<TurnEvent>>>,
    > + Send;
}

Frontends depend on this trait, not on reqwest, ureq or any other HTTP client. This means that the communication protocol remains the same no matter which runtime or application architecture the client uses.

The client also owns its authentication state. The user doesn't need to pass a session token to every call. bootstrap stores the generated API key, sign_in exchanges it for a session token, and every authenticated request automatically uses it.

Provider credentials work in a similar way. They are configured once on the client and are attached only to requests which may contact an LLM provider. They never become part of the prompt, a query parameter or an error message.

Supporting different Rust runtimes

Choosing a common interface also allowed us to provide three different Rust implementations.

The first one is based on reqwest. It is the natural choice for applications already running on Tokio, including the official CLI.

The second one is based on ureq. It performs blocking requests internally and doesn't need an async runtime. Its methods still implement the same async trait, so a program can drive them with even a minimal executor.

Finally, we added an isahc implementation. It is fully asynchronous, but it doesn't require the application to run a Tokio reactor because the HTTP work is driven by its own agent thread.

Why support three HTTP backends for the same API?

Because an SDK should adapt to the application embedding it, and not force the application to change its runtime just to interact with smista.ai.

All implementations support the same features, including server-sent events. A streamed turn yields text deltas, reasoning, tool calls, usage information and the final turn_end event through the same Rust types used by the router.

Rust and TypeScript SDKs are now complete

The Rust SDK now re-exports both the router client and the shared types from smista-core.

This means that a Rust application only needs the smista-sdk crate to define routing policies, inspect models, create sessions and execute tasks.

use smista_sdk::client::{Client, ReqwestClient, RouterClientConfig};
 
let client = ReqwestClient::new(RouterClientConfig::default())?;
client.sign_in().await?;
 
let sessions = client.list_sessions(None, None).await?;

We have also completed the TypeScript SDK. SmistaClient uses the native fetch API and implements every router endpoint, including streamed execution through async iterators.

const client = new SmistaClient({
  baseUrl: 'http://localhost:7331',
  sessionToken,
});
 
for await (const event of await client.streamExecute(sessionId, request)) {
  if (event.type === 'text_delta') {
    process.stdout.write(event.delta);
  }
}

The request and response types are not maintained by hand. They are generated from the same Rust types used by the router and checked against the committed OpenAPI schema. So, once again, there is a single source of truth for the whole communication protocol.

This completes milestone 5 and makes smista.ai an open platform. The official CLI is one possible frontend, but anyone can now build another one without reimplementing routing or depending on internal router components.

Making the CLI local-first without making setup annoying

With the SDKs completed, we started working on the component users will actually interact with every day: smista-cli.

The first part of this milestone was not the TUI. Before rendering a single prompt, we had to make the CLI capable of setting up and managing a local smista.ai installation.

The CLI can now:

• create, inspect, edit and validate configuration files;
• manage global and project-specific provider credentials;
• bootstrap a router user and manage its API key;
• start and stop the local router;
• check the router status;
• automatically start the router when the configured URL is local.

The last point is particularly important for the local-first experience. If the router is already available, the CLI simply connects to it. If it is not running, and the URL points to localhost, the CLI can start it and wait until it becomes healthy.

The same behaviour is deliberately not applied to remote URLs. The CLI will never attempt to start something because a remote router is unreachable.

So the normal usage doesn't require the user to think about two separate processes. There is still one binary, and the local router is an implementation detail unless the user explicitly wants to manage it.

Credentials don't belong in configuration files

Another important part of the CLI work was credential storage.

smista.ai needs to manage different kinds of secrets: the router API key, provider API keys and the local keys used for end-to-end encrypted sessions. Putting all of them directly into config.toml would be both unsafe and inconvenient.

For that reason, the CLI first tries to use the operating system keyring. When the keyring is unavailable, for example on some headless systems, it falls back to a file-backed secret store with restricted permissions. The user can also enforce keyring usage and make startup fail rather than allowing the fallback.

Credentials can be defined globally or for a single project. Project credentials take precedence, which makes it possible to use different provider accounts in different workspaces without changing the global setup.

We also implemented the client side of session encryption. Each encrypted session gets its own XChaCha20-Poly1305 key, which is generated and stored locally by the CLI. The router only receives the public key identifier and the encrypted payload. It never receives or persists the secret key.

This is the same end-to-end encryption model described in the previous update, but it is now backed by the actual client implementation which owns the keys.

The interactive CLI is now alive

Once configuration, credentials and router lifecycle were in place, we could finally build the interactive interface with ratatui.

The CLI is split into independent workers. One reads terminal input, one talks to the router, and the TUI owns the visible state. Messages are passed between them, so streamed model output never blocks keyboard input and a slow network request never freezes the terminal.

The execution flow is already connected end to end. The CLI creates an encrypted session, sends the prompt to the router, renders streamed text and reasoning events, executes allowed local tools, asks for approval when needed, and continues the run with the result.

Approvals can apply once or be remembered for the current session. When the user chooses to always allow a command, the CLI stores the normalised command alias rather than blindly approving every future shell request.

Interruptions also use the continuation protocol implemented by the router. The input listener remains active while a turn is running, so the user can stop the current path or inject new input without throwing away the session state.

Sessions are scoped to the workspace

While implementing session resume, we noticed that a flat list of every session is not useful in a developer tool.

For that reason, sessions now have an opaque scope. The CLI uses the current working directory as its scope, while another frontend could use a repository ID, a workspace name or any other identifier.

The router can filter sessions by exact scope and by a case-insensitive title search. In the CLI, /resume therefore shows the sessions belonging to the current workspace instead of mixing unrelated projects together.

This field stays in clear text even when the session content is encrypted. The router must be able to filter the list, but it still doesn't need to know anything about the messages stored in the session.

The first slash commands

The interactive CLI already supports the commands needed to inspect and control the most important state:

• /model lists available models and lets the user set an explicit preference;
• /providers shows configured local and remote providers;
• /resume restores a session from the current workspace;
• /skills lists the skills discovered by the CLI;
• /status shows the current router status;
• /preview displays the routing decision without calling an LLM;
• /quit, /q and /exit close the application.

The model selector also has an auto option. Selecting it clears the explicit preference and returns control to the deterministic router. This distinction is important: the CLI can express a deliberate override, but it never performs model selection by itself.

Prompts can now reference files by typing @. The TUI completes paths as the user types and attaches existing files to the request while keeping the original reference visible in the prompt.

There is also prompt history navigation, proper input states for lists and approvals, and routing previews which show the detected intent, selected model, matched rule, context, estimated cost and required permissions.

For the first time, the deterministic decisions implemented in the router are visible and usable from a real interactive client.

What's next

Milestone 5 is completed, and most of milestone 6 is now done as well.

The next step is to finish the remaining CLI views and commands, especially the parts around usage, traces, plan and diff review, and then spend time polishing the interaction between streaming, approvals and session resume.

After that, milestone 7 will focus on distribution: producing binaries, installers and release workflows so that smista.ai can finally be used without building it from source.

The router was the brain of smista.ai. The clients gave it a stable interface. Now the CLI is turning it into a tool that can actually become part of a daily developer workflow.