ModelDir returns the directory where models are stored.

NewManager creates a new runtime manager.

NewOllamaClient creates a new Ollama runtime client.a.

NewPreloader creates a new Preloader.

NewTritonClient creates a new Triton runtime client.

NewUpdater creates a new Updater.

NewVLLMClient creates a new VLLM runtime client.

PreferredModelFormat returns the preferred model format.

ErrRequestCanceled is returned when the request is canceled.

Manager manages runtimes.

Preloader preloads models.

Updater updates runtimes at startup.

Client is the interface for managing runtimes.

ClientFactory is the interface for creating a new Client given a model ID.