nexus: Make the runner extensible #866
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
victorges
changed the title
* docker: Detach comfyui from build logic This conda thing is getting in the way, it should just stay separate. * Migrate to uv!! * docker: Create venv on dockerfiles * [DEV] Use StreamDiffusion version that installs missing pip * streamdiffusion,scope: Make sure we use the same venv in base * docker: Use merged streamdiffusion version * docker/app: Keep extraneous (base) deps with uv --inexact * docker: Migrate base image to uv as well * Tighten dependencies for the least changes * trigger build * trigger build * self human review * Limit python up to 3.11 cause of pillow * Fix tests * commit forgotten lock * Commit updated openapi schemas fastapi version got bumped and generates cleaner schemas * Update documentation with new tools * Fix flash_attn installation * Install uv from docker instead of script script fails all the time * Pin safetensors to 0.7.0 All pipelines install it but uv is downgrading for some reason * Make pyproject deps minimal! Actually fixes a bunch of conflicts. Runtime should not depend on ML stack * Remove torch dependency from noop pipeline * Revert "Remove torch dependency from noop pipeline" This reverts commit 20a9f1d. * Use cpu torch for noop * live-ai -> realtime * Fix vllm version for llm pipelie * Add --frozen on uv run commands
* runner: Abstract app initialization * Create simpler start_app function * Small rename in Lv2v interface * Make router a pipeline property * Create realtime dev extra for local dev * live/pipelines: Allow specifying custom pipeline classes * Document loader a bit better * Cleanup prepare_models logic for custom * Allow specifying custom pipelines for main app * Migrate streamdiffusion to new format * Restore some stuff claude deleted for no good reason * Migrate to explicit params in pipe spec * Fix loading of default params
* docker: Fix comfyui build Gotta copy the ai-runner init as well so setuptools doesnt complain * Claude one-shot migrate Scope to separate project * Fix scope image build * Add missing packages that are in the scope dockerimage
* comfyui: Migrate comfy to its own project as well * Install uv on the comfy image * Some adjustmnts to docker * Add comfystream to pyproject * Pin python version * Pin to the right version 🤦
Avoids another integration path
* runner: Rename base package to runner * Fix CI * Move aiohttp dep to base * uv lock all live projects
This was referenced Dec 18, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It now becomes a base library that can be used by separate "pipeline projects" to create custom
pipeline images. Should make it much easier to create custom pipelines without having to re-implement
the whole ai-runner stack.
Custom pipelines (eg
scope) just need to create a separate python project (even on a separate repo)and depend on the
ai-runnerlibrary. Only need to implement a few pieces:Pipelineclass implementation, to prepare models bundle, receive input frames, send output frames and update paramsParamsclass if you want typed params (could just pass some dicts around)pyproject.toml, amain.pyand aDockerfileto build the custom imageYou can check an example pipeline on
./live/scope/. The idea is to move thatto its own repository so it can be developed separately from the runtime.