Skip to content

feat(hardware): implement Can I Run It hardware compatibility and performance engine - #25

Merged
spelech merged 7 commits into
mainfrom
feature/canIrunIt
Aug 31, 2026
Merged

feat(hardware): implement Can I Run It hardware compatibility and performance engine#25
spelech merged 7 commits into
mainfrom
feature/canIrunIt

Conversation

@spelech

@spelech spelech commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Summary

This PR introduces the complete Can I Run It hardware compatibility and performance estimation suite into LocalLLMServerManager, inspired by the methodology of willitrunai.com and deeply integrated with live local NVML GPU & System RAM telemetry.

Key Capabilities

  1. Accurate Mathematical Sizing Engine (ICanIRunItService / CanIRunItService):
    • LLMs: Computes exact model weight footprints across GGUF quantization levels (Q2_K to FP16), dynamic KV cache sizing by context window (2K–128K) and precision (FP16/Q8_0/Q4_0), CUDA overhead, and GPU vs. CPU layer offloading distributions with estimated tokens/sec.
    • Diffusion & Image Generation: Computes weights, CLIP/T5 text encoders, VAE decode buffers, and latent resolution scaling (512x512 to 1536x1536) for Flux.1 Dev/Schnell, SDXL, SD 3.5, and SD 1.5.
    • Video Generation: Computes DiT transformer weights, frame context buffers (49–129 frames), and VAE decode memory for Wan 2.2 14B/1.3B, LTX-Video, and HunyuanVideo.
    • Audio & 3D: Sizing and real-time factor estimations for Kokoro TTS, Faster-Whisper, XTTS-v2, MusicGen Melody, TRELLIS V2, and Hunyuan3D-2.
  2. Hardware REST Minimal APIs (Endpoints/HardwareEndpoints.cs):
    • GET /api/hardware/fit: Dynamic query-string based model fit evaluation with live hardware fallback.
    • POST /api/hardware/evaluate: Polymorphic JSON payload evaluator for custom workloads.
  3. Dedicated Interactive UI Tab ([⚡ Can I Run It]):
    • Live hardware telemetry banner (GPU name, VRAM total/free, RAM total/available).
    • Multi-modality switch bar (Text LLMs, Image Gen, Video Gen, Audio & Speech, 3D Gen).
    • Interactive sliders and model presets with real-time reactive recalculation.
    • Proportional stacked memory allocation bar and color-coded fit verdict card.
  4. Ambient Compatibility Badges across Discovery & Library Cards:
    • Quick fit pills rendered on Hugging Face search results, CivitAI search results, and Ollama installed models with click-to-inspect routing into the Can I Run It tab.
  5. Comprehensive Verification:
    • 400 total unit, integration, headless UI, and Playwright WASM browser tests passing.
    • 0 ESLint errors and 0 TypeScript compile errors.

@spelech
spelech merged commit 2cdc233 into main Aug 31, 2026
2 checks passed
@spelech
spelech deleted the feature/canIrunIt branch August 31, 2026 02:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant