DocsHome

Mission Control Architecture & API Reference

DOCUMENTATION PORTAL

Complete technical reference, API integration guides, system telemetry setup, and NVIDIA NIM AI models for production deployments.

CTRL + K

OVERVIEW

2 articles

Overview

Overview17 min read

Project Summary

The Mission Control Gaming Assistant is a high-performance, real-time agentic system that enhances gaming through advanced computer vision, NVIDIA-accelerated r...

READ DOCS

Overview82 min read

Recent Updates & Major Releases

Curated breakdown of major version releases, architectural milestones, and core engine enhancements in Mission Control.

READ DOCS

ARCHITECTURE

5 articles

Architecture

Architecture10 min read

Distributed Architecture & Fixes

This document provides a comprehensive technical overview of all architecture enhancements, bug fixes, microservices, and database migrations implemented across...

READ DOCS

Architecture9 min read

Major Upgrades & Infrastructure

This document summarizes the major architectural transformations, resilience engineering, cloud deployments, multi-tier database failovers, 24/7 infinite crawle...

READ DOCS

Architecture6 min read

Distributed Library & Load Balancer

Mission Control extends its capabilities with a distributed, load-balanced game-library architecture that decouples heavy web scraping, game launcher crawling, ...

READ DOCS

Architecture8 min read

System Architecture

This document details the production architecture of Mission Control, mapping out the event-driven Electron/React frontend, the concurrent multi-threaded Python...

READ DOCS

Architecture13 min read

Process & Threading

This document outlines the step-by-step process for releasing a new version of the AI Gaming Assistant. The entire pipeline is automated using PowerShell and Py...

READ DOCS

CORE LOGIC

4 articles

Core Logic

Core Logic3 min read

Agentic AI Controller

This document outlines the operational logic for the Agentic AI Mode, detailing how the AI translates user intent and visual data into system actions.

READ DOCS

Core Logic4 min read

AI Personalities

We have successfully implemented and verified the Enriched Agentic AI Mode & Voice Co-Pilot functionality. This upgrade provides a highly contextual, voice-assi...

READ DOCS

Core Logic2 min read

Controller & Gamepad Input Mapping

Complete guide to Mission Control's interactive Native XInput & DirectInput Gamepad Configurator, custom action combos, analog stick deadzone tuning, and force feedback haptics.

READ DOCS

Core Logic6 min read

Open Knowledge Format (OKF)

The Open Knowledge Format OKF is an open, vendor-neutral specification published by Google Cloud to formalize the "LLM-wiki" knowledge pattern. It standardizes ...

READ DOCS

INTEGRATIONS

3 articles

Integrations

Integrations8 min read

NVIDIA NIM Guide

← Back to Main Project Guide./README.md

READ DOCS

Integrations2 min read

NVIDIA Integration

This document outlines the specific NVIDIA hardware and software technologies utilized by the Mission Control Gaming Assistant.

READ DOCS

Integrations5 min read

The Evolution of NVIDIA DLSS

A comprehensive technical breakdown of NVIDIA Deep Learning Super Sampling evolution — from early DLSS 1 AI Super Sampling to DLSS 5 Neural Rendering.

READ DOCS

AI MODELS

2 articles

AI Models

AI Models2 min read

On-Demand AI Model Weights

Comprehensive guide to Mission Control's on-demand neural weights: YOLOv8n, YOLOv8s, and Whisper-Tiny.

READ DOCS

AI Models2 min read

Vision On-Demand AI & Perception Models

Architectural overview of Mission Control's AI Vision pipeline, YOLOv8 object detection, Whisper-Tiny voice recognition, and direct in-app model weight downloader.

READ DOCS

PERFORMANCE

1 articles

Performance

Performance3 min read

FPS & VRAM Optimization

Accurate DirectX frame timing, pybind11 C++ bindings, and CUDA/TensorRT timing integration for high-performance HUD overlay.

READ DOCS

ROADMAPS

2 articles

Roadmaps

Roadmaps11 min read

Product Roadmap

Mission Control is positioned to be the ultimate AI-Powered Gaming Copilot for PC gamers. To achieve this vision and provide a premium experience, the applicati...

READ DOCS

Roadmaps5 min read

Electron App Roadmap

We have systematically integrated the items from this roadmap into the Frontend, Preload Bridge, and Main process of Mission Control. Below is the detailed alig...

READ DOCS

REFERENCE

2 articles

Reference

Reference21 min read

Aero AI Prompt

> Paste this entire prompt into Claude Code, Cursor, or any AI coding assistant to implement all improvements and new features for the Mission Control desktop a...

READ DOCS

Reference92 min read

Patches & Version History

This document contains a detailed history of all patches and updates for the AI Gaming Assistant.

READ DOCS

ENVIRONMENT CONFIGURATION

API Keys & Environment Reference

.env.local / settings.yaml

UTF-8 Encoded

Cloud AI inference via NVIDIA NIM (Llama 3.1/3.2, VLM models)

Requiredbuild.nvidia.com

Game DB: ratings, genres, Metacritic, DLC — 20k req/month free

Optionalrawg.io/apidocs

AI-synthesized web search enrichment — 1k req/month free

Optionalapp.tavily.com

Premium neural TTS voice synthesis (ElevenLabs)

Optionalelevenlabs.io

# Cloud AI Inference (NVIDIA NIM Llama 3.1 / 3.2)
NVIDIA_API_KEY=nvapi-xxxxxxxxxxxxxxxxxxxxxxxxx

# Game Knowledge Graph & Search Enrichment
RAWG_API_KEY=your_rawg_key
TAVILY_API_KEY=tvly-xxxxxxxxxxxx

SYSTEM SPECIFICATIONS

Architecture & Tech Stack Matrix

Architecture Subsystem	Engine & Technology Stack	Status
AI Computer Vision	Pure TensorRT 10.x (YOLOv8 Engine)	Active
Frame Screen Capture	dxcam (DXGI 120fps+) / D3DShot / MSS fallback	Active
Cloud LLM / Vision	NVIDIA NIM — Llama 3.1 8B / 3.2 11B Vision	Active
OCR Text Extraction	RapidOCR / Tesseract (GPU/ONNX-accelerated)	Active
Web Search Enrichment	Wikipedia + SteamSpy + DuckDuckGo + RAWG.io	Active
Voice STT Processing	Google Cloud Speech / Sphinx (offline fallback)	Active
Voice Speech Synthesis	ElevenLabs / Google Cloud TTS / SAPI5	Active
UI & Overlay Engine	Electron + React (HUD) + PyQt6 (Desktop)	Active
Telemetry & Hardware	pynvml (NVML) + DirectX C++ FPS Engine	Active
Hotkey Intercept	pynput GlobalHotKeys + Win32 GetAsyncKeyState	Active
Configuration	PyYAML (settings.yaml) + dotenv (.env)	Active
Security & Binding	Motherboard UUID binding + AES-256 E2EE	Active

Open Source Ecosystem

Something missing in documentation?

Mission Control is completely open source. Contribute new documentation, propose features, or inspect the codebase on GitHub.

View Repository on GitHub