Mission Control — Distributed Architecture & Fixes Documentation

This document provides a comprehensive technical overview of all architecture enhancements, bug fixes, microservices, and database migrations implemented across the Mission Control Distributed Gaming Ecosystem.

---

1. System Architecture Diagram

View Source Definition
SYSTEM ARCHITECTURE DIAGRAMMERMAID SVG ENGINE
Generating visual flowchart...

---

2. Complete Chronological Changelog & Key Fixes

Fix 1: Supabase MCP Configuration Clean-Up

• Issue: Unauthorized MCP server definitions in mcp_config.json were throwing authentication popups and permission errors.
• Resolution: Removed conflicting MCP keys from both global C:\Users\DELL\.gemini\config\mcp_config.json and workspace .agents/mcp_config.json, restoring normal CLI and IDE operation.

---

Fix 2: Autonomous Server Watchdog & Self-Healing Agent

• File: Gaming/distributed_server/server_watchdog_agent.py
• Features Added:
  • Standalone monitoring daemon that checks all cluster microservice ports (:8800, :8811, :8821, :8831, :8841, :8851).
  • Added /api/agent/diagnostics endpoint in catalog_service.py for automated health telemetry and self-healing restarts.

---

Fix 3: Differentiated Portrait Box Art vs. Landscape Hero Banners

• Issue: Previously, vertical card covers and wide horizontal hero banners were identical 460x215 header thumbnails.
• Resolution:
  • Cover URL (cover_url): Upgraded to vertical High-Definition Portrait Box Art (library_600x900_2x.jpg).
  • Banner URL (banner_url): Upgraded to wide horizontal Landscape Hero Art (library_hero.jpg).
  • Migrated 1,937+ existing database records to the new HD format.

---

Fix 4: Linux & Steam Deck Platform Support

• Issue: Platform arrays were defaulting strictly to ["Windows"], missing Linux and Steam Deck compatibility.
• Resolution:
  • Updated all schema builders, database models, and harvesters to default to ["Windows", "Linux"] (and ["Windows", "Linux", "Xbox"] for cross-play titles).
  • Migrated all existing rows in canonical_games to include Linux.

---

Fix 5: Quad-Storefront Harvester & Dynamic Store Aggregation

• Integrated Stores:
  1. Valve Steam Store: Akamai CDN assets, top charts, genre categories, and featured promotions.
  2. GOG Galaxy: DRM-free store catalog, bestselling pages, and classic PC games.
  3. Epic Games Store: Free promotional games, exclusives, and store catalog elements.
  4. Xbox & PC Game Pass: Live dynamic web search across Xbox Game Studios, Microsoft Studios, and Bethesda Softworks.

---

Fix 6: Storefront Exclusivity & Multi-Store Detection Engine

• Logic: Implemented in _detect_exclusivity() within game_harvester.py:
  • Epic Exclusive: E.g. Alan Wake 2, Fortnite $\rightarrow$ ['Epic Games']
  • Steam Exclusive: E.g. Half-Life: Alyx, Portal 2 $\rightarrow$ ['Steam']
  • Xbox & PC Game Pass: E.g. Halo Infinite, Forza Horizon 5, Avowed $\rightarrow$ ['Steam', 'Xbox', 'PC Game Pass']
  • Multi-Store PC Releases: E.g. Cyberpunk 2077, The Witcher 3, Control $\rightarrow$ ['Steam', 'Epic Games', 'GOG Galaxy']

---

Fix 7: Dedicated launchers TEXT[] Column & Schema Migration

• Issue: Store availability was previously buried in JSON metadata or unstructured raw tags.
• Resolution:
  • Added dedicated launchers TEXT[] column to canonical_games in Supabase.
  • Updated queries/upsert_game.sql and db.py persistence layer to store and query launchers directly.

---

Fix 8: Automated Release Date Healer

• File: Gaming/distributed_server/heal_release_dates.py
• Resolution:
  • Discovered that initial chart indexing feeds did not populate release dates.
  • Built an automated release date healer that queries official Steam Store APIs and RAWG.
  • Integrated into the background AI Healer microservice (:8831) to auto-fill missing dates continuously.

---

Fix 9: Non-Blocking Sub-100ms Search Optimization

• Issue: Synchronous AI LLM calls during live search queries caused 5–10 second latency and gateway timeouts.
• Resolution:
  • Decoupled synchronous LLM calls from user-facing search endpoints.
  • Live search now uses instant store genre/tag extraction (0 ms response time) and offloads deep LLM enrichment to background workers.

---

Fix 10: 24/7 Multi-Threaded Infinite Crawler & Lifespan Auto-Boot

• Files: Gaming/distributed_server/crawler_service.py & server.py
• Architecture (6 Concurrent Threads & Dual Watchdog):
  1. SteamCrawlerThread: Unlimited page-by-page streaming across all 100,000+ Steam titles.
  2. GOGCrawlerThread: Continuous streaming across all DRM-Free GOG store pages.
  3. EpicCrawlerThread: Continuous sync of Epic Games Store catalog & promotions.
  4. XboxCrawlerThread: Live dynamic web API searches across all Xbox Game Studios publishers.
  5. AIClassifierThread: Continuous multi-provider AI classification cascade (Gemini, NVIDIA NIM, Groq, OpenRouter).
  6. HealerThread: Auto-heals NULL release dates and launcher tags in real time.
  7. Lifespan Auto-Boot: Embedded into server.py lifespan to start all 6 threads automatically on application startup.
  8. SupervisorWatchdog: Probes workers every 5 seconds and auto-revives any thread if a network timeout or store rate limit occurs.

---

Fix 11: End-to-End User Load Balancing & Reverse Proxy Routing

• File: Gaming/distributed_server/load_balancer.py on Port :8800
• Architecture:
  • Acts as the single entry gateway for all user clients (Website & Electron Desktop App).
  • Routes traffic across 5 distinct upstream pools with round-robin balancing and automatic failover.
  • Isolated background crawling (:8851) from user searches (:8811/:8812) to ensure user queries never experience lag or downtime.

---

Fix 12: High-Availability Multi-Tier Database Architecture & Zero-Downtime Local Fallback

• File: Gaming/distributed_server/db.py
• Architecture (3-Tier Auto-Failover Cascade):
  1. Tier 1 (Primary Cloud Postgres): Connects to primary Supabase instance (DATABASE_URL).
  2. Tier 2 (Hot Standby Cloud Postgres): Instant auto-failover to secondary Postgres host (e.g. Neon, Aiven via FALLBACK_DATABASE_URL).
  3. Tier 3 (Embedded SQLite Replica): Zero-dependency local sqlite3 fallback (catalog_fallback.db). If cloud connectivity drops or times out, user searches, catalog queries, and library stats seamlessly execute offline.
• Background Snapshotting:
  • All write and upsert operations asynchronously mirror and snapshot data to the local SQLite replica via background daemon threads, guaranteeing that local fallback caches remain warm.

---

Fix 13: UptimeRobot Keep-Alive & Root Endpoint Welcome Support

• Files: Gaming/distributed_server/server.py & load_balancer.py
• Features Implemented:
  • Registered @app.get("/") and @app.head("/") welcome endpoints so opening root domains displays active service discovery rather than 404 Not Found.
  • Added dual HTTP method support (GET and HEAD) on /health with automatic SELECT 1 SQL keep-alive queries.
  • Keeps Render web dynos warm (preventing the 15-minute idle shutdown) and prevents Supabase's 7-day free tier inactivity pause.

---

Fix 14: Real-Time Node Presence & Differential Library Sync Engine

• Files: Gaming/backend/handlers/game_handler.py & Gaming/distributed_node/node_service.py
• Problem: Opening the desktop app left node status as offline in Supabase, and game changes (installations, title updates, uninstalls) were not reflected in real-time.
• Resolution:
  • Embedded _ensure_local_node_daemon inside game_handler.py to automatically spawn the node service whenever the app is opened or a user logs in.
  • Machine auto-registers in Supabase with status online and sends live 15-second heartbeats.
  • Synchronizes local library changes with Supabase in real-time with automated SQL INSERT, UPDATE, and DELETE queries.

---

Fix 15: Zero-NULL Database Ingestion & Multi-CDN Image Fallbacks

• File: Gaming/backend/system/db_manager.py
• Problem: Scanned games lacking explicit store metadata produced NULL values in the source and local_banner columns in Supabase.
• Resolution:
  • Added intelligent fallback cascades for source (inferred from platform: steam, epic, gog, xbox, local) and local_banner (resolves to Steam CDN headers or curated game artwork).
  • Executed a global database backfill, reducing NULL values across source and local_banner in Supabase to 0.

---

Fix 16: Automated Store Hint Classification for Node Scan Paths

• Files: Gaming/distributed_server/db.py & Gaming/distributed_node/node_service.py
• Problem: When scan paths were registered in Supabase node_scan_paths, the store_hint column remained NULL.
• Resolution:
  • Implemented automated path parsing that maps directory structures to their corresponding digital store (steam, epic, gog, ea, ubisoft, xbox, battlenet, riot, local).
  • Backfilled existing node_scan_paths records in Supabase so no NULL store hints remain.

---

3. Real-Time Production Database Statistics

---

4. Microservice Port Matrix