AI Agent ef64ba8bcf chore: flatten history and absorb submodules into monolith

2026-05-05 17:10:43 +01:00

25 KiB

Raw Blame History

VS Code Extension Performance Migration Guide

From Vite + Node/TypeScript to Svelte (Webview) + Native Binary (Data Engine)

Scope: You are running a VS Code extension with data-intensive and visually intensive workloads. This guide covers migrating the UI layer to Svelte inside a Webview and offloading heavy computation to a native binary (Go/Rust) spawned from the Extension Host.

1. Architecture Overview

VS Code extensions run in a strictly separated dual-process model. Understanding this is critical to placing each technology correctly.

┌─────────────────────────────────────────────────────────────┐
│                      VS Code Extension                      │
│  ┌─────────────────────┐      ┌─────────────────────────┐   │
│  │   Extension Host    │      │        Webview          │   │
│  │   (Node.js / TS)    │◄────►│    (Browser Environment)│   │
│  │                     │ post │    Svelte 5 + Canvas/   │   │
│  │  • File system      │Mess  │    WebGL + Offscreen    │   │
│  │  • VS Code APIs     │      │    Canvas               │   │
│  │  • Spawns binaries  │      │                         │   │
│  └──────────┬──────────┘      └─────────────────────────┘   │
│             │                                               │
│             │ spawn / IPC                                   │
│             ▼                                               │
│  ┌─────────────────────┐                                    │
│  │   Native Binary     │                                    │
│  │   (Go or Rust)      │                                    │
│  │                     │                                    │
│  │  • Heavy parsing    │                                    │
│  │  • Data transforms  │                                    │
│  │  • Multi-threaded   │                                    │
│  │  • Full I/O access  │                                    │
│  └─────────────────────┘                                    │
└─────────────────────────────────────────────────────────────┘

Key Rules

Layer	Technology	Responsibilities
Extension Host	TypeScript (kept)	VS Code API orchestration, spawning native binary, bridging webview messages
Webview	Svelte 5 + Vite	All UI rendering, user interaction, hardware-accelerated graphics (Canvas/WebGL)
Data Engine	Go or Rust binary	Heavy computation, large file parsing, algorithmic work, background processing

2. Phase 1 — Migrate Webview UI to Svelte

2.1 Why Svelte for VS Code Webviews

Compile-time framework: Svelte compiles to imperative vanilla JavaScript. No virtual DOM overhead.
Small bundle size: Critical for webview load times inside VS Code.
Fine-grained reactivity: Svelte 5 Runes update exactly what changed, ideal for dashboards streaming data from the native binary.
Standard Vite integration: You already use Vite; Svelte has first-class Vite support.

2.2 Project Structure

your-extension/
├── src/
│   ├── extension.ts          # Extension Host entry
│   ├── engine/
│   │   └── spawn.ts          # Native binary lifecycle
│   └── webview/
│       ├── main.ts           # Svelte app mount point
│       ├── App.svelte        # Root component
│       ├── lib/
│       │   ├── CanvasRenderer.svelte   # WebGL/Canvas wrapper
│       │   └── MessageBus.ts           # VS Code API wrapper
│       └── vite.config.ts    # Webview build config
├── native/
│   ├── Cargo.toml / go.mod   # Rust or Go project
│   └── src/main.rs / main.go # Native binary source
├── package.json
└── tsconfig.json

2.3 Webview Build Setup (Vite)

Create a separate Vite config for the webview that builds to a single inlined HTML file.

src/webview/vite.config.ts

import { defineConfig } from 'vite';
import { svelte } from '@sveltejs/vite-plugin-svelte';

export default defineConfig({
  plugins: [svelte()],
  build: {
    outDir: '../../out/webview',
    emptyOutDir: true,
    rollupOptions: {
      input: './index.html',
      output: {
        entryFileNames: 'assets/[name].js',
        chunkFileNames: 'assets/[name].js',
        assetFileNames: 'assets/[name].[ext]',
      },
    },
  },
  css: { devSourcemap: true },
});

src/webview/index.html

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Extension UI</title>
</head>
<body>
  <div id="app"></div>
  <script type="module" src="./main.ts"></script>
</body>
</html>

2.4 VS Code Webview Activation (Extension Host)

src/extension.ts

import * as vscode from 'vscode';
import * as path from 'path';

export function activate(context: vscode.ExtensionContext) {
  const disposable = vscode.commands.registerCommand('ext.openPanel', () => {
    const panel = vscode.window.createWebviewPanel(
      'myExtension',
      'Data Visualizer',
      vscode.ViewColumn.One,
      {
        enableScripts: true,
        localResourceRoots: [vscode.Uri.joinPath(context.extensionUri, 'out', 'webview')],
      }
    );

    const webviewUri = panel.webview.asWebviewUri(
      vscode.Uri.joinPath(context.extensionUri, 'out', 'webview', 'index.html')
    );

    panel.webview.html = getWebviewContent(webviewUri.toString());

    // Bridge: Extension Host ↔ Webview
    panel.webview.onDidReceiveMessage(async (message) => {
      if (message.type === 'requestData') {
        const result = await nativeEngine.query(message.payload);
        panel.webview.postMessage({ type: 'dataChunk', payload: result });
      }
    });
  });

  context.subscriptions.push(disposable);
}

function getWebviewContent(uri: string): string {
  // In production, read the built HTML and replace asset paths.
  return `<!DOCTYPE html>
<html>
<head>
  <base href="${uri}/">
  <script type="module" crossorigin src="${uri}/assets/main.js"></script>
  <link rel="stylesheet" href="${uri}/assets/main.css">
</head>
<body>
  <div id="app"></div>
</body>
</html>`;
}

2.5 Svelte Webview Entry Point

src/webview/main.ts

import { mount } from 'svelte';
import App from './App.svelte';
import { vscode } from './lib/MessageBus';

// Acquire VS Code API (only once)
const vscodeApi = acquireVsCodeApi();

mount(App, {
  target: document.getElementById('app')!,
  props: { vscodeApi }
});

src/webview/lib/MessageBus.ts

export const vscode = {
  postMessage: (msg: unknown) => {
    if (typeof acquireVsCodeApi === 'function') {
      acquireVsCodeApi().postMessage(msg);
    }
  },
  onMessage: (handler: (msg: any) => void) => {
    window.addEventListener('message', (event) => handler(event.data));
  }
};

2.6 Handling Visually Intensive Graphics

For heavy rendering, do not rely on DOM-based charting libraries alone. Use a dedicated renderer component inside Svelte.

src/webview/lib/CanvasRenderer.svelte

<script lang="ts">
  import { onMount } from 'svelte';

  let canvas: HTMLCanvasElement;
  let ctx: CanvasRenderingContext2D | WebGL2RenderingContext;

  interface Props {
    data: Float32Array;
    mode: '2d' | 'webgl';
  }

  let { data, mode }: Props = $props();

  onMount(() => {
    if (mode === 'webgl') {
      ctx = canvas.getContext('webgl2', { antialias: false })!;
      initWebGL(ctx as WebGL2RenderingContext);
    } else {
      ctx = canvas.getContext('2d', { alpha: false })!;
    }
  });

  // Reactive: re-render when data changes
  $effect(() => {
    if (!ctx) return;
    if (mode === '2d') {
      render2D(ctx as CanvasRenderingContext2D, data);
    } else {
      renderWebGL(ctx as WebGL2RenderingContext, data);
    }
  });

  function render2D(c: CanvasRenderingContext2D, d: Float32Array) {
    c.clearRect(0, 0, canvas.width, canvas.height);
    // ... optimized drawing
  }

  function initWebGL(gl: WebGL2RenderingContext) {
    // Compile shaders, create buffers
  }

  function renderWebGL(gl: WebGL2RenderingContext, d: Float32Array) {
    // Upload buffer, drawArrays/drawElements
  }
</script>

<canvas bind:this={canvas} width={800} height={600}></canvas>

Best Practice: For maximum performance, use OffscreenCanvas transferred to a Web Worker so rendering never blocks Svelte's reactive updates or user input.

3. Phase 2 — Data Processing via Native Binary Spawn

3.1 Why Native Binary Spawn Wins

Approach	Pros	Cons
Native Binary Spawn	Full multithreading, direct file I/O, any language, no sandbox limits, crashes don't kill extension host	Requires packaging per platform, async only
WASM in Webview	Near-native speed, no external binary	Single-threaded (mostly), no file system, large Rust/Go WASM output
Node Native Addon (NAPI)	Synchronous calls, zero IPC overhead	Build nightmare, VS Code host compatibility issues, harder to debug
Pure TypeScript	Simple, no build changes	Event loop blocking, memory limits, slow for parsing/processing

Verdict: Spawn a long-lived native binary (Go or Rust) and communicate over stdin/stdout JSON lines or a local gRPC/HTTP socket.

3.2 Choosing Go vs Rust

Factor	Go	Rust
Development Speed	Faster to write, great stdlib	Slower, steep learning curve
Binary Size	Larger (~2-5MB minimum)	Small (~1MB or less with strip+LTO)
Startup Time	Slower (GC + runtime init)	Instant
Memory Control	GC (acceptable for most cases)	Zero-cost, predictable
Concurrency	Goroutines (excellent)	Async/await + threads (excellent)
JSON Performance	Good	Excellent (simdjson-style parsers)
Cross-Compilation	Easy	Easy with `cross` or `cargo-zigbuild`

Recommendation:

Choose Go if you need rapid iteration, networking, or your team already knows it.
Choose Rust if you need maximum throughput, lowest latency, smallest binary, or memory-intensive workloads.

3.3 Communication Protocol: JSON Lines over Stdio

The simplest robust protocol. The extension host spawns the binary and keeps it alive as a daemon.

Protocol Rules

Extension sends one JSON object per line (\n terminated).
Binary responds with one JSON object per line.
Use id for request/response correlation.
Binary can emit unsolicited progress or log messages.

Example Message Flow

// Host -> Binary
{"id": 1, "method": "parseDataset", "params": {"path": "/tmp/large.csv", "columns": ["x", "y", "z"]}}

// Binary -> Host (progress)
{"id": 1, "type": "progress", "percent": 45}

// Binary -> Host (result)
{"id": 1, "type": "result", "data": {"rows": 5000000, "summary": {...}}}

3.4 Extension Host: Binary Lifecycle Manager

src/engine/spawn.ts

import { spawn, ChildProcessWithoutNullStreams } from 'child_process';
import * as path from 'path';
import * as vscode from 'vscode';
import { EventEmitter } from 'events';

interface Request {
  id: number;
  resolve: (value: any) => void;
  reject: (reason?: any) => void;
}

export class NativeEngine {
  private proc: ChildProcessWithoutNullStreams;
  private requests = new Map<number, Request>();
  private idCounter = 0;
  private buffer = '';
  public events = new EventEmitter();

  constructor(binaryPath: string) {
    this.proc = spawn(binaryPath, ['--stdio'], {
      stdio: ['pipe', 'pipe', 'pipe'],
    });

    this.proc.stdout.on('data', (chunk: Buffer) => {
      this.buffer += chunk.toString('utf-8');
      let lineEnd: number;
      while ((lineEnd = this.buffer.indexOf('\n')) !== -1) {
        const line = this.buffer.slice(0, lineEnd);
        this.buffer = this.buffer.slice(lineEnd + 1);
        this.handleLine(line);
      }
    });

    this.proc.stderr.on('data', (chunk) => {
      console.error('[NativeEngine]', chunk.toString());
    });

    this.proc.on('exit', (code) => {
      vscode.window.showErrorMessage(`Native engine exited with code ${code}`);
    });
  }

  private handleLine(line: string) {
    try {
      const msg = JSON.parse(line);
      if (msg.type === 'progress' || msg.type === 'log') {
        this.events.emit(msg.type, msg);
        return;
      }
      const req = this.requests.get(msg.id);
      if (req) {
        this.requests.delete(msg.id);
        if (msg.error) req.reject(msg.error);
        else req.resolve(msg.data ?? msg);
      }
    } catch (e) {
      console.error('Invalid JSON from native engine:', line);
    }
  }

  send<T = any>(method: string, params?: unknown): Promise<T> {
    return new Promise((resolve, reject) => {
      const id = ++this.idCounter;
      this.requests.set(id, { id, resolve, reject });
      const payload = JSON.stringify({ id, method, params }) + '\n';
      this.proc.stdin.write(payload);
    });
  }

  dispose() {
    this.proc.kill();
  }
}

3.5 Native Engine Example: Go

native/main.go

package main

import (
	"bufio"
	"encoding/json"
	"fmt"
	"os"
)

type Request struct {
	ID     int             `json:"id"`
	Method string          `json:"method"`
	Params json.RawMessage `json:"params"`
}

type Response struct {
	ID   int    `json:"id"`
	Data any    `json:"data,omitempty"`
	Error string `json:"error,omitempty"`
}

func main() {
	scanner := bufio.NewScanner(os.Stdin)
	writer := bufio.NewWriter(os.Stdout)

	for scanner.Scan() {
		var req Request
		if err := json.Unmarshal(scanner.Bytes(), &req); err != nil {
			continue
		}

		var resp Response
		resp.ID = req.ID

		switch req.Method {
		case "parseDataset":
			var p ParseParams
			json.Unmarshal(req.Params, &p)
			result, err := parseDataset(p)
			if err != nil {
				resp.Error = err.Error()
			} else {
				resp.Data = result
			}
		default:
			resp.Error = "unknown method"
		}

		out, _ := json.Marshal(resp)
		writer.Write(out)
		writer.WriteByte('\n')
		writer.Flush()
	}
}

type ParseParams struct {
	Path    string   `json:"path"`
	Columns []string `json:"columns"`
}

func parseDataset(p ParseParams) (any, error) {
	// Heavy I/O + CPU work here
	// Use goroutines for parallel processing
	return map[string]any{"rows": 5000000}, nil
}

Build & Package

# Build for host platform
cd native
go build -o ../bin/engine

# Cross-compile for all VS Code targets
goos_list=("darwin" "linux" "windows")
goarch_list=("amd64" "arm64")

for goos in "${goos_list[@]}"; do
  for goarch in "${goarch_list[@]}"; do
    output="../bin/engine-${goos}-${goarch}"
    if [ "$goos" = "windows" ]; then output="${output}.exe"; fi
    GOOS=$goos GOARCH=$goarch go build -ldflags="-s -w" -o "$output"
  done
done

3.6 Native Engine Example: Rust

native/Cargo.toml

[package]
name = "engine"
version = "0.1.0"
edition = "2021"

[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
rayon = "1.8"  # Data parallelism

native/src/main.rs

use serde::{Deserialize, Serialize};
use std::io::{self, BufRead, Write};

#[derive(Debug, Deserialize)]
struct Request {
    id: u64,
    method: String,
    #[serde(default)]
    params: serde_json::Value,
}

#[derive(Debug, Serialize)]
struct Response {
    id: u64,
    #[serde(skip_serializing_if = "Option::is_none")]
    data: Option<serde_json::Value>,
    #[serde(skip_serializing_if = "Option::is_none")]
    error: Option<String>,
}

fn main() {
    let stdin = io::stdin();
    let mut stdout = io::stdout();

    for line in stdin.lock().lines() {
        let Ok(line) = line else { continue };
        let Ok(req) = serde_json::from_str::<Request>(&line) else { continue };

        let mut resp = Response { id: req.id, data: None, error: None };

        match req.method.as_str() {
            "parseDataset" => {
                match parse_dataset(&req.params) {
                    Ok(data) => resp.data = Some(data),
                    Err(e) => resp.error = Some(e.to_string()),
                }
            }
            _ => resp.error = Some("unknown method".into()),
        }

        let out = serde_json::to_string(&resp).unwrap();
        writeln!(stdout, "{}", out).unwrap();
        stdout.flush().unwrap();
    }
}

fn parse_dataset(params: &serde_json::Value) -> Result<serde_json::Value, Box<dyn std::error::Error>> {
    // Use rayon::join or par_iter for heavy work
    Ok(serde_json::json!({ "rows": 5_000_000 }))
}

Build & Package

cd native
cargo build --release
# Cross-compile
cargo install cross
cross build --release --target x86_64-pc-windows-gnu
cross build --release --target x86_64-unknown-linux-gnu
cross build --release --target aarch64-apple-darwin

4. Phase 3 — Integration & Wiring

4.1 Extension Host as the Router

The Extension Host is the only process that can talk to both the Webview and the Native Binary. It must route messages efficiently.

Webview (Svelte)          Extension Host (TS)          Native Binary (Go/Rust)
      |                           |                             |
      |── requestData ───────────►|                             |
      |                           |── parseDataset ─────────────►|
      |                           |◄─ progress / result ──────────|
      |◄─ dataChunk ──────────────|                             |

src/extension.ts (Integration)

import { NativeEngine } from './engine/spawn';
import * as path from 'path';
import * as os from 'os';

let engine: NativeEngine;

export function activate(context: vscode.ExtensionContext) {
  // Resolve platform-specific binary
  const platform = os.platform();
  const arch = os.arch();
  const binName = platform === 'win32' ? 'engine.exe' : 'engine';
  const binPath = path.join(context.extensionPath, 'bin', `${binName}-${platform}-${arch}`);

  engine = new NativeEngine(binPath);

  // Forward progress to webview
  engine.events.on('progress', (msg) => {
    panel.webview.postMessage({ type: 'engineProgress', payload: msg });
  });

  // Handle webview requests
  panel.webview.onDidReceiveMessage(async (msg) => {
    if (msg.type === 'queryEngine') {
      const result = await engine.send(msg.method, msg.params);
      panel.webview.postMessage({ type: 'queryResult', id: msg.id, payload: result });
    }
  });
}

export function deactivate() {
  engine?.dispose();
}

4.2 Svelte: Reactive Data from Engine

src/webview/App.svelte

<script lang="ts">
  import { vscode } from './lib/MessageBus';
  import CanvasRenderer from './lib/CanvasRenderer.svelte';

  let progress = $state(0);
  let dataset = $state<Float32Array | null>(null);

  vscode.onMessage((msg) => {
    if (msg.type === 'engineProgress') {
      progress = msg.payload.percent;
    }
    if (msg.type === 'queryResult') {
      // Convert result to typed array for renderer
      dataset = new Float32Array(msg.payload.data);
    }
  });

  function loadData() {
    vscode.postMessage({
      type: 'queryEngine',
      id: crypto.randomUUID(),
      method: 'parseDataset',
      params: { path: '/tmp/data.csv', columns: ['x', 'y'] }
    });
  }
</script>

<main>
  <button onclick={loadData}>Load Dataset</button>
  {#if progress > 0 && progress < 100}
    <progress value={progress} max={100}></progress>
  {/if}
  {#if dataset}
    <CanvasRenderer {dataset} mode="webgl" />
  {/if}
</main>

5. Phase 4 — Build & Development Workflow

5.1 VS Code Extension Build

Use vsce to package. Ensure your package.json includes the native binaries.

package.json

{
  "name": "your-extension",
  "version": "0.1.0",
  "main": "./out/extension.js",
  "contributes": {
    "commands": [{ "command": "ext.openPanel", "title": "Open Visualizer" }]
  },
  "scripts": {
    "build:webview": "vite build -c src/webview/vite.config.ts",
    "build:extension": "tsc -p ./",
    "build:native": "cd native && go build -o ../bin/engine",
    "build": "npm run build:webview && npm run build:extension && npm run build:native",
    "package": "vsce package"
  },
  "devDependencies": {
    "@sveltejs/vite-plugin-svelte": "^5.0.0",
    "@types/vscode": "^1.90.0",
    "svelte": "^5.0.0",
    "typescript": "^5.6.0",
    "vite": "^6.0.0",
    "vsce": "^2.15.0"
  }
}

5.2 Debugging Setup

.vscode/launch.json

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Run Extension",
      "type": "extensionHost",
      "request": "launch",
      "args": ["--extensionDevelopmentPath=${workspaceFolder}"],
      "outFiles": ["${workspaceFolder}/out/**/*.js"],
      "preLaunchTask": "npm: build"
    },
    {
      "name": "Attach to Native (Go)",
      "type": "go",
      "request": "attach",
      "mode": "local",
      "processId": "${command:pickProcess}"
    },
    {
      "name": "Attach to Native (Rust)",
      "type": "lldb",
      "request": "attach",
      "program": "${workspaceFolder}/bin/engine",
      "pid": "${command:pickProcess}"
    }
  ]
}

Webview Debugging:

Open the Command Palette → Developer: Open Webview Developer Tools
This opens Chrome DevTools for the webview. You can debug Svelte components, profile WebGL, and inspect OffscreenCanvas workers.

6. Migration Strategy & Transition Options

Option A: Incremental Migration (Recommended)

Migrate piece by piece without breaking existing functionality.

Step	Action	Risk
1	Scaffold new Svelte webview alongside existing webview	Low
2	Port one visual panel to Svelte + Canvas/WebGL	Low
3	Extract one heavy data function to native binary	Medium — test parity
4	Switch communication from webview direct TS to Extension Host routing	Medium
5	Deprecate old webview, remove legacy code	Low

Option B: Full Rewrite

Only recommended if the current codebase is small (< 5k lines) or deeply coupled.

Pro: Clean architecture from day one.
Con: Longer time to ship, higher bug risk.

Option C: Hybrid Staged

Keep the existing TypeScript data engine running, but wrap the heaviest function in a native binary. This proves the architecture before full commitment.

// Fallback strategy
async function heavyCompute(data: any) {
  if (nativeEngine.isReady()) {
    return nativeEngine.send('heavyCompute', data);
  }
  // Fallback to legacy TS implementation
  return legacyHeavyCompute(data);
}

7. Performance Checklist

Webview (Svelte + Graphics)

Use Svelte 5 Runes ($state, $derived, $effect) instead of legacy stores where possible.
Never bind massive arrays directly to DOM elements. Pass them to Canvas/WebGL.
Use requestAnimationFrame for all rendering loops; throttle data updates to 60fps.
Move Canvas rendering to a Web Worker with OffscreenCanvas if the main thread drops frames.
Profile with Webview DevTools → Performance tab to confirm GPU compositing.

Native Binary

Keep the binary alive (daemon mode); do not spawn per request.
Use buffered I/O (bufio in Go, BufWriter in Rust) to avoid syscall overhead.
Stream large results in chunks rather than one giant JSON payload.
Use goroutines (Go) or rayon (Rust) for parallel data processing.
Strip symbols and use LTO for smaller binaries: go build -ldflags="-s -w" or cargo build --release with strip = true.

Extension Host

Never await a long native binary operation without yielding; use streaming responses.
Dispose the native binary process on deactivate() to avoid zombie processes.
Validate binary existence on activation and show a user-friendly error if the platform binary is missing.

8. Summary

Concern	Your Stack	Rationale
UI Framework	Svelte 5	Compile-time reactivity, smallest overhead, perfect for VS Code webviews
Graphics	WebGL / Canvas API inside Svelte	Hardware-accelerated, bypasses DOM limits
Data Engine	Go or Rust binary spawned via stdio	True multithreading, full I/O, no sandbox
Build Tool	Vite	Fast HMR, native Svelte plugin, easy webview bundling
Communication	JSON Lines over stdio	Simple, debuggable, language-agnostic

This architecture separates concerns cleanly: Svelte manages reactive UI state, WebGL renders pixels, and a native binary crunches data without blocking either the Extension Host or the Webview.

25 KiB Raw Blame History