replica-omnisciente/vscode_extension_migration_guide.md

804 lines
25 KiB
Markdown

# VS Code Extension Performance Migration Guide
## From Vite + Node/TypeScript to Svelte (Webview) + Native Binary (Data Engine)
> **Scope:** You are running a VS Code extension with data-intensive and visually intensive workloads. This guide covers migrating the UI layer to **Svelte** inside a Webview and offloading heavy computation to a **native binary** (Go/Rust) spawned from the Extension Host.
---
## 1. Architecture Overview
VS Code extensions run in a strictly separated dual-process model. Understanding this is critical to placing each technology correctly.
```
┌─────────────────────────────────────────────────────────────┐
│ VS Code Extension │
│ ┌─────────────────────┐ ┌─────────────────────────┐ │
│ │ Extension Host │ │ Webview │ │
│ │ (Node.js / TS) │◄────►│ (Browser Environment)│ │
│ │ │ post │ Svelte 5 + Canvas/ │ │
│ │ • File system │Mess │ WebGL + Offscreen │ │
│ │ • VS Code APIs │ │ Canvas │ │
│ │ • Spawns binaries │ │ │ │
│ └──────────┬──────────┘ └─────────────────────────┘ │
│ │ │
│ │ spawn / IPC │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ Native Binary │ │
│ │ (Go or Rust) │ │
│ │ │ │
│ │ • Heavy parsing │ │
│ │ • Data transforms │ │
│ │ • Multi-threaded │ │
│ │ • Full I/O access │ │
│ └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
### Key Rules
| Layer | Technology | Responsibilities |
|-------|------------|------------------|
| **Extension Host** | TypeScript (kept) | VS Code API orchestration, spawning native binary, bridging webview messages |
| **Webview** | **Svelte 5** + Vite | All UI rendering, user interaction, hardware-accelerated graphics (Canvas/WebGL) |
| **Data Engine** | **Go or Rust binary** | Heavy computation, large file parsing, algorithmic work, background processing |
---
## 2. Phase 1 — Migrate Webview UI to Svelte
### 2.1 Why Svelte for VS Code Webviews
- **Compile-time framework:** Svelte compiles to imperative vanilla JavaScript. No virtual DOM overhead.
- **Small bundle size:** Critical for webview load times inside VS Code.
- **Fine-grained reactivity:** Svelte 5 Runes update exactly what changed, ideal for dashboards streaming data from the native binary.
- **Standard Vite integration:** You already use Vite; Svelte has first-class Vite support.
### 2.2 Project Structure
```
your-extension/
├── src/
│ ├── extension.ts # Extension Host entry
│ ├── engine/
│ │ └── spawn.ts # Native binary lifecycle
│ └── webview/
│ ├── main.ts # Svelte app mount point
│ ├── App.svelte # Root component
│ ├── lib/
│ │ ├── CanvasRenderer.svelte # WebGL/Canvas wrapper
│ │ └── MessageBus.ts # VS Code API wrapper
│ └── vite.config.ts # Webview build config
├── native/
│ ├── Cargo.toml / go.mod # Rust or Go project
│ └── src/main.rs / main.go # Native binary source
├── package.json
└── tsconfig.json
```
### 2.3 Webview Build Setup (Vite)
Create a separate Vite config for the webview that builds to a single inlined HTML file.
**`src/webview/vite.config.ts`**
```typescript
import { defineConfig } from 'vite';
import { svelte } from '@sveltejs/vite-plugin-svelte';
export default defineConfig({
plugins: [svelte()],
build: {
outDir: '../../out/webview',
emptyOutDir: true,
rollupOptions: {
input: './index.html',
output: {
entryFileNames: 'assets/[name].js',
chunkFileNames: 'assets/[name].js',
assetFileNames: 'assets/[name].[ext]',
},
},
},
css: { devSourcemap: true },
});
```
**`src/webview/index.html`**
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Extension UI</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="./main.ts"></script>
</body>
</html>
```
### 2.4 VS Code Webview Activation (Extension Host)
**`src/extension.ts`**
```typescript
import * as vscode from 'vscode';
import * as path from 'path';
export function activate(context: vscode.ExtensionContext) {
const disposable = vscode.commands.registerCommand('ext.openPanel', () => {
const panel = vscode.window.createWebviewPanel(
'myExtension',
'Data Visualizer',
vscode.ViewColumn.One,
{
enableScripts: true,
localResourceRoots: [vscode.Uri.joinPath(context.extensionUri, 'out', 'webview')],
}
);
const webviewUri = panel.webview.asWebviewUri(
vscode.Uri.joinPath(context.extensionUri, 'out', 'webview', 'index.html')
);
panel.webview.html = getWebviewContent(webviewUri.toString());
// Bridge: Extension Host ↔ Webview
panel.webview.onDidReceiveMessage(async (message) => {
if (message.type === 'requestData') {
const result = await nativeEngine.query(message.payload);
panel.webview.postMessage({ type: 'dataChunk', payload: result });
}
});
});
context.subscriptions.push(disposable);
}
function getWebviewContent(uri: string): string {
// In production, read the built HTML and replace asset paths.
return `<!DOCTYPE html>
<html>
<head>
<base href="${uri}/">
<script type="module" crossorigin src="${uri}/assets/main.js"></script>
<link rel="stylesheet" href="${uri}/assets/main.css">
</head>
<body>
<div id="app"></div>
</body>
</html>`;
}
```
### 2.5 Svelte Webview Entry Point
**`src/webview/main.ts`**
```typescript
import { mount } from 'svelte';
import App from './App.svelte';
import { vscode } from './lib/MessageBus';
// Acquire VS Code API (only once)
const vscodeApi = acquireVsCodeApi();
mount(App, {
target: document.getElementById('app')!,
props: { vscodeApi }
});
```
**`src/webview/lib/MessageBus.ts`**
```typescript
export const vscode = {
postMessage: (msg: unknown) => {
if (typeof acquireVsCodeApi === 'function') {
acquireVsCodeApi().postMessage(msg);
}
},
onMessage: (handler: (msg: any) => void) => {
window.addEventListener('message', (event) => handler(event.data));
}
};
```
### 2.6 Handling Visually Intensive Graphics
For heavy rendering, do **not** rely on DOM-based charting libraries alone. Use a dedicated renderer component inside Svelte.
**`src/webview/lib/CanvasRenderer.svelte`**
```svelte
<script lang="ts">
import { onMount } from 'svelte';
let canvas: HTMLCanvasElement;
let ctx: CanvasRenderingContext2D | WebGL2RenderingContext;
interface Props {
data: Float32Array;
mode: '2d' | 'webgl';
}
let { data, mode }: Props = $props();
onMount(() => {
if (mode === 'webgl') {
ctx = canvas.getContext('webgl2', { antialias: false })!;
initWebGL(ctx as WebGL2RenderingContext);
} else {
ctx = canvas.getContext('2d', { alpha: false })!;
}
});
// Reactive: re-render when data changes
$effect(() => {
if (!ctx) return;
if (mode === '2d') {
render2D(ctx as CanvasRenderingContext2D, data);
} else {
renderWebGL(ctx as WebGL2RenderingContext, data);
}
});
function render2D(c: CanvasRenderingContext2D, d: Float32Array) {
c.clearRect(0, 0, canvas.width, canvas.height);
// ... optimized drawing
}
function initWebGL(gl: WebGL2RenderingContext) {
// Compile shaders, create buffers
}
function renderWebGL(gl: WebGL2RenderingContext, d: Float32Array) {
// Upload buffer, drawArrays/drawElements
}
</script>
<canvas bind:this={canvas} width={800} height={600}></canvas>
```
**Best Practice:** For maximum performance, use **OffscreenCanvas** transferred to a Web Worker so rendering never blocks Svelte's reactive updates or user input.
---
## 3. Phase 2 — Data Processing via Native Binary Spawn
### 3.1 Why Native Binary Spawn Wins
| Approach | Pros | Cons |
|----------|------|------|
| **Native Binary Spawn** | Full multithreading, direct file I/O, any language, no sandbox limits, crashes don't kill extension host | Requires packaging per platform, async only |
| **WASM in Webview** | Near-native speed, no external binary | Single-threaded (mostly), no file system, large Rust/Go WASM output |
| **Node Native Addon (NAPI)** | Synchronous calls, zero IPC overhead | Build nightmare, VS Code host compatibility issues, harder to debug |
| **Pure TypeScript** | Simple, no build changes | Event loop blocking, memory limits, slow for parsing/processing |
**Verdict:** Spawn a long-lived native binary (Go or Rust) and communicate over **stdin/stdout JSON lines** or a local **gRPC/HTTP socket**.
### 3.2 Choosing Go vs Rust
| Factor | Go | Rust |
|--------|-----|------|
| **Development Speed** | Faster to write, great stdlib | Slower, steep learning curve |
| **Binary Size** | Larger (~2-5MB minimum) | Small (~1MB or less with strip+LTO) |
| **Startup Time** | Slower (GC + runtime init) | Instant |
| **Memory Control** | GC (acceptable for most cases) | Zero-cost, predictable |
| **Concurrency** | Goroutines (excellent) | Async/await + threads (excellent) |
| **JSON Performance** | Good | Excellent (simdjson-style parsers) |
| **Cross-Compilation** | Easy | Easy with `cross` or `cargo-zigbuild` |
**Recommendation:**
- Choose **Go** if you need rapid iteration, networking, or your team already knows it.
- Choose **Rust** if you need maximum throughput, lowest latency, smallest binary, or memory-intensive workloads.
### 3.3 Communication Protocol: JSON Lines over Stdio
The simplest robust protocol. The extension host spawns the binary and keeps it alive as a daemon.
**Protocol Rules**
1. Extension sends one JSON object per line (`\n` terminated).
2. Binary responds with one JSON object per line.
3. Use `id` for request/response correlation.
4. Binary can emit unsolicited `progress` or `log` messages.
**Example Message Flow**
```json
// Host -> Binary
{"id": 1, "method": "parseDataset", "params": {"path": "/tmp/large.csv", "columns": ["x", "y", "z"]}}
// Binary -> Host (progress)
{"id": 1, "type": "progress", "percent": 45}
// Binary -> Host (result)
{"id": 1, "type": "result", "data": {"rows": 5000000, "summary": {...}}}
```
### 3.4 Extension Host: Binary Lifecycle Manager
**`src/engine/spawn.ts`**
```typescript
import { spawn, ChildProcessWithoutNullStreams } from 'child_process';
import * as path from 'path';
import * as vscode from 'vscode';
import { EventEmitter } from 'events';
interface Request {
id: number;
resolve: (value: any) => void;
reject: (reason?: any) => void;
}
export class NativeEngine {
private proc: ChildProcessWithoutNullStreams;
private requests = new Map<number, Request>();
private idCounter = 0;
private buffer = '';
public events = new EventEmitter();
constructor(binaryPath: string) {
this.proc = spawn(binaryPath, ['--stdio'], {
stdio: ['pipe', 'pipe', 'pipe'],
});
this.proc.stdout.on('data', (chunk: Buffer) => {
this.buffer += chunk.toString('utf-8');
let lineEnd: number;
while ((lineEnd = this.buffer.indexOf('\n')) !== -1) {
const line = this.buffer.slice(0, lineEnd);
this.buffer = this.buffer.slice(lineEnd + 1);
this.handleLine(line);
}
});
this.proc.stderr.on('data', (chunk) => {
console.error('[NativeEngine]', chunk.toString());
});
this.proc.on('exit', (code) => {
vscode.window.showErrorMessage(`Native engine exited with code ${code}`);
});
}
private handleLine(line: string) {
try {
const msg = JSON.parse(line);
if (msg.type === 'progress' || msg.type === 'log') {
this.events.emit(msg.type, msg);
return;
}
const req = this.requests.get(msg.id);
if (req) {
this.requests.delete(msg.id);
if (msg.error) req.reject(msg.error);
else req.resolve(msg.data ?? msg);
}
} catch (e) {
console.error('Invalid JSON from native engine:', line);
}
}
send<T = any>(method: string, params?: unknown): Promise<T> {
return new Promise((resolve, reject) => {
const id = ++this.idCounter;
this.requests.set(id, { id, resolve, reject });
const payload = JSON.stringify({ id, method, params }) + '\n';
this.proc.stdin.write(payload);
});
}
dispose() {
this.proc.kill();
}
}
```
### 3.5 Native Engine Example: Go
**`native/main.go`**
```go
package main
import (
"bufio"
"encoding/json"
"fmt"
"os"
)
type Request struct {
ID int `json:"id"`
Method string `json:"method"`
Params json.RawMessage `json:"params"`
}
type Response struct {
ID int `json:"id"`
Data any `json:"data,omitempty"`
Error string `json:"error,omitempty"`
}
func main() {
scanner := bufio.NewScanner(os.Stdin)
writer := bufio.NewWriter(os.Stdout)
for scanner.Scan() {
var req Request
if err := json.Unmarshal(scanner.Bytes(), &req); err != nil {
continue
}
var resp Response
resp.ID = req.ID
switch req.Method {
case "parseDataset":
var p ParseParams
json.Unmarshal(req.Params, &p)
result, err := parseDataset(p)
if err != nil {
resp.Error = err.Error()
} else {
resp.Data = result
}
default:
resp.Error = "unknown method"
}
out, _ := json.Marshal(resp)
writer.Write(out)
writer.WriteByte('\n')
writer.Flush()
}
}
type ParseParams struct {
Path string `json:"path"`
Columns []string `json:"columns"`
}
func parseDataset(p ParseParams) (any, error) {
// Heavy I/O + CPU work here
// Use goroutines for parallel processing
return map[string]any{"rows": 5000000}, nil
}
```
**Build & Package**
```bash
# Build for host platform
cd native
go build -o ../bin/engine
# Cross-compile for all VS Code targets
goos_list=("darwin" "linux" "windows")
goarch_list=("amd64" "arm64")
for goos in "${goos_list[@]}"; do
for goarch in "${goarch_list[@]}"; do
output="../bin/engine-${goos}-${goarch}"
if [ "$goos" = "windows" ]; then output="${output}.exe"; fi
GOOS=$goos GOARCH=$goarch go build -ldflags="-s -w" -o "$output"
done
done
```
### 3.6 Native Engine Example: Rust
**`native/Cargo.toml`**
```toml
[package]
name = "engine"
version = "0.1.0"
edition = "2021"
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
rayon = "1.8" # Data parallelism
```
**`native/src/main.rs`**
```rust
use serde::{Deserialize, Serialize};
use std::io::{self, BufRead, Write};
#[derive(Debug, Deserialize)]
struct Request {
id: u64,
method: String,
#[serde(default)]
params: serde_json::Value,
}
#[derive(Debug, Serialize)]
struct Response {
id: u64,
#[serde(skip_serializing_if = "Option::is_none")]
data: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
error: Option<String>,
}
fn main() {
let stdin = io::stdin();
let mut stdout = io::stdout();
for line in stdin.lock().lines() {
let Ok(line) = line else { continue };
let Ok(req) = serde_json::from_str::<Request>(&line) else { continue };
let mut resp = Response { id: req.id, data: None, error: None };
match req.method.as_str() {
"parseDataset" => {
match parse_dataset(&req.params) {
Ok(data) => resp.data = Some(data),
Err(e) => resp.error = Some(e.to_string()),
}
}
_ => resp.error = Some("unknown method".into()),
}
let out = serde_json::to_string(&resp).unwrap();
writeln!(stdout, "{}", out).unwrap();
stdout.flush().unwrap();
}
}
fn parse_dataset(params: &serde_json::Value) -> Result<serde_json::Value, Box<dyn std::error::Error>> {
// Use rayon::join or par_iter for heavy work
Ok(serde_json::json!({ "rows": 5_000_000 }))
}
```
**Build & Package**
```bash
cd native
cargo build --release
# Cross-compile
cargo install cross
cross build --release --target x86_64-pc-windows-gnu
cross build --release --target x86_64-unknown-linux-gnu
cross build --release --target aarch64-apple-darwin
```
---
## 4. Phase 3 — Integration & Wiring
### 4.1 Extension Host as the Router
The Extension Host is the only process that can talk to both the Webview and the Native Binary. It must route messages efficiently.
```
Webview (Svelte) Extension Host (TS) Native Binary (Go/Rust)
| | |
|── requestData ───────────►| |
| |── parseDataset ─────────────►|
| |◄─ progress / result ──────────|
|◄─ dataChunk ──────────────| |
```
**`src/extension.ts` (Integration)**
```typescript
import { NativeEngine } from './engine/spawn';
import * as path from 'path';
import * as os from 'os';
let engine: NativeEngine;
export function activate(context: vscode.ExtensionContext) {
// Resolve platform-specific binary
const platform = os.platform();
const arch = os.arch();
const binName = platform === 'win32' ? 'engine.exe' : 'engine';
const binPath = path.join(context.extensionPath, 'bin', `${binName}-${platform}-${arch}`);
engine = new NativeEngine(binPath);
// Forward progress to webview
engine.events.on('progress', (msg) => {
panel.webview.postMessage({ type: 'engineProgress', payload: msg });
});
// Handle webview requests
panel.webview.onDidReceiveMessage(async (msg) => {
if (msg.type === 'queryEngine') {
const result = await engine.send(msg.method, msg.params);
panel.webview.postMessage({ type: 'queryResult', id: msg.id, payload: result });
}
});
}
export function deactivate() {
engine?.dispose();
}
```
### 4.2 Svelte: Reactive Data from Engine
**`src/webview/App.svelte`**
```svelte
<script lang="ts">
import { vscode } from './lib/MessageBus';
import CanvasRenderer from './lib/CanvasRenderer.svelte';
let progress = $state(0);
let dataset = $state<Float32Array | null>(null);
vscode.onMessage((msg) => {
if (msg.type === 'engineProgress') {
progress = msg.payload.percent;
}
if (msg.type === 'queryResult') {
// Convert result to typed array for renderer
dataset = new Float32Array(msg.payload.data);
}
});
function loadData() {
vscode.postMessage({
type: 'queryEngine',
id: crypto.randomUUID(),
method: 'parseDataset',
params: { path: '/tmp/data.csv', columns: ['x', 'y'] }
});
}
</script>
<main>
<button onclick={loadData}>Load Dataset</button>
{#if progress > 0 && progress < 100}
<progress value={progress} max={100}></progress>
{/if}
{#if dataset}
<CanvasRenderer {dataset} mode="webgl" />
{/if}
</main>
```
---
## 5. Phase 4 — Build & Development Workflow
### 5.1 VS Code Extension Build
Use `vsce` to package. Ensure your `package.json` includes the native binaries.
**`package.json`**
```json
{
"name": "your-extension",
"version": "0.1.0",
"main": "./out/extension.js",
"contributes": {
"commands": [{ "command": "ext.openPanel", "title": "Open Visualizer" }]
},
"scripts": {
"build:webview": "vite build -c src/webview/vite.config.ts",
"build:extension": "tsc -p ./",
"build:native": "cd native && go build -o ../bin/engine",
"build": "npm run build:webview && npm run build:extension && npm run build:native",
"package": "vsce package"
},
"devDependencies": {
"@sveltejs/vite-plugin-svelte": "^5.0.0",
"@types/vscode": "^1.90.0",
"svelte": "^5.0.0",
"typescript": "^5.6.0",
"vite": "^6.0.0",
"vsce": "^2.15.0"
}
}
```
### 5.2 Debugging Setup
**`.vscode/launch.json`**
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Run Extension",
"type": "extensionHost",
"request": "launch",
"args": ["--extensionDevelopmentPath=${workspaceFolder}"],
"outFiles": ["${workspaceFolder}/out/**/*.js"],
"preLaunchTask": "npm: build"
},
{
"name": "Attach to Native (Go)",
"type": "go",
"request": "attach",
"mode": "local",
"processId": "${command:pickProcess}"
},
{
"name": "Attach to Native (Rust)",
"type": "lldb",
"request": "attach",
"program": "${workspaceFolder}/bin/engine",
"pid": "${command:pickProcess}"
}
]
}
```
**Webview Debugging:**
- Open the Command Palette → `Developer: Open Webview Developer Tools`
- This opens Chrome DevTools for the webview. You can debug Svelte components, profile WebGL, and inspect OffscreenCanvas workers.
---
## 6. Migration Strategy & Transition Options
### Option A: Incremental Migration (Recommended)
Migrate piece by piece without breaking existing functionality.
| Step | Action | Risk |
|------|--------|------|
| 1 | Scaffold new Svelte webview alongside existing webview | Low |
| 2 | Port one visual panel to Svelte + Canvas/WebGL | Low |
| 3 | Extract one heavy data function to native binary | Medium — test parity |
| 4 | Switch communication from webview direct TS to Extension Host routing | Medium |
| 5 | Deprecate old webview, remove legacy code | Low |
### Option B: Full Rewrite
Only recommended if the current codebase is small (< 5k lines) or deeply coupled.
- Pro: Clean architecture from day one.
- Con: Longer time to ship, higher bug risk.
### Option C: Hybrid Staged
Keep the existing TypeScript data engine running, but wrap the heaviest function in a native binary. This proves the architecture before full commitment.
```typescript
// Fallback strategy
async function heavyCompute(data: any) {
if (nativeEngine.isReady()) {
return nativeEngine.send('heavyCompute', data);
}
// Fallback to legacy TS implementation
return legacyHeavyCompute(data);
}
```
---
## 7. Performance Checklist
### Webview (Svelte + Graphics)
- [ ] Use Svelte 5 Runes (`$state`, `$derived`, `$effect`) instead of legacy stores where possible.
- [ ] Never bind massive arrays directly to DOM elements. Pass them to Canvas/WebGL.
- [ ] Use `requestAnimationFrame` for all rendering loops; throttle data updates to 60fps.
- [ ] Move Canvas rendering to a Web Worker with `OffscreenCanvas` if the main thread drops frames.
- [ ] Profile with Webview DevTools Performance tab to confirm GPU compositing.
### Native Binary
- [ ] Keep the binary alive (daemon mode); do not spawn per request.
- [ ] Use buffered I/O (`bufio` in Go, `BufWriter` in Rust) to avoid syscall overhead.
- [ ] Stream large results in chunks rather than one giant JSON payload.
- [ ] Use goroutines (Go) or `rayon` (Rust) for parallel data processing.
- [ ] Strip symbols and use LTO for smaller binaries: `go build -ldflags="-s -w"` or `cargo build --release` with `strip = true`.
### Extension Host
- [ ] Never `await` a long native binary operation without yielding; use streaming responses.
- [ ] Dispose the native binary process on `deactivate()` to avoid zombie processes.
- [ ] Validate binary existence on activation and show a user-friendly error if the platform binary is missing.
---
## 8. Summary
| Concern | Your Stack | Rationale |
|---------|-----------|-----------|
| **UI Framework** | **Svelte 5** | Compile-time reactivity, smallest overhead, perfect for VS Code webviews |
| **Graphics** | **WebGL / Canvas API** inside Svelte | Hardware-accelerated, bypasses DOM limits |
| **Data Engine** | **Go or Rust binary** spawned via stdio | True multithreading, full I/O, no sandbox |
| **Build Tool** | **Vite** | Fast HMR, native Svelte plugin, easy webview bundling |
| **Communication** | **JSON Lines over stdio** | Simple, debuggable, language-agnostic |
This architecture separates concerns cleanly: Svelte manages reactive UI state, WebGL renders pixels, and a native binary crunches data without blocking either the Extension Host or the Webview.