Multi-Language WASM Development: Building Polyglot Applications for Cloud-Native
Multi-Language WASM Development: Building Polyglot Applications for Cloud-Native
WebAssembly's language-agnostic runtime enables unprecedented flexibility in application development. By combining the strengths of different programming languages within a single WASM application, developers can optimize for performance, productivity, and maintainability. This comprehensive guide explores multi-language WASM development patterns, tooling, and best practices for cloud-native deployments.
Table of Contents
- Multi-Language WASM Architecture
- Language Support and Toolchains
- Component Model and Interface Types
- Rust-Centric Development
- JavaScript and TypeScript Integration
- Go and WASM Development
- C/C++ Native Integration
- Python and Dynamic Languages
- Inter-Language Communication
- Performance Optimization Strategies
- Testing and Debugging
- Deployment Patterns
Multi-Language WASM Architecture
Language Integration Patterns
Polyglot WASM Application Architecture:
┌─────────────────────────────────────────────────────┐
│ Host Environment │
│ ┌─────────────────────────────────────────────┐ │
│ │ WASM Runtime │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ Rust │ │ Go │ │ JS │ │ │
│ │ │ Component│ │Component │ │Component │ │ │
│ │ └─────┬────┘ └─────┬────┘ └─────┬────┘ │ │
│ │ │ │ │ │ │
│ │ ┌─────┴─────────────┴─────────────┴─────┐ │ │
│ │ │ WIT Interface Layer │ │ │
│ │ └───────────────────────────────────────┘ │ │
│ │ ┌───────────────────────────────────────┐ │ │
│ │ │ WASI System Layer │ │ │
│ │ └───────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
Component Composition Strategies
# Multi-language application design
application_architecture:
data_processing:
language: rust
reason: "Performance-critical algorithms"
business_logic:
language: go
reason: "Concurrent request handling"
user_interface:
language: javascript
reason: "DOM manipulation and events"
system_integration:
language: c
reason: "Low-level system access"
machine_learning:
language: python
reason: "ML library ecosystem"
Language Support and Toolchains
Language Maturity Matrix
WASM Language Support (2024):
┌────────────────────────────────────────────────────┐
│ Language │ Maturity │ WASI │ Components │ Perf │
├────────────┼──────────┼───────┼────────────┼───────┤
│ Rust │ ★★★★★ │ ★★★★★ │ ★★★★★ │ ★★★★★ │
│ C/C++ │ ★★★★★ │ ★★★★☆ │ ★★★☆☆ │ ★★★★★ │
│ Go │ ★★★★☆ │ ★★★★☆ │ ★★★☆☆ │ ★★★★☆ │
│ JavaScript │ ★★★★☆ │ ★★★☆☆ │ ★★☆☆☆ │ ★★★☆☆ │
│ AssemblyScript │ ★★★☆☆│ ★★★☆☆ │ ★★☆☆☆ │ ★★★★☆ │
│ Python │ ★★☆☆☆ │ ★★☆☆☆ │ ★☆☆☆☆ │ ★★☆☆☆ │
│ C#/.NET │ ★★★☆☆ │ ★★★☆☆ │ ★★☆☆☆ │ ★★★☆☆ │
│ Java │ ★★☆☆☆ │ ★★☆☆☆ │ ★☆☆☆☆ │ ★★☆☆☆ │
└────────────────────────────────────────────────────┘
Toolchain Setup
# Comprehensive multi-language setup
# Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup target add wasm32-wasi
cargo install wasm-tools
# Go with WASM support
go install golang.org/x/tools/cmd/goimports@latest
export GOOS=wasip1
export GOARCH=wasm
# Node.js and WASM tools
npm install -g @assemblyscript/cli
npm install -g wabt
# C/C++ with WASI-SDK
wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-linux.tar.gz
tar xf wasi-sdk-20.0-linux.tar.gz
export WASI_SDK_PATH=/opt/wasi-sdk-20.0
# Python WASM (Pyodide)
pip install pyodide-build
# Component tooling
cargo install wit-bindgen-cli
cargo install wasm-component-ld
Component Model and Interface Types
WIT Interface Definition
// world.wit - Interface definition
package example:[email protected];
interface data-processing {
record user-data {
id: u64,
name: string,
email: string,
score: f64,
}
process-users: func(users: list<user-data>) -> list<user-data>;
calculate-statistics: func(data: list<f64>) -> tuple<f64, f64, f64>;
}
interface business-logic {
use data-processing.{user-data};
enum validation-error {
invalid-email,
duplicate-id,
missing-field,
}
validate-user: func(user: user-data) -> result<user-data, validation-error>;
apply-business-rules: func(users: list<user-data>) -> list<user-data>;
}
interface ui-components {
render-table: func(data: string) -> string;
handle-event: func(event-type: string, data: string) -> string;
}
world multi-lang-app {
import wasi:filesystem/[email protected];
import wasi:http/[email protected];
export data-processing;
export business-logic;
export ui-components;
}
Component Binding Generation
# Generate language bindings
wit-bindgen rust --world multi-lang-app --out-dir rust/src/bindings world.wit
wit-bindgen go --world multi-lang-app --out-dir go/bindings world.wit
wit-bindgen js --world multi-lang-app --out-dir js/bindings world.wit
wit-bindgen c --world multi-lang-app --out-dir c/bindings world.wit
Rust-Centric Development
High-Performance Core Components
// rust/src/data_processor.rs
use crate::bindings::exports::example::multi_lang::data_processing::{
Guest, UserData
};
use rayon::prelude::*;
pub struct DataProcessor;
impl Guest for DataProcessor {
fn process_users(users: Vec<UserData>) -> Vec<UserData> {
// Parallel processing with Rayon
users
.into_par_iter()
.map(|mut user| {
// Expensive computation
user.score = calculate_advanced_score(&user);
normalize_data(&mut user);
user
})
.collect()
}
fn calculate_statistics(data: Vec<f64>) -> (f64, f64, f64) {
if data.is_empty() {
return (0.0, 0.0, 0.0);
}
let sum: f64 = data.par_iter().sum();
let mean = sum / data.len() as f64;
let variance = data
.par_iter()
.map(|x| (x - mean).powi(2))
.sum::<f64>() / data.len() as f64;
let std_dev = variance.sqrt();
(mean, variance, std_dev)
}
}
fn calculate_advanced_score(user: &UserData) -> f64 {
// Complex scoring algorithm
let email_score = if user.email.contains('@') && user.email.contains('.') {
10.0
} else {
0.0
};
let name_score = user.name.len() as f64 * 0.5;
let id_score = (user.id as f64).log10() * 2.0;
(email_score + name_score + id_score).max(0.0).min(100.0)
}
fn normalize_data(user: &mut UserData) {
user.name = user.name.trim().to_string();
user.email = user.email.to_lowercase();
}
// Build configuration
// Cargo.toml
[package]
name = "data-processor"
version = "0.1.0"
edition = "2021"
[lib]
crate-type = ["cdylib"]
[dependencies]
wit-bindgen = "0.16"
rayon = "1.8"
serde = { version = "1.0", features = ["derive"] }
[profile.release]
opt-level = "z"
lto = true
strip = true
Rust Component Build
# Build Rust component
cd rust
cargo component build --release
# Verify component
wasm-tools component wit target/wasm32-wasi/release/data_processor.wasm
# Optimize
wasm-opt -Os target/wasm32-wasi/release/data_processor.wasm \
-o ../components/data-processor.wasm
JavaScript and TypeScript Integration
UI Component Implementation
// js/src/ui-components.ts
import { UiComponents } from './bindings/ui-components';
export class UIComponentsImpl implements UiComponents {
renderTable(data: string): string {
const users = JSON.parse(data);
const html = `
<table class="users-table">
<thead>
<tr>
<th>ID</th>
<th>Name</th>
<th>Email</th>
<th>Score</th>
</tr>
</thead>
<tbody>
${users.map((user: any) => `
<tr data-user-id="${user.id}">
<td>${user.id}</td>
<td>${user.name}</td>
<td>${user.email}</td>
<td>${user.score.toFixed(2)}</td>
</tr>
`).join('')}
</tbody>
</table>
`;
return html;
}
handleEvent(eventType: string, data: string): string {
const eventData = JSON.parse(data);
switch (eventType) {
case 'user-click':
return this.handleUserClick(eventData);
case 'filter-change':
return this.handleFilterChange(eventData);
case 'sort-column':
return this.handleSortColumn(eventData);
default:
return JSON.stringify({ error: 'Unknown event type' });
}
}
private handleUserClick(data: any): string {
const user = data.user;
const modal = `
<div class="user-modal">
<h2>User Details</h2>
<p><strong>ID:</strong> ${user.id}</p>
<p><strong>Name:</strong> ${user.name}</p>
<p><strong>Email:</strong> ${user.email}</p>
<p><strong>Score:</strong> ${user.score}</p>
<button onclick="closeModal()">Close</button>
</div>
`;
return JSON.stringify({ action: 'showModal', content: modal });
}
private handleFilterChange(data: any): string {
const { field, value, users } = data;
const filtered = users.filter((user: any) => {
const fieldValue = user[field]?.toString().toLowerCase() || '';
return fieldValue.includes(value.toLowerCase());
});
return JSON.stringify({ action: 'updateTable', users: filtered });
}
private handleSortColumn(data: any): string {
const { column, direction, users } = data;
const sorted = [...users].sort((a, b) => {
const aVal = a[column];
const bVal = b[column];
if (typeof aVal === 'number' && typeof bVal === 'number') {
return direction === 'asc' ? aVal - bVal : bVal - aVal;
}
const aStr = aVal.toString().toLowerCase();
const bStr = bVal.toString().toLowerCase();
if (direction === 'asc') {
return aStr.localeCompare(bStr);
} else {
return bStr.localeCompare(aStr);
}
});
return JSON.stringify({ action: 'updateTable', users: sorted });
}
}
JavaScript Build Pipeline
{
"name": "ui-components",
"version": "1.0.0",
"type": "module",
"scripts": {
"build": "rollup -c",
"build:wasm": "jco componentize ui-components.js -o ui-components.wasm"
},
"devDependencies": {
"@rollup/plugin-typescript": "^11.1.5",
"@rollup/plugin-node-resolve": "^15.2.3",
"rollup": "^4.9.0",
"@bytecodealliance/jco": "^1.0.0",
"typescript": "^5.3.0"
}
}
// rollup.config.js
import typescript from '@rollup/plugin-typescript';
import { nodeResolve } from '@rollup/plugin-node-resolve';
export default {
input: 'src/ui-components.ts',
output: {
file: 'ui-components.js',
format: 'es'
},
plugins: [
nodeResolve(),
typescript()
]
};
Go and WASM Development
Business Logic Implementation
// go/business_logic.go
package main
import (
"encoding/json"
"regexp"
"strings"
"sync"
)
//go:generate wit-bindgen go --world multi-lang-app --out-dir bindings .
type BusinessLogic struct{}
type UserData struct {
ID uint64 `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Score float64 `json:"score"`
}
type ValidationError int
const (
InvalidEmail ValidationError = iota
DuplicateID
MissingField
)
var emailRegex = regexp.MustCompile(`^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$`)
func (bl *BusinessLogic) ValidateUser(user UserData) (UserData, *ValidationError) {
// Validate required fields
if user.Name == "" || user.Email == "" {
err := MissingField
return user, &err
}
// Validate email format
if !emailRegex.MatchString(user.Email) {
err := InvalidEmail
return user, &err
}
// Additional validations
user.Name = strings.TrimSpace(user.Name)
user.Email = strings.ToLower(strings.TrimSpace(user.Email))
return user, nil
}
func (bl *BusinessLogic) ApplyBusinessRules(users []UserData) []UserData {
// Use goroutines for concurrent processing
var wg sync.WaitGroup
userChan := make(chan UserData, len(users))
// Worker pool for processing
numWorkers := 4
if len(users) < numWorkers {
numWorkers = len(users)
}
// Start workers
for i := 0; i < numWorkers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for user := range userChan {
processedUser := applyRules(user)
// Send processed user back (simplified)
_ = processedUser
}
}()
}
// Send users to workers
go func() {
defer close(userChan)
for _, user := range users {
userChan <- user
}
}()
wg.Wait()
// Apply global rules
result := make([]UserData, 0, len(users))
seenIDs := make(map[uint64]bool)
for _, user := range users {
// Remove duplicates
if seenIDs[user.ID] {
continue
}
seenIDs[user.ID] = true
// Apply business rules
processedUser := applyRules(user)
result = append(result, processedUser)
}
return result
}
func applyRules(user UserData) UserData {
// Business rule: VIP users get score bonus
if strings.Contains(strings.ToLower(user.Email), "vip") {
user.Score += 10.0
}
// Business rule: Score caps
if user.Score > 100.0 {
user.Score = 100.0
} else if user.Score < 0.0 {
user.Score = 0.0
}
// Business rule: Name formatting
words := strings.Fields(user.Name)
for i, word := range words {
if len(word) > 0 {
words[i] = strings.ToUpper(word[:1]) + strings.ToLower(word[1:])
}
}
user.Name = strings.Join(words, " ")
return user
}
func main() {
// WASM component entry point
}
Go Build Configuration
# Build Go component
cd go
export GOOS=wasip1
export GOARCH=wasm
go mod init business-logic
go mod tidy
# Generate bindings
go generate
# Build component
go build -o business-logic.wasm
# Convert to component
wasm-tools component new business-logic.wasm \
--wit ../world.wit \
-o ../components/business-logic.wasm
C/C++ Native Integration
System-Level Operations
// c/system_interface.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/stat.h>
#include <fcntl.h>
// Generated bindings
#include "bindings/system_interface.h"
typedef struct {
char* data;
size_t size;
size_t capacity;
} buffer_t;
// High-performance file operations
int read_large_file(const char* path, buffer_t* buffer) {
struct stat st;
if (stat(path, &st) != 0) {
return -1;
}
size_t file_size = st.st_size;
if (buffer->capacity < file_size) {
buffer->data = realloc(buffer->data, file_size);
if (!buffer->data) {
return -1;
}
buffer->capacity = file_size;
}
int fd = open(path, O_RDONLY);
if (fd < 0) {
return -1;
}
ssize_t bytes_read = read(fd, buffer->data, file_size);
close(fd);
if (bytes_read != file_size) {
return -1;
}
buffer->size = file_size;
return 0;
}
// Memory-efficient data processing
void process_binary_data(buffer_t* buffer) {
if (!buffer || !buffer->data) return;
// SIMD-optimized processing (simplified)
size_t i;
unsigned char* data = (unsigned char*)buffer->data;
// Process in chunks for cache efficiency
const size_t chunk_size = 4096;
for (i = 0; i < buffer->size; i += chunk_size) {
size_t end = (i + chunk_size < buffer->size) ? i + chunk_size : buffer->size;
// Process chunk
for (size_t j = i; j < end; j++) {
// Example: simple encryption/transformation
data[j] = (data[j] ^ 0xAA) + 1;
}
}
}
// Low-level network operations
int create_socket_connection(const char* host, int port) {
// Simplified socket creation
// In real implementation, would use proper socket APIs
printf("Creating connection to %s:%d\n", host, port);
// Return mock file descriptor
return 3;
}
// Performance-critical sorting
void quicksort_doubles(double* arr, int low, int high) {
if (low < high) {
int pi = partition_doubles(arr, low, high);
quicksort_doubles(arr, low, pi - 1);
quicksort_doubles(arr, pi + 1, high);
}
}
int partition_doubles(double* arr, int low, int high) {
double pivot = arr[high];
int i = (low - 1);
for (int j = low; j <= high - 1; j++) {
if (arr[j] < pivot) {
i++;
swap_doubles(&arr[i], &arr[j]);
}
}
swap_doubles(&arr[i + 1], &arr[high]);
return (i + 1);
}
void swap_doubles(double* a, double* b) {
double temp = *a;
*a = *b;
*b = temp;
}
// WASM exports
__attribute__((export_name("process_file")))
int process_file(const char* path) {
buffer_t buffer = {0};
if (read_large_file(path, &buffer) != 0) {
return -1;
}
process_binary_data(&buffer);
// Write processed data back
int fd = open(path, O_WRONLY | O_TRUNC);
if (fd >= 0) {
write(fd, buffer.data, buffer.size);
close(fd);
}
free(buffer.data);
return 0;
}
__attribute__((export_name("sort_array")))
void sort_array(double* data, int length) {
quicksort_doubles(data, 0, length - 1);
}
C/C++ Build System
# CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(system_interface)
set(CMAKE_C_STANDARD 11)
set(CMAKE_CXX_STANDARD 17)
# WASI SDK configuration
set(CMAKE_SYSTEM_NAME WASI)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR wasm32)
set(CMAKE_C_COMPILER "${WASI_SDK_PATH}/bin/clang")
set(CMAKE_CXX_COMPILER "${WASI_SDK_PATH}/bin/clang++")
set(CMAKE_LINKER "${WASI_SDK_PATH}/bin/wasm-ld")
# Optimization flags
set(CMAKE_C_FLAGS_RELEASE "-O3 -flto -fno-exceptions")
set(CMAKE_CXX_FLAGS_RELEASE "-O3 -flto -fno-exceptions")
# Source files
set(SOURCES
system_interface.c
bindings/system_interface_bindings.c
)
# Build WASM module
add_executable(system_interface ${SOURCES})
# Link flags
target_link_options(system_interface PRIVATE
-Wl,--no-entry
-Wl,--export-all
-Wl,--allow-undefined
)
# Set output name
set_target_properties(system_interface PROPERTIES OUTPUT_NAME "system_interface.wasm")
# Build C component
cd c
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make
# Convert to component
wasm-tools component new system_interface.wasm \
--wit ../../world.wit \
-o ../../components/system-interface.wasm
Python and Dynamic Languages
ML and Data Science Integration
# python/ml_processor.py
import json
import numpy as np
from typing import List, Dict, Any, Tuple
from dataclasses import dataclass
import pickle
from io import BytesIO
@dataclass
class UserData:
id: int
name: str
email: str
score: float
class MLProcessor:
def __init__(self):
self.model = None
self.scaler = None
self.feature_names = ['name_length', 'email_length', 'domain_score', 'char_diversity']
def extract_features(self, users: List[UserData]) -> np.ndarray:
"""Extract ML features from user data"""
features = []
for user in users:
# Feature engineering
name_length = len(user.name)
email_length = len(user.email)
# Domain scoring
domain = user.email.split('@')[-1] if '@' in user.email else ''
domain_score = self._score_domain(domain)
# Character diversity
char_diversity = len(set(user.name.lower())) / max(len(user.name), 1)
features.append([name_length, email_length, domain_score, char_diversity])
return np.array(features)
def _score_domain(self, domain: str) -> float:
"""Score email domain based on common patterns"""
common_domains = {
'gmail.com': 0.9,
'yahoo.com': 0.8,
'outlook.com': 0.8,
'hotmail.com': 0.7,
'aol.com': 0.6
}
if domain in common_domains:
return common_domains[domain]
elif domain.endswith('.edu'):
return 0.95
elif domain.endswith('.gov'):
return 0.98
elif domain.endswith('.com'):
return 0.7
else:
return 0.5
def train_model(self, users: List[UserData], labels: List[float]):
"""Train ML model on user data"""
try:
from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score
except ImportError:
# Fallback to simple linear model
return self._train_simple_model(users, labels)
# Extract features
X = self.extract_features(users)
y = np.array(labels)
# Split data
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
# Scale features
self.scaler = StandardScaler()
X_train_scaled = self.scaler.fit_transform(X_train)
X_test_scaled = self.scaler.transform(X_test)
# Train model
self.model = RandomForestRegressor(
n_estimators=100,
max_depth=10,
random_state=42
)
self.model.fit(X_train_scaled, y_train)
# Evaluate
y_pred = self.model.predict(X_test_scaled)
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)
return {
'mse': mse,
'r2': r2,
'feature_importance': dict(zip(self.feature_names, self.model.feature_importances_))
}
def _train_simple_model(self, users: List[UserData], labels: List[float]):
"""Fallback simple model when sklearn unavailable"""
X = self.extract_features(users)
y = np.array(labels)
# Simple linear regression using normal equation
# Add bias term
X_with_bias = np.column_stack([np.ones(X.shape[0]), X])
# Calculate weights: w = (X^T X)^-1 X^T y
XtX = X_with_bias.T @ X_with_bias
XtX_inv = np.linalg.pinv(XtX) # Pseudo-inverse for numerical stability
Xty = X_with_bias.T @ y
self.model = XtX_inv @ Xty
# Calculate R²
y_pred = X_with_bias @ self.model
ss_res = np.sum((y - y_pred) ** 2)
ss_tot = np.sum((y - np.mean(y)) ** 2)
r2 = 1 - (ss_res / ss_tot)
return {'r2': r2, 'model': 'simple_linear'}
def predict_scores(self, users: List[UserData]) -> List[float]:
"""Predict scores for users"""
if self.model is None:
return [50.0] * len(users) # Default score
X = self.extract_features(users)
if hasattr(self.model, 'predict'): # sklearn model
if self.scaler:
X = self.scaler.transform(X)
predictions = self.model.predict(X)
else: # simple model
X_with_bias = np.column_stack([np.ones(X.shape[0]), X])
predictions = X_with_bias @ self.model
# Clip predictions to valid range
return np.clip(predictions, 0, 100).tolist()
def save_model(self, path: str):
"""Save trained model"""
model_data = {
'model': self.model,
'scaler': self.scaler,
'feature_names': self.feature_names
}
with open(path, 'wb') as f:
pickle.dump(model_data, f)
def load_model(self, path: str):
"""Load trained model"""
with open(path, 'rb') as f:
model_data = pickle.load(f)
self.model = model_data['model']
self.scaler = model_data.get('scaler')
self.feature_names = model_data.get('feature_names', self.feature_names)
# WASM interface functions
def process_ml_training(users_json: str, labels_json: str) -> str:
"""Train ML model and return metrics"""
try:
users_data = json.loads(users_json)
labels_data = json.loads(labels_json)
users = [UserData(**user) for user in users_data]
labels = labels_data
processor = MLProcessor()
metrics = processor.train_model(users, labels)
return json.dumps({
'status': 'success',
'metrics': metrics
})
except Exception as e:
return json.dumps({
'status': 'error',
'message': str(e)
})
def predict_user_scores(users_json: str, model_path: str = None) -> str:
"""Predict scores for users"""
try:
users_data = json.loads(users_json)
users = [UserData(**user) for user in users_data]
processor = MLProcessor()
if model_path:
processor.load_model(model_path)
predictions = processor.predict_scores(users)
return json.dumps({
'status': 'success',
'predictions': predictions
})
except Exception as e:
return json.dumps({
'status': 'error',
'message': str(e)
})
if __name__ == "__main__":
# Example usage
sample_users = [
{"id": 1, "name": "John Doe", "email": "[email protected]", "score": 0},
{"id": 2, "name": "Jane Smith", "email": "[email protected]", "score": 0}
]
result = predict_user_scores(json.dumps(sample_users))
print(result)
Python Build with Pyodide
# build.py
import os
import subprocess
import shutil
from pathlib import Path
def build_python_wasm():
"""Build Python code to WASM using Pyodide"""
# Install dependencies
subprocess.run([
'pip', 'install', 'pyodide-build', 'numpy', 'scikit-learn'
], check=True)
# Create package structure
package_dir = Path('ml_processor_pkg')
package_dir.mkdir(exist_ok=True)
# Copy source files
shutil.copy('ml_processor.py', package_dir)
# Create setup.py
setup_py = '''
from setuptools import setup, find_packages
setup(
name="ml_processor",
version="1.0.0",
packages=find_packages(),
install_requires=[
"numpy",
"scikit-learn"
]
)
'''
with open(package_dir / 'setup.py', 'w') as f:
f.write(setup_py)
# Build WASM package
os.chdir(package_dir)
subprocess.run([
'pyodide', 'build'
], check=True)
print("Python WASM build complete!")
if __name__ == "__main__":
build_python_wasm()
Inter-Language Communication
Component Composition Framework
// composer/src/main.rs
use wasmtime::{Engine, Store, Module, Instance, Config};
use wasmtime_wasi::WasiCtxBuilder;
use anyhow::Result;
use serde_json::Value;
pub struct MultiLanguageApp {
engine: Engine,
store: Store<WasiCtx>,
data_processor: Instance,
business_logic: Instance,
ui_components: Instance,
ml_processor: Option<Instance>,
}
impl MultiLanguageApp {
pub async fn new() -> Result<Self> {
// Configure WASM runtime
let mut config = Config::new();
config.wasm_component_model(true);
config.async_support(true);
let engine = Engine::new(&config)?;
// Setup WASI context
let wasi = WasiCtxBuilder::new()
.inherit_stdio()
.inherit_env()?
.preopened_dir("/tmp", "tmp")?
.build();
let mut store = Store::new(&engine, wasi);
// Load components
let data_processor_module = Module::from_file(&engine, "components/data-processor.wasm")?;
let business_logic_module = Module::from_file(&engine, "components/business-logic.wasm")?;
let ui_components_module = Module::from_file(&engine, "components/ui-components.wasm")?;
// Instantiate components
let data_processor = Instance::new(&mut store, &data_processor_module, &[])?;
let business_logic = Instance::new(&mut store, &business_logic_module, &[])?;
let ui_components = Instance::new(&mut store, &ui_components_module, &[])?;
// Optional ML processor
let ml_processor = if std::path::Path::new("components/ml-processor.wasm").exists() {
let ml_module = Module::from_file(&engine, "components/ml-processor.wasm")?;
Some(Instance::new(&mut store, &ml_module, &[])?)
} else {
None
};
Ok(Self {
engine,
store,
data_processor,
business_logic,
ui_components,
ml_processor,
})
}
pub async fn process_user_request(&mut self, request: UserRequest) -> Result<String> {
match request.action.as_str() {
"process_data" => self.handle_data_processing(request.data).await,
"validate_users" => self.handle_user_validation(request.data).await,
"render_ui" => self.handle_ui_rendering(request.data).await,
"ml_predict" => self.handle_ml_prediction(request.data).await,
"composite" => self.handle_composite_operation(request.data).await,
_ => Ok(json!({"error": "Unknown action"}).to_string()),
}
}
async fn handle_data_processing(&mut self, data: Value) -> Result<String> {
// Call Rust data processor
let process_users = self.data_processor
.get_typed_func::<(Vec<u8>,), (Vec<u8>,)>(&mut self.store, "process_users")?;
let input = serde_json::to_vec(&data)?;
let (output,) = process_users.call_async(&mut self.store, (input,)).await?;
Ok(String::from_utf8(output)?)
}
async fn handle_user_validation(&mut self, data: Value) -> Result<String> {
// Call Go business logic
let validate_user = self.business_logic
.get_typed_func::<(Vec<u8>,), (Vec<u8>,)>(&mut self.store, "validate_user")?;
let input = serde_json::to_vec(&data)?;
let (output,) = validate_user.call_async(&mut self.store, (input,)).await?;
Ok(String::from_utf8(output)?)
}
async fn handle_ui_rendering(&mut self, data: Value) -> Result<String> {
// Call JavaScript UI components
let render_table = self.ui_components
.get_typed_func::<(String,), (String,)>(&mut self.store, "render_table")?;
let input = data.to_string();
let (output,) = render_table.call_async(&mut self.store, (input,)).await?;
Ok(output)
}
async fn handle_ml_prediction(&mut self, data: Value) -> Result<String> {
// Call Python ML processor if available
if let Some(ml_processor) = &self.ml_processor {
let predict_scores = ml_processor
.get_typed_func::<(String,), (String,)>(&mut self.store, "predict_user_scores")?;
let input = data.to_string();
let (output,) = predict_scores.call_async(&mut self.store, (input,)).await?;
Ok(output)
} else {
Ok(json!({"error": "ML processor not available"}).to_string())
}
}
async fn handle_composite_operation(&mut self, data: Value) -> Result<String> {
// Complex workflow across multiple components
// Step 1: Process data (Rust)
let processed_data = self.handle_data_processing(data.clone()).await?;
let processed: Value = serde_json::from_str(&processed_data)?;
// Step 2: Validate users (Go)
let validated_data = self.handle_user_validation(processed).await?;
let validated: Value = serde_json::from_str(&validated_data)?;
// Step 3: ML prediction (Python, if available)
let final_data = if self.ml_processor.is_some() {
let predicted_data = self.handle_ml_prediction(validated).await?;
serde_json::from_str(&predicted_data)?
} else {
validated
};
// Step 4: Render UI (JavaScript)
let ui_html = self.handle_ui_rendering(final_data).await?;
Ok(json!({
"status": "success",
"html": ui_html
}).to_string())
}
}
#[derive(serde::Deserialize)]
struct UserRequest {
action: String,
data: Value,
}
use wasmtime_wasi::WasiCtx;
Message Passing Interface
// message_bus.rs
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use tokio::sync::{mpsc, oneshot};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Message {
pub id: String,
pub from: String,
pub to: String,
pub method: String,
pub payload: serde_json::Value,
pub reply_to: Option<String>,
}
#[derive(Debug)]
pub struct MessageBus {
channels: Arc<Mutex<HashMap<String, mpsc::UnboundedSender<Message>>>>,
pending_replies: Arc<Mutex<HashMap<String, oneshot::Sender<Message>>>>,
}
impl MessageBus {
pub fn new() -> Self {
Self {
channels: Arc::new(Mutex::new(HashMap::new())),
pending_replies: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn register_component(&self, component_id: String) -> mpsc::UnboundedReceiver<Message> {
let (tx, rx) = mpsc::unbounded_channel();
let mut channels = self.channels.lock().unwrap();
channels.insert(component_id, tx);
rx
}
pub async fn send_message(&self, message: Message) -> Result<(), String> {
let channels = self.channels.lock().unwrap();
if let Some(sender) = channels.get(&message.to) {
sender.send(message).map_err(|e| e.to_string())?;
Ok(())
} else {
Err(format!("Component {} not found", message.to))
}
}
pub async fn send_request(&self, mut message: Message) -> Result<Message, String> {
let (reply_tx, reply_rx) = oneshot::channel();
let request_id = Uuid::new_v4().to_string();
message.id = request_id.clone();
// Store reply channel
{
let mut pending = self.pending_replies.lock().unwrap();
pending.insert(request_id, reply_tx);
}
// Send message
self.send_message(message).await?;
// Wait for reply
reply_rx.await.map_err(|e| e.to_string())
}
pub async fn send_reply(&self, reply: Message) -> Result<(), String> {
if let Some(reply_to) = &reply.reply_to {
let mut pending = self.pending_replies.lock().unwrap();
if let Some(sender) = pending.remove(reply_to) {
sender.send(reply).map_err(|e| e.to_string())?;
return Ok(());
}
}
Err("No pending request found".to_string())
}
}
// Component wrapper for easy integration
pub struct ComponentWrapper {
pub component_id: String,
pub message_bus: Arc<MessageBus>,
pub receiver: mpsc::UnboundedReceiver<Message>,
}
impl ComponentWrapper {
pub fn new(component_id: String, message_bus: Arc<MessageBus>) -> Self {
let receiver = message_bus.register_component(component_id.clone());
Self {
component_id,
message_bus,
receiver,
}
}
pub async fn call_component(&self, target: &str, method: &str, payload: serde_json::Value) -> Result<serde_json::Value, String> {
let message = Message {
id: Uuid::new_v4().to_string(),
from: self.component_id.clone(),
to: target.to_string(),
method: method.to_string(),
payload,
reply_to: None,
};
let response = self.message_bus.send_request(message).await?;
Ok(response.payload)
}
pub async fn handle_messages<F>(&mut self, mut handler: F)
where
F: FnMut(Message) -> Result<Option<Message>, String>,
{
while let Some(message) = self.receiver.recv().await {
match handler(message.clone()) {
Ok(Some(reply)) => {
let mut reply_msg = reply;
reply_msg.reply_to = Some(message.id);
if let Err(e) = self.message_bus.send_reply(reply_msg).await {
eprintln!("Failed to send reply: {}", e);
}
},
Ok(None) => {
// No reply needed
},
Err(e) => {
eprintln!("Error handling message: {}", e);
// Send error reply
let error_reply = Message {
id: Uuid::new_v4().to_string(),
from: self.component_id.clone(),
to: message.from,
method: "error".to_string(),
payload: serde_json::json!({"error": e}),
reply_to: Some(message.id),
};
let _ = self.message_bus.send_reply(error_reply).await;
}
}
}
}
}
Performance Optimization Strategies
Cross-Language Memory Management
// memory_manager.rs
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use std::alloc::{GlobalAlloc, Layout, System};
pub struct SharedMemoryManager {
pools: Arc<Mutex<HashMap<usize, Vec<*mut u8>>>>,
allocations: Arc<Mutex<HashMap<*mut u8, Layout>>>,
}
impl SharedMemoryManager {
pub fn new() -> Self {
Self {
pools: Arc::new(Mutex::new(HashMap::new())),
allocations: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn allocate(&self, size: usize, align: usize) -> *mut u8 {
let layout = Layout::from_size_align(size, align).unwrap();
// Try to reuse from pool
if let Ok(mut pools) = self.pools.lock() {
if let Some(pool) = pools.get_mut(&size) {
if let Some(ptr) = pool.pop() {
return ptr;
}
}
}
// Allocate new
unsafe {
let ptr = System.alloc(layout);
if !ptr.is_null() {
if let Ok(mut allocations) = self.allocations.lock() {
allocations.insert(ptr, layout);
}
}
ptr
}
}
pub fn deallocate(&self, ptr: *mut u8) {
if let Ok(mut allocations) = self.allocations.lock() {
if let Some(layout) = allocations.remove(&ptr) {
// Add to pool for reuse
if let Ok(mut pools) = self.pools.lock() {
let pool = pools.entry(layout.size()).or_insert_with(Vec::new);
pool.push(ptr);
return;
}
}
}
// Fallback: actually deallocate
unsafe {
System.dealloc(ptr, Layout::from_size_align_unchecked(0, 1));
}
}
pub fn cleanup_pools(&self) {
if let Ok(mut pools) = self.pools.lock() {
for (size, pool) in pools.iter_mut() {
for ptr in pool.drain(..) {
unsafe {
let layout = Layout::from_size_align_unchecked(*size, 1);
System.dealloc(ptr, layout);
}
}
}
}
}
}
// Global memory manager instance
static MEMORY_MANAGER: once_cell::sync::Lazy<SharedMemoryManager> =
once_cell::sync::Lazy::new(|| SharedMemoryManager::new());
// WASM-compatible allocator
#[no_mangle]
pub extern "C" fn wasm_alloc(size: usize) -> *mut u8 {
MEMORY_MANAGER.allocate(size, 8)
}
#[no_mangle]
pub extern "C" fn wasm_dealloc(ptr: *mut u8) {
MEMORY_MANAGER.deallocate(ptr);
}
Performance Monitoring
// profiler.rs
use std::time::{Duration, Instant};
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
#[derive(Debug, Clone)]
pub struct PerformanceMetrics {
pub function_name: String,
pub language: String,
pub call_count: u64,
pub total_duration: Duration,
pub avg_duration: Duration,
pub min_duration: Duration,
pub max_duration: Duration,
}
pub struct PerformanceProfiler {
metrics: Arc<Mutex<HashMap<String, PerformanceMetrics>>>,
}
impl PerformanceProfiler {
pub fn new() -> Self {
Self {
metrics: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn start_measurement(&self, function_name: &str, language: &str) -> MeasurementGuard {
MeasurementGuard {
profiler: Arc::clone(&self.metrics),
function_name: function_name.to_string(),
language: language.to_string(),
start_time: Instant::now(),
}
}
pub fn get_metrics(&self) -> Vec<PerformanceMetrics> {
let metrics = self.metrics.lock().unwrap();
metrics.values().cloned().collect()
}
pub fn reset_metrics(&self) {
let mut metrics = self.metrics.lock().unwrap();
metrics.clear();
}
}
pub struct MeasurementGuard {
profiler: Arc<Mutex<HashMap<String, PerformanceMetrics>>>,
function_name: String,
language: String,
start_time: Instant,
}
impl Drop for MeasurementGuard {
fn drop(&mut self) {
let duration = self.start_time.elapsed();
let key = format!("{}::{}", self.language, self.function_name);
if let Ok(mut metrics) = self.profiler.lock() {
let metric = metrics.entry(key).or_insert_with(|| PerformanceMetrics {
function_name: self.function_name.clone(),
language: self.language.clone(),
call_count: 0,
total_duration: Duration::new(0, 0),
avg_duration: Duration::new(0, 0),
min_duration: Duration::from_secs(u64::MAX),
max_duration: Duration::new(0, 0),
});
metric.call_count += 1;
metric.total_duration += duration;
metric.avg_duration = metric.total_duration / metric.call_count as u32;
metric.min_duration = metric.min_duration.min(duration);
metric.max_duration = metric.max_duration.max(duration);
}
}
}
// Macro for easy profiling
#[macro_export]
macro_rules! profile {
($profiler:expr, $lang:expr, $func:expr, $body:expr) => {
{
let _guard = $profiler.start_measurement($func, $lang);
$body
}
};
}
Testing and Debugging
Multi-Language Test Suite
// tests/integration_test.rs
use tokio_test;
use serde_json::json;
use crate::MultiLanguageApp;
#[tokio::test]
async fn test_data_processing_pipeline() {
let mut app = MultiLanguageApp::new().await.unwrap();
let test_data = json!([
{"id": 1, "name": "John Doe", "email": "[email protected]", "score": 0.0},
{"id": 2, "name": "Jane Smith", "email": "[email protected]", "score": 0.0}
]);
// Test Rust data processing
let processed = app.handle_data_processing(test_data.clone()).await.unwrap();
let processed_users: serde_json::Value = serde_json::from_str(&processed).unwrap();
assert!(processed_users.is_array());
assert_eq!(processed_users.as_array().unwrap().len(), 2);
// Verify scores were calculated
for user in processed_users.as_array().unwrap() {
assert!(user["score"].as_f64().unwrap() > 0.0);
}
}
#[tokio::test]
async fn test_business_logic_validation() {
let mut app = MultiLanguageApp::new().await.unwrap();
let test_cases = vec![
// Valid user
json!({"id": 1, "name": "John Doe", "email": "[email protected]", "score": 50.0}),
// Invalid email
json!({"id": 2, "name": "Jane Smith", "email": "invalid-email", "score": 75.0}),
// Missing name
json!({"id": 3, "name": "", "email": "[email protected]", "score": 25.0}),
];
for test_case in test_cases {
let result = app.handle_user_validation(test_case.clone()).await.unwrap();
let validation_result: serde_json::Value = serde_json::from_str(&result).unwrap();
// Check validation logic
if test_case["email"].as_str().unwrap().contains('@') &&
!test_case["name"].as_str().unwrap().is_empty() {
assert_eq!(validation_result["status"], "valid");
} else {
assert_eq!(validation_result["status"], "invalid");
}
}
}
#[tokio::test]
async fn test_ui_rendering() {
let mut app = MultiLanguageApp::new().await.unwrap();
let users_data = json!([
{"id": 1, "name": "John Doe", "email": "[email protected]", "score": 85.5},
{"id": 2, "name": "Jane Smith", "email": "[email protected]", "score": 92.3}
]);
let html_result = app.handle_ui_rendering(users_data).await.unwrap();
// Verify HTML structure
assert!(html_result.contains("<table"));
assert!(html_result.contains("John Doe"));
assert!(html_result.contains("Jane Smith"));
assert!(html_result.contains("85.5"));
assert!(html_result.contains("92.3"));
}
#[tokio::test]
async fn test_composite_workflow() {
let mut app = MultiLanguageApp::new().await.unwrap();
let initial_data = json!([
{"id": 1, "name": " john doe ", "email": "[email protected]", "score": 0.0},
{"id": 2, "name": "jane smith", "email": "[email protected]", "score": 0.0}
]);
let result = app.handle_composite_operation(initial_data).await.unwrap();
let composite_result: serde_json::Value = serde_json::from_str(&result).unwrap();
assert_eq!(composite_result["status"], "success");
assert!(composite_result["html"].is_string());
let html = composite_result["html"].as_str().unwrap();
assert!(html.contains("John Doe")); // Name should be properly formatted
assert!(html.contains("[email protected]")); // Email should be normalized
}
#[tokio::test]
async fn test_performance_benchmarks() {
let mut app = MultiLanguageApp::new().await.unwrap();
// Generate large dataset
let large_dataset: Vec<serde_json::Value> = (0..10000)
.map(|i| json!({
"id": i,
"name": format!("User {}", i),
"email": format!("user{}@test.com", i),
"score": 0.0
}))
.collect();
let start_time = std::time::Instant::now();
// Test processing performance
let result = app.handle_data_processing(json!(large_dataset)).await.unwrap();
let processing_time = start_time.elapsed();
// Verify performance requirements
assert!(processing_time.as_millis() < 5000, "Processing took too long: {:?}", processing_time);
let processed_users: serde_json::Value = serde_json::from_str(&result).unwrap();
assert_eq!(processed_users.as_array().unwrap().len(), 10000);
}
#[tokio::test]
async fn test_error_handling() {
let mut app = MultiLanguageApp::new().await.unwrap();
// Test with invalid JSON
let invalid_data = json!("invalid_structure");
let result = app.handle_data_processing(invalid_data).await;
assert!(result.is_err() || result.unwrap().contains("error"));
// Test with missing required fields
let incomplete_data = json!([{"id": 1}]); // Missing name and email
let validation_result = app.handle_user_validation(incomplete_data).await.unwrap();
let parsed_result: serde_json::Value = serde_json::from_str(&validation_result).unwrap();
assert_eq!(parsed_result["status"], "invalid");
assert!(parsed_result["error"].is_string());
}
Debugging Tools
// debug_tools.rs
use std::collections::HashMap;
use serde_json::Value;
pub struct DebugTracer {
trace_log: Vec<TraceEntry>,
enabled: bool,
}
#[derive(Debug, Clone)]
pub struct TraceEntry {
pub timestamp: std::time::SystemTime,
pub component: String,
pub function: String,
pub input: String,
pub output: String,
pub duration: std::time::Duration,
}
impl DebugTracer {
pub fn new() -> Self {
Self {
trace_log: Vec::new(),
enabled: std::env::var("WASM_DEBUG").is_ok(),
}
}
pub fn trace_call(&mut self, component: &str, function: &str, input: &Value, output: &Value, duration: std::time::Duration) {
if !self.enabled {
return;
}
let entry = TraceEntry {
timestamp: std::time::SystemTime::now(),
component: component.to_string(),
function: function.to_string(),
input: serde_json::to_string_pretty(input).unwrap_or_default(),
output: serde_json::to_string_pretty(output).unwrap_or_default(),
duration,
};
self.trace_log.push(entry);
// Print real-time trace
println!("[TRACE] {}::{} ({:?})", component, function, duration);
}
pub fn export_trace(&self, format: &str) -> String {
match format {
"json" => serde_json::to_string_pretty(&self.trace_log).unwrap_or_default(),
"html" => self.generate_html_report(),
_ => self.generate_text_report(),
}
}
fn generate_html_report(&self) -> String {
let mut html = String::from(r#"
<!DOCTYPE html>
<html>
<head>
<title>WASM Multi-Language Debug Trace</title>
<style>
body { font-family: monospace; margin: 20px; }
.trace-entry { border: 1px solid #ccc; margin: 10px; padding: 10px; }
.component { color: #0066cc; font-weight: bold; }
.function { color: #cc6600; }
.duration { color: #009900; }
.io-data { background: #f5f5f5; padding: 5px; margin: 5px 0; }
</style>
</head>
<body>
<h1>Multi-Language WASM Debug Trace</h1>
"#);
for entry in &self.trace_log {
html.push_str(&format!(r#"
<div class="trace-entry">
<div>
<span class="component">{}</span>::<span class="function">{}</span>
<span class="duration">({:?})</span>
</div>
<div>Timestamp: {:?}</div>
<div class="io-data">
<h4>Input:</h4>
<pre>{}</pre>
</div>
<div class="io-data">
<h4>Output:</h4>
<pre>{}</pre>
</div>
</div>
"#, entry.component, entry.function, entry.duration, entry.timestamp, entry.input, entry.output));
}
html.push_str("</body></html>");
html
}
fn generate_text_report(&self) -> String {
let mut report = String::from("=== WASM Multi-Language Debug Trace ===\n\n");
for entry in &self.trace_log {
report.push_str(&format!(
"[{:?}] {}::{} ({:?})\nInput: {}\nOutput: {}\n\n",
entry.timestamp, entry.component, entry.function, entry.duration,
entry.input, entry.output
));
}
report
}
}
// Component-specific debugging
pub fn debug_rust_component(input: &str) -> String {
if std::env::var("DEBUG_RUST").is_ok() {
println!("[RUST DEBUG] Input: {}", input);
}
// Process input
let output = format!("Processed: {}", input);
if std::env::var("DEBUG_RUST").is_ok() {
println!("[RUST DEBUG] Output: {}", output);
}
output
}
#[no_mangle]
pub extern "C" fn debug_log(level: i32, component_ptr: *const u8, component_len: usize, message_ptr: *const u8, message_len: usize) {
unsafe {
let component = std::str::from_utf8(std::slice::from_raw_parts(component_ptr, component_len)).unwrap_or("unknown");
let message = std::str::from_utf8(std::slice::from_raw_parts(message_ptr, message_len)).unwrap_or("invalid message");
let level_str = match level {
0 => "DEBUG",
1 => "INFO",
2 => "WARN",
3 => "ERROR",
_ => "UNKNOWN",
};
println!("[{}] [{}] {}", level_str, component, message);
}
}
Deployment Patterns
Kubernetes Multi-Language Deployment
# k8s/multi-lang-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: multi-lang-wasm-app
labels:
app: multi-lang-wasm
spec:
replicas: 10
selector:
matchLabels:
app: multi-lang-wasm
template:
metadata:
labels:
app: multi-lang-wasm
spec:
runtimeClassName: wasmtime
containers:
- name: app-orchestrator
image: registry.example.com/multi-lang-orchestrator:v1.0.0
ports:
- containerPort: 8080
env:
- name: WASM_COMPONENTS_PATH
value: "/app/components"
- name: DEBUG_MODE
value: "false"
- name: PERFORMANCE_MONITORING
value: "true"
resources:
requests:
memory: "50Mi"
cpu: "100m"
limits:
memory: "200Mi"
cpu: "500m"
volumeMounts:
- name: wasm-components
mountPath: /app/components
readOnly: true
- name: config
mountPath: /app/config
readOnly: true
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 3
periodSeconds: 5
volumes:
- name: wasm-components
configMap:
name: wasm-components
- name: config
configMap:
name: app-config
---
apiVersion: v1
kind: ConfigMap
metadata:
name: wasm-components
data:
data-processor.wasm: |
# Base64 encoded Rust component
business-logic.wasm: |
# Base64 encoded Go component
ui-components.wasm: |
# Base64 encoded JavaScript component
ml-processor.wasm: |
# Base64 encoded Python component
---
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
data:
app.yaml: |
server:
port: 8080
host: "0.0.0.0"
components:
data_processor:
language: rust
file: data-processor.wasm
max_memory: 50MB
timeout: 30s
business_logic:
language: go
file: business-logic.wasm
max_memory: 30MB
timeout: 15s
ui_components:
language: javascript
file: ui-components.wasm
max_memory: 20MB
timeout: 10s
ml_processor:
language: python
file: ml-processor.wasm
max_memory: 100MB
timeout: 60s
optional: true
performance:
enable_profiling: true
trace_calls: false
memory_monitoring: true
---
apiVersion: v1
kind: Service
metadata:
name: multi-lang-wasm-service
spec:
selector:
app: multi-lang-wasm
ports:
- protocol: TCP
port: 80
targetPort: 8080
type: LoadBalancer
CI/CD Pipeline for Multi-Language Components
# .github/workflows/multi-lang-build.yml
name: Multi-Language WASM Build
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
jobs:
build-rust:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Rust
uses: actions-rs/toolchain@v1
with:
toolchain: stable
target: wasm32-wasi
- name: Build Rust component
run: |
cd rust
cargo build --target wasm32-wasi --release
wasm-opt -Os target/wasm32-wasi/release/data_processor.wasm \
-o ../artifacts/data-processor.wasm
- name: Upload Rust artifact
uses: actions/upload-artifact@v3
with:
name: rust-component
path: artifacts/data-processor.wasm
build-go:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: '1.21'
- name: Build Go component
run: |
cd go
export GOOS=wasip1
export GOARCH=wasm
go build -o business-logic.wasm
wasm-tools component new business-logic.wasm \
--wit ../world.wit \
-o ../artifacts/business-logic.wasm
- name: Upload Go artifact
uses: actions/upload-artifact@v3
with:
name: go-component
path: artifacts/business-logic.wasm
build-javascript:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Build JavaScript component
run: |
cd js
npm ci
npm run build
jco componentize ui-components.js -o ../artifacts/ui-components.wasm
- name: Upload JavaScript artifact
uses: actions/upload-artifact@v3
with:
name: js-component
path: artifacts/ui-components.wasm
build-python:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Build Python component
run: |
cd python
pip install pyodide-build
python build.py
cp ml_processor_pkg/dist/ml_processor.wasm ../artifacts/ml-processor.wasm
- name: Upload Python artifact
uses: actions/upload-artifact@v3
with:
name: python-component
path: artifacts/ml-processor.wasm
test-integration:
needs: [build-rust, build-go, build-javascript, build-python]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Download all artifacts
uses: actions/download-artifact@v3
with:
path: components/
- name: Setup test environment
run: |
# Install WASM runtime
curl -sSL https://wasmtime.dev/install.sh | bash
source ~/.bashrc
# Install test tools
cargo install wasm-tools
- name: Run integration tests
run: |
cd tests
cargo test --release -- --test-threads=1
- name: Performance benchmarks
run: |
cd benchmarks
cargo bench
build-container:
needs: [test-integration]
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Download components
uses: actions/download-artifact@v3
with:
path: components/
- name: Build container image
run: |
docker build -t multi-lang-wasm:${{ github.sha }} .
docker tag multi-lang-wasm:${{ github.sha }} multi-lang-wasm:latest
- name: Push to registry
run: |
echo ${{ secrets.REGISTRY_PASSWORD }} | docker login -u ${{ secrets.REGISTRY_USERNAME }} --password-stdin
docker push multi-lang-wasm:${{ github.sha }}
docker push multi-lang-wasm:latest
deploy:
needs: [build-container]
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Deploy to Kubernetes
run: |
echo "${{ secrets.KUBECONFIG }}" | base64 -d > kubeconfig
export KUBECONFIG=kubeconfig
kubectl set image deployment/multi-lang-wasm-app \
app-orchestrator=multi-lang-wasm:${{ github.sha }}
kubectl rollout status deployment/multi-lang-wasm-app
Conclusion
Multi-language WebAssembly development represents the future of polyglot programming in cloud-native environments. By leveraging the strengths of different programming languages within a unified WASM runtime, developers can build applications that are simultaneously performant, maintainable, and feature-rich.
Key Benefits Realized
- ✅ Language Specialization: Use the right language for each component
- ✅ Unified Deployment: Single WASM runtime for all components
- ✅ Type Safety: Strong interfaces between language boundaries
- ✅ Performance: Near-native speed across all languages
- ✅ Security: Sandboxed execution with controlled capabilities
- ✅ Portability: Run anywhere with WASM support
Best Practices Summary
- Design clear interfaces using WIT for component communication
- Choose languages strategically based on specific use cases
- Implement comprehensive testing across language boundaries
- Monitor performance at the component and system level
- Plan for debugging with proper tooling and tracing
- Optimize memory usage with shared pools and efficient allocation
Future Directions
The multi-language WASM ecosystem continues to evolve rapidly:
- Enhanced component model support across more languages
- Better debugging tools for polyglot applications
- Improved performance through advanced compilation techniques
- Richer standard libraries for WASI compatibility
- Cloud provider integration with managed WASM services
Next Steps
Ready to explore WASM orchestration platforms? Our next article covers SpinKube and Fermyon for managing multi-language WASM applications at scale!
Resources
- Component Model Specification
- WIT Interface Definition
- Language-Specific WASM Guides
- Multi-Language Examples
The era of language silos is ending. Welcome to the age of polyglot WebAssembly! 🚀