Dynamic Linking Fundamentals for Software Modules

In modern software development, enabling dynamic updates or module additions without redeploying the core application is a common requirement. This capability is foundational for features like hot-swapping and plugin architectures.

Platform-Specific Implementations

Two primary mechanisms exist: Dynamic Link Libraries (DLLs) on Windows and Dynamic Shared Objects (DSOs, or .so files) on Linux systems.

Windows Dynamic Link Libraries

• Enable code and resource sharing across multiple processes.
• Facilitate modular program design with separate component files.
• Allow cross-language interoperability through standardized binary interfaces.
• Simplify updates by replacing individual modules.
• Provide groundwork for runtime module hot-swapping.
• Expose programming APIs for external consumption.

Linux Dynamic Shared Objects

DSOs offer the same core advantages as DLLs while addressing platform-specific challenges. They often serve as compatibility layers, abstracting low-level system interfaces to provide consistent APIs for applications.

Interpreted Language Approach

For interpreted languages, dynamic behavior is inherent since source code is executed directly. Modifying the source implements updates without compilasion steps.

Go's Native Dynamic Capabilities

Go, as a compiled language, doesn't natively support dynamic source code loading. The standard library provides the plugin package, but with significant limitations.

go build -buildmode=plugin

Current constraints (as of Go 1.19) include:

• Limited platform support (Linux, FreeBSD, macOS only)
• No unload capability after loading
• Requires reflection for accessing plugin symbols
• Official development appears stagnant

Third-Party Interpreter: Yaegi

Yaegi (Yet Another Go Interpreter) enables Go source code execution at runtime. Originally developed for the Traefik proxy's plugin system, it supports three primary use cases:

Embedded Interpreter Pattern

package main

import (
    "github.com/traefik/yaegi/interp"
    "github.com/traefik/yaegi/stdlib"
)

func main() {
    engine := interp.New(interp.Options{})
    engine.Use(stdlib.Symbols)
    
    _, err := engine.Eval(`import "fmt"`)
    if err != nil {
        panic(err)
    }
    
    _, err = engine.Eval(`fmt.Println("Running Go dynamically")`)
    if err != nil {
        panic(err)
    }
}

Dynamic Extension Framework

package main

import "github.com/traefik/yaegi/interp"

const extensionCode = `package mathutil
func Double(n int) int { return n * 2 }`

func main() {
    vm := interp.New(interp.Options{})
    
    _, err := vm.Eval(extensionCode)
    if err != nil {
        panic(err)
    }
    
    funcSymbol, err := vm.Eval("mathutil.Double")
    if err != nil {
        panic(err)
    }
    
    doubleFunc := funcSymbol.Interface().(func(int) int)
    result := doubleFunc(21)
    println(result) // Output: 42
}

REPL Environment

Yaegi provides a read-eval-print-loop interface:

$ yaegi
> x := 10 + 32
> import "fmt"
> fmt.Printf("Result: %d\n", x)
Result: 42

Data Exchange Between Host and Interpreter

Input Mechanisms

Multiple approaches exist for passing data into the interpreter environment:

package main

import (
    "fmt"
    "github.com/traefik/yaegi/interp"
    "github.com/traefik/yaegi/stdlib"
)

func main() {
    // Method 1: Command-line arguments
    vm1 := interp.New(interp.Options{
        Args: []string{"test-value"},
    })
    vm1.Use(stdlib.Symbols)
    vm1.Eval(`import "os"`)
    vm1.Eval(`println("Argument:", os.Args[0])`)
    
    // Method 2: Environment variables
    vm2 := interp.New(interp.Options{
        Env: []string{"CONFIG_PATH=/etc/app"},
    })
    vm2.Use(stdlib.Symbols)
    vm2.Eval(`import "os"`)
    vm2.Eval(`println("Config:", os.Getenv("CONFIG_PATH"))`)
    
    // Method 3: Direct assignment
    vm3 := interp.New(interp.Options{})
    vm3.Use(stdlib.Symbols)
    vm3.Eval(`externalData := "injected"`)
    vm3.Eval(`println("Data:", externalData)`)
    
    // Method 4: Function injection
    vm4 := interp.New(interp.Options{})
    vm4.Use(stdlib.Symbols)
    vm4.Eval(`var sharedConfig map[string]interface{}`)
    vm4.Eval(`func LoadConfig(cfg map[string]interface{}) { 
        sharedConfig = cfg 
    }`)
    
    loader, _ := vm4.Eval("LoadConfig")
    configLoader := loader.Interface().(func(map[string]interface{}))
    configLoader(map[string]interface{}{
        "timeout": 30,
        "debug":   true,
    })
    vm4.Eval(`println("Timeout:", sharedConfig["timeout"])`)
}

Output Mechanisms

Retrieving data from the interpreter context:

package main

import (
    "fmt"
    "github.com/traefik/yaegi/interp"
    "github.com/traefik/yaegi/stdlib"
)

func main() {
    // Direct evaluation retrieval
    engine := interp.New(interp.Options{})
    engine.Use(stdlib.Symbols)
    engine.Eval(`computedValue := 100 * 2`)
    
    val, _ := engine.Eval("computedValue")
    numericResult := val.Interface().(int)
    fmt.Printf("Value: %d\n", numericResult)
    
    // Function-based retrieval
    engine.Eval(`
        func RetrieveResult() int {
            return computedValue * 2
        }
    `)
    
    retriever, _ := engine.Eval("RetrieveResult")
    resultFunc := retriever.Interface().(func() int)
    fmt.Printf("Doubled: %d\n", resultFunc())
    
    // Global symbol inspection
    engine.Eval(`appName := "demo"`)
    engine.Eval(`appVersion := 2`)
    
    for symbolName, symbolValue := range engine.Globals() {
        value := symbolValue.Interface()
        switch v := value.(type) {
        case string:
            fmt.Printf("%s -> %q\n", symbolName, v)
        case int:
            fmt.Printf("%s -> %d\n", symbolName, v)
        }
    }
}

Implementation Through Abstract Syntax Trees

Interpreters typical work by parsing source code into Abstract Syntax Trees (ASTs). Go's standard library provides robust AST manipulation capabilities through the go/ast package.

AST Representation Example

Consider the expression: A!=1 && (B>1 || (C<1 && A>2))

This compiles to a tree structure where each node represents a syntactic element:

• Binary expressions (&&, ||, !=, >, <)
• Identifiers (A, B, C)
• Literal values (1, 2)
• Parenthesized sub-expressions

AST Evaluation Implementation

package main

import (
    "fmt"
    "go/ast"
    "go/parser"
    "go/token"
    "strconv"
)

type EvaluationContext map[string]int

func main() {
    ctx := EvaluationContext{"A": 1, "B": 2, "C": 3}
    
    expressions := []string{
        `A == 1 && (B > 1 || C < 1)`,
        `A != 1 && (B > 1 || (C < 1 && A > 2))`,
    }
    
    for _, expr := range expressions {
        parsed, _ := parser.ParseExpr(expr)
        result := evaluateExpression(parsed, ctx)
        fmt.Printf("%s => %v\n", expr, result)
    }
}

func evaluateExpression(node ast.Expr, ctx EvaluationContext) interface{} {
    switch n := node.(type) {
    case *ast.BinaryExpr:
        left := evaluateExpression(n.X, ctx)
        right := evaluateExpression(n.Y, ctx)
        
        switch n.Op.String() {
        case "&&":
            return left.(bool) && right.(bool)
        case "||":
            return left.(bool) || right.(bool)
        case ">":
            return left.(int) > right.(int)
        case "<":
            return left.(int) < right.(int)
        case "==":
            return left.(int) == right.(int)
        case "!=":
            return left.(int) != right.(int)
        }
        
    case *ast.BasicLit:
        if n.Kind == token.INT {
            val, _ := strconv.Atoi(n.Value)
            return val
        }
        
    case *ast.Ident:
        return ctx[n.Name]
        
    case *ast.ParenExpr:
        return evaluateExpression(n.X, ctx)
    }
    
    return nil
}