Why I Built WitDatabase: The Plugin Problem Nobody Solved

The Problem

It started with a simple requirement: a plugin system where each plugin could have its own database.

The plugins needed to be fully autonomous — self-contained assemblies that could be loaded and unloaded without affecting the host application. Each plugin would store its own data, run its own migrations, and manage its own schema. The host application used Entity Framework Core with PostgreSQL in production, so naturally, the plugins should use EF Core too.

Simple enough, right?

I tried SQLite first. It's the obvious choice for embedded scenarios. But SQLite has native dependencies — platform-specific binaries that need to be present at runtime. In a plugin isolation context, those dependencies don't resolve correctly. The plugin can't find them, even when they're physically present in the plugin folder.

I spent days on this. I tried copying the native libraries manually. I tried different package configurations. I tried embedding the full SQLite distribution. Nothing worked reliably.

Then I looked at LiteDB. It's pure .NET, no native dependencies — exactly what I needed. But LiteDB is a document database. My EF Core models, my migrations, my LINQ queries — none of it was compatible. I'd have to rewrite everything for a different paradigm.

I searched for alternatives. There were none.

No pure-managed .NET database with full EF Core support existed.

So I built one.

Why SQLite Fails in Plugins

SQLite itself is excellent. The problem is how it's packaged for .NET.

When you add Microsoft.Data.Sqlite to your project, you're not just getting C# code. You're getting native binaries:

• e_sqlite3.dll on Windows
• libe_sqlite3.so on Linux
• libe_sqlite3.dylib on macOS

These binaries are compiled C code. They're loaded at runtime through P/Invoke. And this is where plugin isolation breaks down.

How Plugin Isolation Works

In a plugin architecture, each plugin is loaded into its own AssemblyLoadContext. This provides isolation — plugins can have different versions of dependencies without conflicting. The host application and each plugin live in separate worlds.

But native libraries don't play by these rules.

Native binaries are loaded by the operating system, not by .NET. They're resolved from specific paths: the application directory, system PATH, or explicitly configured locations. They don't respect AssemblyLoadContext boundaries.

What Goes Wrong

When a plugin tries to use SQLite:

1. The plugin's managed code loads correctly into its isolated context
2. The code calls SQLite, which triggers P/Invoke
3. .NET asks the OS to load e_sqlite3.dll
4. The OS looks in the host application's directory — not the plugin's directory
5. The native library isn't found
6. Exception: DllNotFoundException or Unable to load DLL

You can copy the native binaries to the plugin folder. You can set DllImportSearchPath. You can try runtime configuration. I tried all of it.

Sometimes it works on one platform and fails on another. Sometimes it works in development and fails in production. Sometimes it works until you load a second plugin that also uses SQLite — then you get version conflicts.

The errors are maddening:

System.DllNotFoundException: Unable to load DLL 'e_sqlite3'
SQLite.Interop.dll cannot be loaded
The specified module could not be found

After days of wrestling with this, I accepted the truth: native dependencies and plugin isolation are fundamentally incompatible. Not just difficult — incompatible.

Why LiteDB Isn't the Answer

LiteDB is a great project. Ten thousand GitHub stars, millions of NuGet downloads, years of production use. It's pure .NET with no native dependencies — exactly what I needed technically.

But LiteDB is a document database.

// LiteDB way
var collection = db.GetCollection<Customer>("customers");
collection.Insert(new Customer { Name = "John", Email = "john@example.com" });
var results = collection.Query()
    .Where(x => x.Name.StartsWith("J"))
    .ToList();

This is not SQL. This is not Entity Framework. The query API is different. The data model is different. The entire paradigm is different.

My production code used EF Core:

// EF Core way
var customers = await context.Customers
    .Include(c => c.Orders)
    .Where(c => c.Name.StartsWith("J"))
    .ToListAsync();

I had migrations. I had relationships with foreign keys. I had complex LINQ queries with joins and groupings. None of this translates naturally to a document model.

Yes, there's an EF Core adapter for LiteDB. I looked at it. It's a valiant effort, but it's fighting against the grain. Document databases don't have schemas. They don't have foreign key constraints. They don't have the same transaction semantics. The adapter can simulate some of this, but it's not the same thing.

I didn't want to rewrite my data layer. I didn't want to learn a new query API. I didn't want to give up migrations.

I wanted to change one line of configuration and have everything else stay the same.

The Real Requirement

Let me state clearly what I actually needed:

One DbContext. One set of migrations. Multiple deployment scenarios.

// The same DbContext everywhere
public class PluginDbContext : DbContext
{
    public DbSet<Customer> Customers => Set<Customer>();
    public DbSet<Order> Orders => Set<Order>();
    
    // Same relationships, same configuration
    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Order>()
            .HasOne(o => o.Customer)
            .WithMany(c => c.Orders)
            .HasForeignKey(o => o.CustomerId);
    }
}

// Production: PostgreSQL
options.UseNpgsql(connectionString);

// Plugin deployment: embedded, isolated, no native deps
options.UseWitDb("Data Source=plugin_data.witdb");

// Integration tests: fast, no infrastructure
options.UseWitDb("Data Source=:memory:");

The migrations generated for PostgreSQL should work with the embedded database. The LINQ queries should execute identically. The relationships should be enforced. The transactions should be ACID-compliant.

And critically: no native dependencies. The plugin assembly and its dependencies must be pure .NET code that runs in an isolated AssemblyLoadContext without any platform-specific binaries.

This requirement isn't exotic. It's what you need for:

• Plugin architectures with data persistence
• Integration tests without database containers
• Portable demos that run from a USB drive
• Desktop applications with embedded storage
• CI pipelines that don't need Docker

The tooling gap was clear. I wasn't asking for something unreasonable. I was asking for a pure-managed EF Core database provider.

It didn't exist. So I built WitDatabase.

The Solution: Pure Managed Database

WitDatabase is 100% C#. No P/Invoke. No native binaries. No platform-specific code.

When you add OutWit.Database.EntityFramework to your project, you're getting:

• A complete SQL parser and query engine
• B+Tree and LSM-Tree storage engines
• ACID transactions with WAL journaling
• Full EF Core provider implementation

All in managed code. All running in any AssemblyLoadContext. All working on any platform .NET supports — including Blazor WebAssembly.

[[Svg Src="./witdatabase-efcore-provider-stack.svg" Alt="witdatabase-efcore-provider-stack"]]

The EF Core integration isn't an adapter or a compatibility layer. It's a proper provider that implements the same interfaces as Npgsql, Pomelo.MySql, or Microsoft.Data.SqlClient. Migrations work. LINQ translation works. Change tracking works.

The difference is what's underneath: pure managed code that loads and runs anywhere .NET runs.

How It Works in Practice

The Plugin Scenario

Your host application loads plugins dynamically:

// Host application
var context = new AssemblyLoadContext("Plugin1", isCollectible: true);
var pluginAssembly = context.LoadFromAssemblyPath(pluginPath);
var pluginType = pluginAssembly.GetType("MyPlugin.Plugin");
var plugin = Activator.CreateInstance(pluginType);

Inside the plugin, you configure your DbContext:

// Inside the plugin assembly
public class PluginStartup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddDbContext<PluginDbContext>(options =>
            options.UseWitDb("Data Source=plugin_data.witdb"));
    }
}

That's it. No native binaries to copy. No runtime configuration to tweak. No platform-specific code paths. The plugin loads, the database initializes, queries execute.

The Test Scenario

Your integration tests need a database, but spinning up PostgreSQL containers is slow:

public class CustomerServiceTests : IDisposable
{
    private readonly AppDbContext _context;
    
    public CustomerServiceTests()
    {
        var options = new DbContextOptionsBuilder<AppDbContext>()
            .UseWitDb("Data Source=:memory:")  // In-memory for speed
            .Options;
            
        _context = new AppDbContext(options);
        _context.Database.EnsureCreated();
    }
    
    [Fact]
    public async Task CreateCustomer_ShouldPersist()
    {
        var service = new CustomerService(_context);
        
        var customer = await service.CreateAsync("John", "john@example.com");
        
        Assert.NotEqual(0, customer.Id);
        Assert.Equal("John", customer.Name);
    }
}

Same DbContext as production. Same EF Core queries. No Docker. No connection strings to external databases. Tests run in milliseconds.

The Demo Scenario

You need to show your application to a client. They don't have PostgreSQL installed. They don't want to install anything.

// appsettings.Demo.json
{
    "ConnectionStrings": {
        "Default": "Data Source=demo_data.witdb"
    }
}

Ship the .exe and the .witdb file. Double-click to run. The demo works offline, on any Windows/Mac/Linux machine, with realistic data.

Configuration-Based Switching

For maximum flexibility, switch providers based on configuration:

var provider = configuration["Database:Provider"];
var connectionString = configuration.GetConnectionString("Default");

services.AddDbContext<AppDbContext>(options =>
{
    switch (provider)
    {
        case "PostgreSQL":
            options.UseNpgsql(connectionString);
            break;
        case "SqlServer":
            options.UseSqlServer(connectionString);
            break;
        case "Embedded":
            options.UseWitDb(connectionString);
            break;
    }
});

One codebase. One set of migrations. Multiple deployment targets.

The Scenarios This Enables

Once you have a pure-managed EF Core database, new possibilities open up:

CI Pipeline Impact

A typical CI pipeline with database tests:

# Before: need database service
services:
  - postgres:14
  
steps:
  - name: Wait for database
    run: sleep 10
  - name: Run tests
    run: dotnet test

# After: no external dependencies
steps:
  - name: Run tests
    run: dotnet test

No service containers. No waiting for database startup. No connection string management. Tests just run.

Plugin Architecture Impact

[[Svg Src="./witdatabase-plugin-deployment-comparison.svg" Alt="witdatabase-plugin-deployment-comparison"]]

No native binaries to manage. No platform-specific builds. No DllNotFoundException at runtime.

Migration Compatibility

The promise of "same migrations, different providers" requires some explanation.

EF Core migrations are provider-specific to some degree. A migration generated for PostgreSQL might use SERIAL for auto-increment, while SQL Server uses IDENTITY. WitDatabase has its own SQL dialect.

However, the model is the same. The entities, relationships, indexes, and constraints are defined in your C# code. When you switch providers, you regenerate migrations for that provider, but your model code doesn't change.

// This model works with any provider
public class Order
{
    public int Id { get; set; }
    public int CustomerId { get; set; }
    public decimal TotalAmount { get; set; }
    public DateTime OrderDate { get; set; }
    
    public Customer Customer { get; set; }
    public List<OrderItem> Items { get; set; }
}

Workflow for Multiple Providers

In practice, you maintain separate migration sets:

[[Svg Src="./witdatabase-migrations-structure.svg" Alt="witdatabase-migrations-structure"]]

Or, for simpler scenarios, use EnsureCreated() which creates the schema without migrations:

// For tests and demos where migration history doesn't matter
context.Database.EnsureCreated();

What's Compatible

The goal isn't perfect migration portability — it's code portability. Your DbContext, your entities, your services, your queries — all of that is identical. Only the provider configuration and migration files differ.

Performance Tradeoff

Let's be honest about performance.

SQLite is written in highly optimized C. It's been tuned for 20+ years. WitDatabase is written in C#. Managed code with garbage collection cannot match hand-optimized native code in raw throughput.

For bulk operations on large datasets, SQLite will be faster. That's physics, not a bug.

But performance isn't everything.

For the scenarios where WitDatabase shines — plugins, tests, demos, desktop apps — the performance is more than sufficient:

For a plugin storing configuration and user preferences? Plenty fast.

For integration tests running hundreds of queries? Faster than Docker startup.

For a desktop app with thousands of records? Imperceptible to users.

When NOT to Use WitDatabase

• High-throughput production servers (use PostgreSQL, SQL Server)
• Datasets with millions of rows and complex analytics
• Scenarios where native SQLite performance is essential

When WitDatabase is the Right Choice

• Plugin isolation is required
• Native dependencies are problematic
• Test speed matters more than production parity
• Deployment simplicity is valuable
• Cross-platform consistency is needed

The tradeoff is clear: you give up some raw performance in exchange for deployment flexibility and architectural simplicity. For many scenarios, that's an excellent trade.

The Gap That Existed

Before WitDatabase, here was the landscape for .NET embedded databases:

The EF Core In-Memory provider deserves a mention — it's pure .NET and works in plugins. But it's not a real database. It doesn't persist data. It doesn't support all SQL operations. It's designed for testing simple scenarios, not as a real storage engine.

WitDatabase fills the gap: a pure-managed, relational database with full EF Core support.

Conclusion

I didn't set out to build a database engine. I set out to solve a problem.

The problem was simple: I needed an embedded database that worked in plugin isolation with Entity Framework Core. The existing options didn't work — SQLite because of native dependencies, LiteDB because of the document model.

WitDatabase exists because that gap existed.

If you're building plugin architectures, if you're tired of managing database containers in CI, if you want portable demos that just work — this might be what you're looking for.

It's not a replacement for PostgreSQL or SQL Server in production. It's the embedded option that didn't exist before: pure .NET, relational, full EF Core, zero native dependencies.

Get Started

dotnet add package OutWit.Database.EntityFramework

services.AddDbContext<AppDbContext>(options =>
    options.UseWitDb("Data Source=app.witdb"));

That's all it takes. Your existing EF Core code works unchanged.

The source code is on GitHub. Issues and contributions welcome.