Caching

cachingperformancememorytuning

Pages read from disk are kept in memory, so reading the same page again costs nothing. Pages written to are held there too, dirty, until something flushes them. The cache is the reason a working set that fits in memory behaves as though the disk were not involved.

Data Source=app.witdb;Cache=clock;Cache Size=5000
csharp
var db = new WitDatabaseBuilder()
    .WithFilePath("app.witdb")
    .WithBTree()
    .WithClockCache(cacheSize: 5000)
    .Build();

Two policies ship. Both hold the same pages and differ only in which one they throw out when the cache is full.

Clock

The default. Each page carries a bit saying it has been touched recently. When room is needed, a hand sweeps around the pages: a page with the bit set has it cleared and survives, a page without it is evicted. A page has to go untouched for a full sweep to be thrown out.

That gives it a useful property. A query that reads a large table once touches each page once, so those pages arrive with their bit clear and are the first to go, while the pages your application keeps returning to survive the sweep. A scan passes through the cache instead of emptying it.

The implementation is split across shards, each with its own lock and its own hand, so concurrent readers rarely wait for each other. Sixteen at the default cache size, fewer for a small one, since a shard holding one page is no use to anybody.

LRU

Evicts whichever page has gone longest without being touched. Simple, predictable, and a good fit when recent data is the data you want: sessions, the last hour of orders, anything where age is a proxy for interest.

Its weakness is the case Clock handles. One scan of a large table can walk every hot page out of the cache, and the next request that would have hit goes to disk. It is also a single lock rather than sixteen.

Data Source=app.witdb;Cache=lru;Cache Size=5000

Sizing it

Cache Size is a number of pages, defaulting to 1,000. Memory is that times the page size, so the default is about 4 MB and each 1,000 pages after it costs another 4.

Cache size Memory at 4 KB pages
1,000 4 MB
5,000 20 MB
25,000 100 MB
250,000 1 GB

The size that matters is the one that holds your working set: the pages a typical stretch of activity actually touches, which is usually much smaller than the database. A cache large enough to hold it turns most reads into memory reads, and every page beyond it is memory spent on nothing.

Raising the cache is also the cheapest thing to try when an encrypted database feels slow, since a page held in memory is decrypted once rather than on every read.

Dirty pages

A page modified in memory is dirty until it reaches the disk, which happens when a transaction commits, when the page is evicted, when the database closes, or when you ask:

csharp
db.Flush();
await db.FlushAsync();

A page currently checked out by a caller cannot be evicted, so a cache full of pinned pages has nowhere to put a new one. In ordinary use nothing stays pinned for long.

What the cache will tell you

An open database can report how full its cache is: pages held, and how many of those are dirty.

That is occupancy, and it is a reading taken at the moment you ask, so anything displaying it needs to say when it was taken or offer to take it again.

There is no hit rate. Neither cache counts hits or misses, so the number is absent from the engine rather than merely unexposed, and no amount of wiring at a higher level would produce one. Sizing the cache is therefore done by measuring what you care about, which is query time, rather than by watching a ratio.

The LSM store is not paged and has no page cache to ask about.

Writing your own

IPageCache is a provider like the rest, so a cache with a policy suited to your access pattern, or one backed by something other than local memory, is a class rather than a fork. The async members have default implementations calling their synchronous counterparts, which is enough unless your storage is async-only, as the browser's is.

See architecture for registering one.

Where to go next