In-memory Umbraco Search

“Can Umbraco Search run purely in memory?”

I’ve had this question a few times from people who are currently using in-memory indexes for their Examine search.

The answer? “Yes, but…” 😄

In-memory Examine indexes

It is entirely possible to use in-memory Examine indexes, as outlined in Shannon’s blog post from 2023.

This might be a (temporary) solution if you’re suffering from index data corruption. However, it comes with more than a few back-draws, including:

• Increased memory footprint.
• Prolonged start-up times for site search.

In-memory Umbraco Search

Since Umbraco Search uses Examine by default, it is entirely possible to run Umbraco Search purely in memory, and it doesn’t even require a lot of code 👏

The trick is to reconfigure the default Examine directory factory, so Examine creates in-memory indexes rather than physical ones for the relevant indexes:

public static class MyUmbracoBuilderExtensions
{
    public static IUmbracoBuilder UseInMemoryExamine(this IUmbracoBuilder builder)
    {
        // Use in-memory directories for all the default search indexes
        foreach (var indexAlias in new[]
                 {
                     Constants.IndexAliases.DraftContent,
                     Constants.IndexAliases.DraftMedia,
                     Constants.IndexAliases.DraftMembers,
                     Constants.IndexAliases.PublishedContent
                 })
        {
            builder
                .Services
                .Configure<LuceneDirectoryIndexOptions>(
                    indexAlias,
                    options => options.DirectoryFactory = new InMemoryDirectoryFactory());
        }

        return builder;
    }
}

…where InMemoryDirectoryFactory is a directory factory which creates RAM directories:

internal class InMemoryDirectoryFactory : DirectoryFactoryBase
{
    private RandomIdRAMDirectory? _randomIdRamDirectory;

    protected override Directory CreateDirectory(LuceneIndex luceneIndex, bool forceUnlock)
    {
        _randomIdRamDirectory = new RandomIdRAMDirectory();
        return _randomIdRamDirectory;
    }

    protected override void Dispose(bool disposing)
    {
        base.Dispose(disposing);
        _randomIdRamDirectory?.Dispose();
    }

    // RAM directory with per-instance unique lock ID 
    private class RandomIdRAMDirectory : RAMDirectory
    {
        private readonly string _lockId = Guid.NewGuid().ToString();

        public override string GetLockID() => _lockId;
    }
}

The UseInMemoryExamine() extension method must be invoked after adding the Examine search provider, otherwise the reconfiguration will not have any effect:

public class SiteComposer : IComposer
{
    public void Compose(IUmbracoBuilder builder)
    {
        builder
            // add core services for search abstractions
            .AddSearchCore()
            // use the Examine search provider
            .AddExamineSearchProvider()
            // configure Examine to use in-memory indexes 
            .UseInMemoryExamine();

        // ...
    }
}

In this GitHub repo you’ll find a demo site which runs Umbraco Search in memory. It’s an upgraded version of a site from one of my previous blog posts. As always, the Umbraco admin login is:

• Username: admin@localhost
• Password: SuperSecret123

…but?

So, it is entirely possible to run Umbraco Search purely in memory. But should you?

Probably not.

While Umbraco Search is greatly optimized for reindexing, it still takes a toll on the database at start-up. And if you’re load balancing, this is even multiplied by the number of running instances 😱

Also, you still face the issue of memory consumption when having the search indexes in memory.

In other words, use this with caution - and exclusively on small sites with limited content 🤏

If you’re trying to get rid of Examine indexes on disk, consider either ExamineX for managed, out-of-process Examine search, or try your hand with one of my search providers for Elasticsearch, Typesense or Algolia

Happy (in-memory) searching 💜