Skip to main content

​
Prerequisites

  • Bun runtime installed
  • API keys for providers and LLM judges you want to use

​
Install MemoryBench

git clone https://github.com/supermemoryai/memorybench
cd memorybench
bun install

​
Configure API Keys

Create a .env.local file in the root directory: 
# Memory Providers (add keys for providers you want to test)
SUPERMEMORY_API_KEY=your_key
MEM0_API_KEY=your_key
ZEP_API_KEY=your_key

# LLM Judges (at least one required)
OPENAI_API_KEY=your_key
ANTHROPIC_API_KEY=your_key
GOOGLE_API_KEY=your_key
You only need API keys for the providers and judges you plan to use. For example, to benchmark Supermemory with GPT-4o as judge, you only need SUPERMEMORY_API_KEY and OPENAI_API_KEY.

​
Verify Installation

bun run src/index.ts help
You should see the list of available commands.

​
Start the Web Interface

bun run src/index.ts serve
Opens at http://localhost:3000.

​
Next Steps