Now processing 1B+ events daily · Free Preview available

The modern product
experimentation platform

Sequential testing, CUPED variance reduction, multi-armed bandits, and AI-powered experiment design — all completely free.

Java, Python, React, JS, Go, iOS, Android, Ruby, PHP, .NET, Elixir & Edge SDKs  ·  SOC 2 compliant  ·  Enterprise SSO  ·  Deploy to your AWS

Free Preview · All features included · Self-hosted on your AWS

1B+
Events daily
<50ms
P99 latency
99.99%
Uptime SLA
20–40%
Sample size reduction via CUPED
14
SDK languages (Python, JS, Java, React, Go, iOS, Android, Flutter, React Native, Edge, Ruby, PHP, .NET, Elixir)
SOC 2
SOC 2 / ISO 27001 ready

Everything you need to experiment

From feature flags to enterprise-grade statistical methods — all included

Core Platform

A/B Testing

Run multivariate experiments with statistical rigor. Built-in significance testing, confidence intervals, and automated SHIP / KEEP / CONTINUE recommendations.

Feature Flags

Deploy safely with gradual rollouts, staged schedules, and kill switches. Bulk toggle, SSE audit stream, and full change history included.

Real-time Analytics

See results as they happen. Trend charts, cumulative and daily views, sample size meters, and days-to-significance estimates. Live experiment results stream in real time via WebSocket — LiveResultsPanel with a pulsing “LIVE” indicator pushes z-test p-values, relative lift, and significance updates continuously as data arrives.

Safety Monitoring

Automated guardrails monitor error rates and performance. Auto-rollback on anomaly detection protects your users from bad releases.

Advanced Targeting

20+ targeting operators including semantic versioning, geo-distance, time windows, and JSON path. Rules engine evaluates 125K+ ops/sec.

Developer First

SDKs for Python, JavaScript, Java, React, Go, iOS (Swift), and Android (Kotlin). RESTful APIs with OpenAPI docs. RBAC with custom roles — ADMIN, DEVELOPER, ANALYST, VIEWER.

Advanced Statistical Methods

Sequential Testing

mSPRT-based early stopping with always-valid confidence intervals. Stop experiments early when you have evidence, without inflating false positive rates. Alpha spending via O'Brien-Fleming and Pocock boundaries.

CUPED Variance Reduction

Reach significance 20–40% faster using pre-experiment covariate adjustment (OLS θ). Winsorization handles outliers. Works for both conversion and numeric metrics.

Multi-Armed Bandit

Maximize conversions during experiments. Choose Thompson Sampling, UCB1, or Epsilon-Greedy. Background scheduler auto-reallocates traffic to winning variants.

Interaction Detection

Detect cross-experiment interference before it corrupts your results. Jaccard overlap analysis, chi-squared interaction tests, novelty effect detection, and SUTVA violation alerts.

AI Experiment Design

Claude API–powered hypothesis generation and design suggestions. MCP Server integrates with Claude Code, Cursor, and other coding assistants so your AI tools can design and interpret experiments.

Pre-experiment Power Analysis

Answer “How long do I need to run this experiment?” before you launch. Compute required sample size, MDE, and runtime estimates using the Fleiss z-test formula. Bonferroni correction for multi-variant tests. Interactive power curve chart and AI planning advice via Claude. Try the calculator →

Warehouse-Native Analytics

Query experiment results directly in Snowflake, BigQuery, Redshift, Databricks, ClickHouse, or MySQL. SQL-sanitized sync keeps your warehouse as the source of truth.

LLM / AI Model Evaluation

OSS exclusive

LLM/AI Model Evaluation

Compare prompt versions, model variants (GPT-4o vs Claude vs Gemini), agent configs, and system prompts against real business metrics. Consistent-hash assignment ensures reproducibility. Built-in cost estimation, latency tracking, human rating collection, and LLM-as-judge automated scoring. The only OSS experimentation platform with native LLM experiment support.

GPT-4oClaude 3.5 SonnetGemini 1.5 ProMistralLocal / Ollama

LLM-as-Judge Scoring

  • Use Claude to automatically score responses 0–1
  • Custom criteria: helpfulness, accuracy, safety, empathy
  • Welch t-test p-values and Cohen's d effect size
  • 95% confidence intervals on all metrics
  • Per-token cost breakdown by provider and model

Traffic Management & Governance

Mutual Exclusion Groups

Prevent experiment contamination using consistent-hashing traffic partitioning. Global holdout groups measure the cumulative impact of your entire experimentation program.

Dimensional Analysis

Break down results by device, country, plan, or any user attribute. Bonferroni-corrected per-segment analysis with heterogeneous treatment effect (HTE) detection.

Audit Logging & Compliance

HMAC-SHA256 signed tamper-proof audit trail with real-time SSE stream. SOC 2 Type II & ISO 27001 compliance reports with CSV/JSON export. Custom RBAC roles and effective permissions resolution.

HIPAA Compliance

Built for healthcare and HIPAA-covered entities. Fernet AES-128-CBC PHI encryption, 6-year PHIAuditLog retention with complete access tracking, BAA (Business Associate Agreement) management with expiry tracking, data residency controls per organization, and 11 dedicated HIPAA compliance API endpoints.

SDK Ecosystem & Integrations

Java SDK

Native Java integration with Spring Boot auto-configuration starter. OkHttp client, LRU+TTL caching, and MD5-based consistent hash bucketing for stable variant assignment.

React SDK

React hooks and HOC — useFeatureFlag, useExperiment, useMultipleFlags, and withExperimentation HOC. SSR support via ServerClient with no hydration mismatch.

Go

Go SDK

Native Go client for microservices and CLIs. Zero external dependencies, context-aware API, goroutine-safe concurrent evaluation with built-in LRU caching.

iOS Swift SDK

Native Swift SDK for iOS 14+ and macOS 11+. Modern async/await API, offline fallback with local cache, and SwiftUI-ready feature flag bindings.

🤖

Android Kotlin SDK

Native Kotlin SDK for Android (minSdk 21). Coroutines-first API, OkHttp client, offline persistence, and Jetpack Compose usage examples included.

Edge SDK

Edge-native evaluation for Cloudflare Workers, Vercel Edge, and Deno Deploy. Sub-millisecond flag decisions with pure-JS MD5 consistent hashing — zero Node.js dependencies, zero cold starts.

💎

Ruby SDK

Native Ruby gem with zero runtime dependencies. Thread-safe Mutex TTL cache, Net::HTTP client, and consistent MD5 hash bucketing.

🐘

PHP SDK

Composer package with ext-json and ext-curl only. PSR-compatible, consistent MD5 hash, configurable TTL cache.

🔷

.NET SDK

netstandard2.1 + net6.0 multi-target. System.Text.Json, HttpClient wrapper, LRU cache with TTL, and xUnit-tested consistent hashing.

⚗️

Elixir SDK

Hex package using :httpc and Jason. GenServer-backed ETS cache for zero-allocation lookups, fully OTP-compatible, 93 ExUnit tests.

Third-party Integrations

Jira, Salesforce, and GitHub webhooks — sync experiment status, automatically create issues, and push results to your existing toolchain without leaving the platform.

Full Bayesian Statistics

Beta-Binomial posteriors, Monte Carlo PtBB simulations, Bayes Factor (BF10) via Savage-Dickey, credible intervals, and an automatic Bayesian stopping rule — alongside your frequentist analysis.

Split URL Testing

Server-side URL splitting via Lambda@Edge — no client-side JavaScript, works with any frontend framework. Persistent cookie-based assignment ensures a consistent experience across visits.

No-Code Experiment Builder

5-step guided wizard for experiment creation — define hypotheses, set variants, choose metrics, configure targeting, and launch — all without writing a line of code.

Enterprise SSO

SAML 2.0 and OIDC/OAuth2 for Okta, Azure AD, Google Workspace, GitHub, and OneLogin. Centralized identity management with role mapping from your IdP groups.

Multi-tenant Workspaces

Isolate experiments and feature flags per team, project, or product area. Each workspace has its own members, scoped API keys, and resource limits. Role hierarchy: OWNER, ADMIN, DEVELOPER, ANALYST, VIEWER.

Works with your stack

Seamlessly integrate with the tools you already use

AWS
Cloud Infrastructure
Snowflake
Data Warehouse
BigQuery
Data Warehouse
Redshift
Data Warehouse
Segment
CDP Integration
Slack
Notifications
SendGrid
Email Alerts
Claude
AI Design via MCP
Jira
Issue Sync
Salesforce
CRM Integration
GitHub
Webhook & Issues

Simple to integrate

Get started in minutes — SDKs for Python, JavaScript, Java, React, Go, iOS, Android, Ruby, PHP, .NET, and Elixir

JavaScriptPythonJavaReactGoiOS (Swift)Android (Kotlin)RubyPHP.NETElixir
JavaScript
import { Experimently } from '@experimently/sdk';

const client = new Experimently('YOUR_API_KEY');

// Get variant + track conversion
const variant = await client.getVariant(
  'checkout-redesign', userId
);

client.track('purchase_completed', userId, {
  revenue: 99.99
});

// Feature flag with targeting
const isEnabled = await client.isEnabled(
  'new-checkout', userId,
  { country: 'US', plan: 'enterprise' }
);
Java (Spring Boot)
// application.properties
experimently.api-key=YOUR_API_KEY
experimently.cache.ttl-seconds=60

// Auto-configured via Spring starter
@Autowired
ExperimentlyClient client;

String variant = client.getVariant(
    "checkout-redesign", userId
);

boolean enabled = client.isEnabled(
    "new-checkout", userId,
    Map.of("country", "US", "plan", "enterprise")
);
React
import {
  ExperimentlyProvider,
  useFeatureFlag,
  useVariant,
} from '@experimently/react';

// Wrap your app
<ExperimentlyProvider apiKey="YOUR_API_KEY">
  <App />
</ExperimentlyProvider>

// Use in any component
function Checkout() {
  const variant = useVariant('checkout-redesign');
  const isNewUi = useFeatureFlag('new-checkout');

  return isNewUi ? <NewFlow /> : <OldFlow />;
}
Python
from experimently import ExperimentlyClient

client = ExperimentlyClient(api_key="YOUR_API_KEY")

# Get experiment variant
variant = client.get_variant(
    "checkout-redesign", user_id
)

# Track conversion
client.track("purchase_completed", user_id,
    properties={"revenue": 99.99})

# Feature flag with targeting
enabled = client.is_enabled(
    "new-checkout", user_id,
    context={"country": "US", "plan": "enterprise"}
)

How we compare

Enterprise-grade statistical methods — self-hosted, no price tag

FeatureExperimentlyEppo†GrowthBookStatsig*OptimizelyLaunchDarkly
Pre-experiment Power Analysis✅ Free tool⚠️ Basic⚠️ Basic⚠️ Enterprise
A/B / Multivariate Testing⚠️ Add-on
Feature Flags
Sequential Testing (mSPRT)✅ (default)
CUPED Variance Reduction✅ (binary + numeric)✅ CUPED++⚠️ Numeric only⚠️ Add-on
Multi-Armed Bandit✅ (3 algorithms)✅ AI bandits
Full Bayesian (BF10 + Monte Carlo)⚠️ Partial
Interaction Detection✅ (SUTVA, Jaccard)⚠️ Partial⚠️ Partial
Mutual Exclusion Groups⚠️⚠️ Partial
Global Holdout Group⚠️⚠️
Dimensional Analysis + HTE✅ + Bonferroni⚠️ Partial
Warehouse-Native Analytics✅ (6 connectors)✅ + Databricks✅ + ClickHouse
Split URL Testing (Lambda@Edge)
LLM/AI Experiment Support✅ Native (EP-046)⚠️ Beta
AI Experiment Design✅ Claude + MCP✅ LLM eval✅ MCP (beta)✅ Opal AI
Enterprise SSO (free tier)✅ 🎉❌ Enterprise❌ Enterprise⚠️ Enterprise⚠️ Enterprise⚠️ Enterprise
HIPAA Compliance (PHI encryption + BAA)
Java SDK✅ Spring Boot
Go SDK
iOS + Android + Ruby + PHP + .NET + Elixir SDKs✅ + RN + Flutter✅ + Flutter + Roku
Self-host in your AWS account✅ CDK❌ SaaS only✅ Docker
Audit Log + SSE Stream✅ HMAC-signed⚠️⚠️
Jira / Salesforce / GitHub⚠️⚠️⚠️ Partial
Team Workspaces / Project Isolation✅ Native (EP-057)⚠️ Org-level only✅ Organizations✅ Projects⚠️ Enterprise
Vendor independence✅ Independent⚠️ Acquired by Datadog✅ Series A⚠️ Acquired by OpenAI⚠️ PE-backed
Pricing (entry)🎉 Free Preview~$42K/yr avg$0 → $40/user/moFree → usage$36K+/year$0 → $20K+/yr

* Statsig acquired by OpenAI ($1.1B, Sept 2025). † Eppo acquired by Datadog (~$220M, May 2025). Future roadmaps subject to change. ⚠️ = Partial or enterprise-only.

Free Preview

Every feature is available — self-host on your own AWS account at no cost.

$0/month

Self-hosted on your AWS account — you own your data

Unlimited A/B experiments
Feature flags with staged rollouts
Sequential testing (mSPRT)
CUPED variance reduction
Multi-armed bandit (3 algorithms)
Full Bayesian (BF10, Monte Carlo)
Split URL testing via Lambda@Edge
Interaction detection
AI experiment design (Claude API)
Warehouse analytics (Snowflake, BigQuery, Redshift, Databricks, ClickHouse, MySQL)
Java SDK + Spring Boot starter
React SDK (hooks, HOC, SSR)
Go SDK (zero deps, goroutine-safe)
iOS Swift SDK (async/await, offline)
Android Kotlin SDK (Coroutines, Compose)
Ruby SDK (zero deps, net/http, thread-safe)
PHP SDK (composer, ext-curl only)
.NET SDK (netstandard2.1+net6.0)
Elixir SDK (GenServer+ETS cache, OTP)
Enterprise SSO (SAML 2.0 + OIDC)
Jira, Salesforce & GitHub integrations
SOC 2 / ISO 27001 compliance reports
Full audit log + custom RBAC roles
Slack + email alerting
Multi-tenant workspaces (OWNER / ADMIN / DEVELOPER / ANALYST / VIEWER)
Get started — deploy to your AWS

Start experimenting today

Sequential testing, CUPED, Bayesian stats, Java/React/JS/Go/iOS/Android SDKs, Enterprise SSO, and AI design — all completely free. Deploy to your own AWS.

Free Preview · Self-hosted · All features included