BSI: Bandit Simulation for Average Reward Inference

Upload logged bandit data, choose the policy you want to evaluate, and get a confidence interval for its mean reward over a horizon of T rounds. BSI fits a simulator of the environment from the logs, rolls the evaluation policy out in it, and carries the simulator's estimation error into the interval.

1

Problem

2

Logged data

CSV, one row per round
or drop it here

3

Context distribution

4

Policy to evaluate

5

Monte Carlo rollouts M

How M is chosen

  1. Roll the policy out m₀ times in the simulator fitted from your logs.
  2. Resample those pilot rollouts B times with replacement, and compute the interval width from each resample.
  3. Measure how much the width varies: ρ̂ = m₀ · Var(width) / mean(width)².
  4. Set M = max(m₀, ⌈z²1−τ/2 · ρ̂ / ε²⌉). With M rollouts, the Monte Carlo error changes the interval width by less than a fraction ε, with probability 1 − τ.
  5. If that M is above your cap, BSI uses the cap and warns you that the Monte Carlo error may be larger than ε.
Advanced settings
6

Horizon and confidence level