Posts by Tags

bandits

Confidence Intervals for Bandit Policies: A BSI Demo

less than 1 minute read

Published:

After running a bandit algorithm, how well would a different policy have done? Bandit Simulation for Inference (BSI) fits a simulator of the environment from logged bandit data and uses it to build a confidence interval for the mean reward of any policy, including adaptive ones like Thompson sampling. This demo walks through the workflow: upload logged data, choose the policy to evaluate and the horizon, and see the checks that flag when the interval may not be reliable. The interval it reports is an illustration, not the output of the BSI code.

bayesian optimization

Choose the Length Scale: Gaussian Process Regression

less than 1 minute read

Published:

A Gaussian process sees five observations of a hidden function, and you have to choose its length scale. Can you pick one whose 95% band covers the function, without making the band uselessly wide? Some of the functions hide surprises between the observations.

Bake the Best Cake: A Sequential Decision Game

less than 1 minute read

Published:

You have 20 cakes to bake, and each recipe is a choice of sugar and oven temperature. Every cake you bake shows you how good that recipe tastes, so you have to decide when to explore new recipes and when to stay near the best one so far. This is the exploration–exploitation trade-off at the heart of bandit algorithms and Bayesian optimization.

gaussian processes

Choose the Length Scale: Gaussian Process Regression

less than 1 minute read

Published:

A Gaussian process sees five observations of a hidden function, and you have to choose its length scale. Can you pick one whose 95% band covers the function, without making the band uselessly wide? Some of the functions hide surprises between the observations.

interactive

Choose the Length Scale: Gaussian Process Regression

less than 1 minute read

Published:

A Gaussian process sees five observations of a hidden function, and you have to choose its length scale. Can you pick one whose 95% band covers the function, without making the band uselessly wide? Some of the functions hide surprises between the observations.

Confidence Intervals for Bandit Policies: A BSI Demo

less than 1 minute read

Published:

After running a bandit algorithm, how well would a different policy have done? Bandit Simulation for Inference (BSI) fits a simulator of the environment from logged bandit data and uses it to build a confidence interval for the mean reward of any policy, including adaptive ones like Thompson sampling. This demo walks through the workflow: upload logged data, choose the policy to evaluate and the horizon, and see the checks that flag when the interval may not be reliable. The interval it reports is an illustration, not the output of the BSI code.

Exploring the Bivariate Normal Distribution

less than 1 minute read

Published:

How does the variance–covariance matrix shape a bivariate normal distribution? Type in your own matrix and see the density of \((X_1, X_2)\) as a 3D surface you can rotate. Try making the covariance close to its limit to watch the distribution collapse onto a line.

Bake the Best Cake: A Sequential Decision Game

less than 1 minute read

Published:

You have 20 cakes to bake, and each recipe is a choice of sugar and oven temperature. Every cake you bake shows you how good that recipe tastes, so you have to decide when to explore new recipes and when to stay near the best one so far. This is the exploration–exploitation trade-off at the heart of bandit algorithms and Bayesian optimization.

probability

Exploring the Bivariate Normal Distribution

less than 1 minute read

Published:

How does the variance–covariance matrix shape a bivariate normal distribution? Type in your own matrix and see the density of \((X_1, X_2)\) as a 3D surface you can rotate. Try making the covariance close to its limit to watch the distribution collapse onto a line.

sequential decision making

Confidence Intervals for Bandit Policies: A BSI Demo

less than 1 minute read

Published:

After running a bandit algorithm, how well would a different policy have done? Bandit Simulation for Inference (BSI) fits a simulator of the environment from logged bandit data and uses it to build a confidence interval for the mean reward of any policy, including adaptive ones like Thompson sampling. This demo walks through the workflow: upload logged data, choose the policy to evaluate and the horizon, and see the checks that flag when the interval may not be reliable. The interval it reports is an illustration, not the output of the BSI code.

Bake the Best Cake: A Sequential Decision Game

less than 1 minute read

Published:

You have 20 cakes to bake, and each recipe is a choice of sugar and oven temperature. Every cake you bake shows you how good that recipe tastes, so you have to decide when to explore new recipes and when to stay near the best one so far. This is the exploration–exploitation trade-off at the heart of bandit algorithms and Bayesian optimization.