A casino has four slot machines, each with an unknown payoff distribution. You have a limited number of pulls. Which machine do you play, and when do you stop exploring? Tune the machines and the strategy parameters below, then watch four classic algorithms compete on reward and regret.
Pure greedy tries each machine once, then always plays the current best average — cheap, but easily fooled by early luck. ε-greedy plays greedy but explores a random machine with probability ε. UCB picks by an optimistic upper confidence bound, exploring machines it is unsure about. Thompson sampling draws a plausible value for each machine from its posterior and plays the winner. Regret is the cumulative payoff lost versus always playing the true best machine — the headline "cost of indecision" number.