Forum Archive :
Rollouts
The confidence interval is a range. For example if you see "the 95%
confidence interval is .72 +- .03" the confidence interval is [.69, .75].
Requiring that the 95% CI levels do not overlap is a very stringent
condition.
Roughly, to find the margin of error for the difference you should add the
two +- values and divide by SQRT(2), approx 1.41. (This works if the +-
values are about equal, which will usually be the case if the plays were
rolled out the same number of times.)
As an example:
Play A 95% CI is .70 +- .10
Play B 95% CI is .55 +- .10
The 95% CI for the difference is .15 +- .141 = [.009, .291] This does not
include 0 so you have greater than 95% confidence (barely) that play A is
better.
The +- values have a term "margin of error" and for a 95% confidence
interval are calculated as approximately 1.96 * std error. The 1.96 value
comes from the normal distribution and is chosen so that P( -1.96 < Z <
1.96) = .95 where Z is a normally distributed variable with mean 0 and std.
dev 1.
Working backwards here we can determine that the std errors for A and B are
.10/1.96 = .05102
The std error for the difference is (.05102+.05102)/SQRT(2) = .07215 The
difference of .15 is that value times 2.08. Using a normal table (or
software) P(-2.08 < Z < 2.08) = .9624 so there is 96.24% confidence in my
example that play A is better than play B.
It is left as an exercise what would be the confidence if the plays were
actually .2 (the sum of the 95% confidence interval margins of error)
apart.
|
|
|
|
Rollouts
- Advice (David Montgomery, Apr 1996)
- Cautionary tale (Kit Woolsey, Sept 1995)
- Combining rollouts (Gregg Cattanach+, Dec 2003)
- Confidence intervals (Bob Koca, Nov 2010)
- Confidence intervals (Timothy Chow, May 2010)
- Confidence intervals (Gerry Tesauro, Feb 1994)
- Cubeless vs centered-cube rollouts (Ron Karr, Dec 1997)
- Duplicate dice (David Montgomery, June 1998)
- How reliable are rollouts? (David Montgomery, Aug 1999)
- Level-5 versus level-6 rollouts (Michael J. Zehr, June 1998)
- Level-5 versus level-6 rollouts (Chuck Bower, Aug 1997)
- Positions with inaccurate rollouts (Douglas Zare, Oct 2002)
- Reporting results of rollouts (David Montgomery, June 1995)
- Rollout settings (Lokicol+, Apr 2010)
- Settlement limit (Michael J. Zehr, Apr 1998)
- Settlement limit (Kit Woolsey, Dec 1997)
- Settlement limit in races (Alexander Nitschke, Dec 1997)
- Some guidelines (Kit Woolsey, Apr 1996)
- Standard error and JSD (rambiz+, Feb 2011)
- Standard error and JSD (Stick+, Oct 2007)
- Systematic error (Chuck Bower, Oct 1996)
- Tips for doing rollouts (Douglas Zare, June 2002)
- Truncated rollouts (Gregg Cattanach, Oct 2002)
- Truncated rollouts: pros and cons (Jason Lee+, Jan 2006)
- What is a rollout? (Gregg Cattanach, Dec 1999)
From GammOnLine
Long message
Recommended reading
Recent addition
|
| |
|