Type I and Type II errors (in the FRM)
發布時間：2010-01-19 共1頁
Type I and Type II errors (in the FRM)Hypothesis testing consist of seven generic steps (paraphrased from Daniel & Terrell, 1986):
It can be difficult to memorize which error is worse, between Type I and Type II errors. The reason is that no hard rule applies to the formulation of the null (denote H0) and alternative hypothesis (denote Ha).The null is not necessarily the "good" or "bad" outcome. The nullhypothesis is the hypothesis to be tested; the alternative is thehypothesis to be accepted when the null is rejected. (please note asubtlety that I won’t impose on the discussion below: we technically donot accept the null, but rather, we fail to reject the null).
The two types of mutually exclusive errors than can be committed are a Type I (i.e., reject a true null hypothesis) and a Type II (accept a false null hypothesis):
Common applications of hypothesis testing, as they show up in the Quantitative section of the FRM,are tests of significance of sample means and regression parameters;e.g., test of whether the slope parameter is positive, test of whetherthe coefficient of determination (r2) is significant. Butalso common is the simple test of a sample statistic. For example,assume we calculate a sample mean as zero. We want to know whether thisoutcome reflects a population mean that is also zero. In this case, thenull hypothesis could be "the population mean is zero" and thealternative could be "the population mean is positive." Under thisparticular framing of the null hypothesis, the errors look like thefollowing:
Nowmove to the credit section, which has been a source of confusion. Underthe Neyman-Pearson Decision Rule, de Servigny clearly implies that thenull hypothesis is "the firm will default:" i.e., this is a bad firm.This particular framing of the null is confusing because we areaccustomed to thinking of Type II errors as the worse types. But seebelow how it is really just a matter of the definition of the null:
Finally,move to Basel II in operational risk and consider the so-called trafficlight system of model backtesting. For this situation, the Baselframers defined the null as "the risk model is accurate." As defined,you can see why we are back to a situation where the Type II error isarguably worse (i.e., we mistakenly accept a faulty model):
- State hypothesis
- Identify probability distribution and appropriate (corresponding) test statistic
- Specifysignificance level (one minus confidence level; i.e., for value atrisk, typically 5%/95% significance/confidence or 1%/99%significance/confidence)
- State decision rule
- Calculate test statistic
- Make statistical decision
- Make economic or investment decision
It can be difficult to memorize which error is worse, between Type I and Type II errors. The reason is that no hard rule applies to the formulation of the null (denote H0) and alternative hypothesis (denote Ha).The null is not necessarily the "good" or "bad" outcome. The nullhypothesis is the hypothesis to be tested; the alternative is thehypothesis to be accepted when the null is rejected. (please note asubtlety that I won’t impose on the discussion below: we technically donot accept the null, but rather, we fail to reject the null).
The two types of mutually exclusive errors than can be committed are a Type I (i.e., reject a true null hypothesis) and a Type II (accept a false null hypothesis):
Common applications of hypothesis testing, as they show up in the Quantitative section of the FRM,are tests of significance of sample means and regression parameters;e.g., test of whether the slope parameter is positive, test of whetherthe coefficient of determination (r2) is significant. Butalso common is the simple test of a sample statistic. For example,assume we calculate a sample mean as zero. We want to know whether thisoutcome reflects a population mean that is also zero. In this case, thenull hypothesis could be "the population mean is zero" and thealternative could be "the population mean is positive." Under thisparticular framing of the null hypothesis, the errors look like thefollowing:
Nowmove to the credit section, which has been a source of confusion. Underthe Neyman-Pearson Decision Rule, de Servigny clearly implies that thenull hypothesis is "the firm will default:" i.e., this is a bad firm.This particular framing of the null is confusing because we areaccustomed to thinking of Type II errors as the worse types. But seebelow how it is really just a matter of the definition of the null:
Finally,move to Basel II in operational risk and consider the so-called trafficlight system of model backtesting. For this situation, the Baselframers defined the null as "the risk model is accurate." As defined,you can see why we are back to a situation where the Type II error isarguably worse (i.e., we mistakenly accept a faulty model):
