Probing the Nature of Inferential Decisions: Fine-Tuning the False Negative Error [β-Risk%]

In our experiential-milieu, the False Positive Error [FPE] is the ubiquitous choice used to profile and understand the results of inferential analyses. Rarely, are the False Negative Error [FNE] and its ancestral-derivative: The β-Risk% invited into the Analytical conversation to offer an enrichment of the scope of the inferential-intel used to inform the Decision-Making process. This seems to be the case because usually the FPE-intel is generated from a p-value that effectively is the only inference-intel used by the Analyst. The other FPE-inference-intel derives from Sir R. A. Fisher who suggested that adequate statistical-intel is best created by fixing an a prior specified FPE[α] that marks a Point along the Probability-abscissa forming a binary-partition: A H_o− Non-rejection-zone & H_a− acceptance-zone. This binary-partition invites a What-If-conjecture called the β-Risk% that the p-value inferential-model does not “naturally” facilitate. In practice, we have noticed that the conjectural-feature of creating the β-Risk% has resulted in confusion and invites Gaming of the β-Risk%-intel. We offer a β-Risk%-protocol that, if followed, will enhance the overall decision-impact by partnering the FPE with the FNE. Additionally, in addressing computing the β-Risk%, we offer a discussion of two probability contexts: (i) the population standard deviation σPop is discernable, and (ii) the σPop may be computed. In addition, to facilitate using these ideas in creating inferential-intel, we have programmed these two β-Risk% contexts as stand-alone VBA-Excel Open-access Platforms.

The Other Side of the Inference Coin: The False Negative Error

Statistical Decision-Making is Risky Business

Every decision to take an action is based on a Random Sample from an assumed Population where the reliability of the inferential profile is dependent on many conditions. Thus, there is a “Caveat Emptor-ish” aspect to making decisions based upon inferential profiles. Specifically, you may be: Well, trained in Math/Stat, Very organized, Carful—boarding on obsessive, Selective in choosing the correct Statistical Models, Attentive in correctly applying them. and Fastidious in clearly presented the results—but –alas!!! there is an unavoidable non-trivial chance that your analysis leads to taking the wrong course of action!

The Ubiquitous Errors

There are two such Errors that are unavoidable and “semi”-independent. These are labeled as: A False Positive Error [FPE] or an α-Error or Type I Error; the other is called: A False Negative Error [FNE] or a β-Error or a Type II Error. These are practically defined as:

• A FPE-Risk[α%] occurs when the Statistical Profile that was created indicates that there IS likely an intervention-effect—i.e., a “positive” result relative to the Null—when, in fact, there is NO actual effect/result. This happens overall α% of the time and so then logically the result is TRUE [1 −α]% of the time—this is a frequentist-interpretation and so is an approximation. Also, the post-experiment computed—i.e., actual FPE-Risk[α%] is never = zero.

• A FNE-Risk[β%] occurs when the Statistical Profile that was created indicates that there is NOT likely an effect—i.e., the Null or NO-result is the likely State of Nature—when, in fact, there is an actual intervention effect/result. This happens overall β% of the time and so then logically the result is TRUE[1 −β%] of the time. This, also, is a frequentist-interpretation and so is an approximation. Also, the post-experiment computed—i.e., actual FNE-Risk[β%] is never = zero.

Over the years, we have struggled mightily to convey this critical inferential diagnostic FPE- & FNE-information to our students and consultation partners—with marginal success. To take a “positive” action, we decided to create this tutorial that is accompanied by a VBA interactive decision-platform—We reasoned that most of the students in the AI-eEverything World would enjoy this sort of inferential “Video-Game” and actually learn the Error-issues in Data Analytics.

Following, we will offer a discussion only of the False Negative Error or the β-Error or Type II Error as this seems to present the most difficulty to the students.

The False Negative β%-Error: A set of Decision Analytic Platforms in VBA

Overview

In what follows, we will consider two versions of profiling the FNE:

1. Assuming that the population is discernable, after some vetting tests, the FNE can be inferentially profiled, and

2. Assuming that the construction of the population, after some vetting tests, the FNE can be inferentially profiled.

The Population is Discernable

In this case, the analyst has access to past records where: (i) their recording protocols have not been modified over time to any material extent, (ii) this past data is longitudinally extensive, (iii) random samples can be taken, (iv) these datasets have been used successfully in previous studies and been vetted as inferentially reliable, and (v) the data is relevant to the questions of inferential interest. These conditions were extracted from our course text by Tamhane and Dunlop (2000, p. 211–222).

Testing the Effect of a Modification of the Diet of a Herd of Livestock

Here we are referencing a version of the example in Ott (1992, p. 221–225) used in our Statistical Decision-making course. Assume there is a herd [a Population of Animals] of Livestock and the Analyst is testing a NEW Feed Mix [NFMix] that is expected to increase the weight of the Livestock in the herd.

There is a Pre-Intervention Phase where the weights of the Livestock of the Population Herd were measured using a Walk-Over Scale. This was done over a two-week period and the weights of the Livestock were electronically recorded. Specifically, all the animals in the Herd [including the RFI-tagged animals] were weighed on three occasions: (i) The Monday of the week before the Intervention [NFMix], (ii) Randomly, over the Tuesday through the Saturday of the week before the Intervention [NFMix], and (iii) the Sunday before the Intervention [NFMix]. The weighted-average of these three weights was computed and used as the Base-Line Weight for the Herd [including the RFI-tagged animals]. Then, on Monday, the Intervention [NFMix] was activated and continued for three months.

A Dietary Intervention was undertaken where 30 randomly selected animals were RFI-Tagged and this group was to receive the Intervention[NFMix]-diet—the intention which was to increase their weight relative to the final weight of the non-intervention animals. The Pre-Intervention weights of these 30 animals were inferentially tested v. the rest of the Herd and the p-value of the Mean differences was FPE[p-value: = 0.546] indicating no likely difference in the Mean weight of the two groups before the Intervention[NFMix], and

The Post Intervention Phase where the ALL the Animals were re-weighed. In this case, the Livestock that did not receive the Intervention[NFMix] were weighed as were the Livestock that received the NFMix Diet.

The Testing Focus

After the basic Nature of the Study is profiled, the next critical question is:

• What is the Nature of the DATA that will form an inferential profile that can be relied upon to be the best intel to inform the decision-makers?

As most all of Sir R.A. Fishers’ inferential creations reference the Gaussian sub-culture, we always test to determine:

•If the Random Sample from the Population of Interest suggests that the Population of Interest, as profiled by the Random Sample, rationalizes using the Normal [μ, σ] Probability Density Function [PDF] to provide the inferential profiles that are used to inform the Decision-Making Process.

We will, thus, offer vetting test-protocols that we have found to be simple and useful in testing the Sample to garner intel regarding the Nature of the Sampled Population. Specifically, we are collecting vetting-intel so as to make a reasoned selection among the following FPE[Null]s:

• $H_a$: The Sample Profile is likely representative of random selections from a Gaussian Normal [μ, σ] Population. This is the Reject $H_o$ alternative assumed State of Nature, where:

• $H_o$: The Sample Profile is likely accrued from a Non-Gaussian Normal [μ, σ] Population.

Consequence of the FPE-Null Action

If the vetting analytics rationalize Rejecting $H_o$; then, the analyst may logically select among the Cornucopia of inferential models in the Gaussian-family; if this not the case, then, the relative sparce number of Non-Gaussian platforms will be needed.

To this end, we have used a very simple and intuitive testing platform to provide information on the reasoned musing needed to select between: accepting or rejecting $H_o$ in the vetting context.

The Vetting Test for the Inferred Nature of the Population

After years of using various testing protocols that offer intel re: Accepting or Rejecting the vetting $H_o$, beginning with the Kolmogorov-Smirnov [K-S] test which is “biased” to Rejecting $H_o$, we have for some time used the [SAS™[JMP™[Normal Quantile Plot]]]. The Normal Quantile Plot, usually referred to as: The Q-Q Profile, is an ordered sample-point-profile [Lowest to Highest] delivered in the two-dimensional CC-Space. For reading acuity, sections of the Q-Q Profile boundary-line are presented on either side of the Median. The Red-Boundary Lines are variable Lilliefors Confidence Tracking Limits [LCTL] of the sample-points; the LCTL are symmetric relative to the Linear Normal Orientation Line (Conover, 1980). The LCTL are essentially the Kolmogorov-Smirnov [K-S]-values used to test for Normality of the sampled Population except the LCTL have been adjusted/corrected for the fact the sample test-points are drawn from a Population where the first two Moments of the distribution are the Sampled Moments from an unknown Population whereas the K-S test assume that the N{μ & σ} are known. Thus, the Q-Q Profile is preferred to the K-S and the many variants for testing for the population Normality by way of the Sampled Profile.

Suggestions on the Use of the Q-Q Profile

Experientially, we have developed the following heuristic for the Q-Q Profile. We count the number of sample-points that are outside the LCTLs. If >15% of the Sampled Points are exterior to the LCTL, then the suggested Action is: Fail to Reject the Q-Q [Null[$H_o$]] that: The Data was randomly sampled from a non-Normal Population.

As a related simple vetting indication, we also use the Ratio Difference between the Mean & the Median—a logical and simple indication of Skewness. Our computation is:

$$\begin{array}{rcl}{R}_{Median}^{Mean}& =& ABS[[Mean-Median]/\\ & & \left[AVERAGE\left[Mean\mathrm{\&}Median\right]\right]]\end{array}$$

Experientially, we have concluded that Skewness can trouble the β-Risk% analysis if the ABS [${\text{R}}_{\text{Medain}}^{\text{Mean}}$] > than 15%. In this vetting context, if the Mean is > [1.075/0.925] × Median, this may signal an Alert for a possible asymmetry that may call into question the actual Nature of the Distribution assumed for the Population. Caveat[Reality Check] These heuristic measures and values may be conditioned on the Nature of Data that we have, almost uniformly, used in our inferential testing—the vast majority have been Market Trading Data—that is often ln-transformed.

The test of normality for the Q-Q profile of the intervention[NFMix Appendix A: n = 30], there were NO points that are outside the LCTL; this Q-Q profile is a strong indication that the intervention[NFMix] data is not drawn from a non-normal population. Thus, our FPE[Null [action]] is reject ${\text{H}}_{\text{o}}$. For the ${\text{R}}_{\text{Median}}^{\text{Mean}}$ vetting measure, where the Mean[428.9667] & Median[429.5] are basically the same; this is a visual vetting-indication of the ${\text{R}}_{\text{Medain}}^{\text{Mean}}$ measure presented following:

$$\begin{array}{r}{R}_{Medain}^{Mean}\left\{0.124\mathrm{\%}\right\}=\end{array}$$

$$\begin{array}{rcl}& & ABS\left[[Mean\left[428.9667\right]-Median\left[429.5\right]]\right]\\ & & /AVERAGE[428.9667;429.5]=\left\{0.124\mathrm{\%}\right\}\end{array}$$

With the Q [15%]–Q [15%] Profile and that of the ${\text{R}}_{\text{Medain}}^{\text{Mean}}$, both of which, suggest rejecting the [${\text{H}}_{\text{o}}$] that: The data was randomly sampled from a non-normal population, the analyst may, we suggest, proceed with analysis assuming that the Population from which the sample was selected was likely Normal[μ,σ]. However, the {μ & σ} are still unknown but they can still be calibrated using sampling and related techniques conditioned on the suggestions in section 2.2. Summary Overview All we are indicating, is that the Nature of the Population is likely Normal and so our calibrations are only a matter of fixing the μ-value and σ-value in this likely Normal Population setting.

The Intervention[NFMix] Testing

Following, are the Inputs to the VBA-platform [Discernable Population]. This platform only considers Directional test expectations where the null is no intervention[NFMix] Effect noted as: $H_o$, and thus, the alternative is: there IS evidence of an effect noted as: $H_a$ where $H_a$ is > $H_o$.

Execution of the Intervention[NFMix] Protocol

The pre-testing intervention population of the entire herd that were fed only the traditional feed mix had a mean [${\text{H}}_{\text{o}}$]: [${\mathrm{\mu }}_{\text{Ho}}$] = 419 lbs.

For the Post-Intervention Event the Actual Mean of the 30 Livestock was [$\overline{\text{x}}$] = 428.9667 lbs. These are only the 30 animals that were randomly selected and fed the NFMix (Table I).

Table I. These are Illustrative Weights of the Herd Animals on the Intervention [NFMix]

402	428	412	450	450	472	472	464	453	414
398	438	452	433	453	414	410	400	431	389
465	396	440	421	436	394	462	389	405	426

Input to the VBA-Platform [Discernable Population]

Following are the VBA-Inputs and the computational aspects of the VBA-platform:

• [$1_{VBA}$] The mean weight of the herd livestock that were fed the traditional feed mix was [$H_o[\mu ]$] = 419.00 lbs.

• [$2_{VBA}$] the mean weight for the 30 livestock fed the NFMix was [${\overline{x}}_{NFMix}$] = 428.9667 lbs. These are the 30 Livestock that were randomly selected and fed only the NFMix [Noted in Appendix A].

• [$3_{VBA}$] sample size [n]: 30 these are the livestock that were fed only the NFMix.

• [$4_{VBA}$] Directional testing false positive error [FPE]-Rate% expectation to rationalize actions based upon the testing profile [${\alpha }_{Decision}$]: 1% [0.01]

• [$5_{VBA}$] Standard deviation of herd-population that only were fed the traditional mix post [${\sigma }_{Pop}$]: 25.8699 lbs computed over all the sample points taken.

The VBA Computation Set

With this INPUT, Table II displays the following profile.

Table II. Results from the VBA Code

Profile to determine the FNE	Sample [n = 30]
95% confidence interval[Mean]	{419.7093: 438.2241}
A priori testing expectation rate%	1.0% & N[0,1] = 2.326385
Actual [428.9667] v. [$H_o$]: 419.00]	N[0,1] = 2.110169 & 1.74%
FNE-action implication	Possible as 1.74% > 1.0%

Discussion

The 95% sampled confidence interval [[95%[$\overline{x}$]CI]] is the best source of possible alternative population estimates to be used if the analyst decides to create False Negative Error[β-Risk%]-intel to enhance the decision-making inferential intel created for the FPE-profile. To rationalize this analytic phase, we offer the following. The 95% CI: What do we learn? Assuming that:

1. The sample is of sufficient size, about 30 observations, are all that are usually needed for the central limit theorem to be in effect for the sampling frame, and

2. The sample was drawn randomly from a defined Population. For our inferential protocol, the nature of the population is likely normal [μ,σ] according to the {Q[15%] Q[15%]}-Profile and the ${\text{R}}_{\text{Medain}}^{\text{Mean}}$ vetting-measure.

3. Assuming that this intel in 2. rationalized reject the vetting FPE[Null[${\text{H}}_{\text{o}}$]], then and only then,

4. The 95% confidence interval of the sampled mean [95%[$\overline{\text{x}}$]CI] is the range-profile of the possible population mean [${\mathrm{\mu }}_{\text{Pop}}$]-values such that 95% of the time the TRUE population mean [${\mathrm{\mu }}_{\text{Pop}}$] is somewhere in that particular [95%[$\overline{\text{x}}$]CI] and, of course, 5% of the time the TRUE population mean [${\mathrm{\mu }}_{\text{Pop}}$] is NOT somewhere in that particular [95%[$\overline{\text{x}}$]CI]. Caveat for logical consistency, the [95%[$\overline{\text{x}}$]CI] was computed using the [${\mathrm{\sigma }}_{\text{Pop}}$]: 25.8699 lbs to form the standard error of the mean.

Thus, in summary, the [95%[$\overline{\text{x}}$]CI] profile of the likely population means [${\mathrm{\mu }}_{\text{Pop}}$] will be valuable-intel in selecting reasonable alternative population mean [${\mathrm{\mu }}_{\text{APop}}$]-values to probe, if necessary, the FNE[β-Risk%] as an enhancement to the FPE[α%]-profile.

The VBA-analysis of Table II the Computational Details

Following, we will provide the computations that are programmed in the VBA[Population Discernable]-platform.

The measured standard deviation of the population 25.8699 [σ] is used to form the standard error of the ${\hat{\sigma }}_{Error}$[Mean], for the following computation of the FPE[$z_{N[0,1]}$]:

$$\begin{array}{r}\left[z_{N[0,1]}\right]=2.110169=\left[\left[428.9667\text{–}419.0\right]/\left[25.8699/\sqrt{30}\right]\right]\end{array}$$

The Logic of Decoding the False Negative Error[FNE]

As the computed FPE[p-value[α-Rate%]] of [${\text{z}}_{\text{N}[0,1]}$[2.110169]] is: 1.7423% and this is greater than the a priori decision-election of a FPE[p-value[α-Rate%]] of: 1.0%—i.e., the desired False Positive Error Rate% cut-point for analysis, but is “sort of close” to 1.0%, it is not unreasonable for the analyst to examine the Nature of the False Negative Error—the β-Risk% before making the decision on the ONLY two choices in the decision-context:

I. Failing to Reject the H_o, thus indicating that 419.0 lbs inferentially remains the likely state of nature weight after the intervention[NFMix], or

II. Reject the H_o, thus accepting H_a— i.e., the Intervention resulted in the state of nature where the population [μ] is >419.0 lbs indicating that the NFMix inferentially resulted in an increase in weight.

The β-Risk% Decision Issue: Needed Clarifications

The β-Risk% Decision-Making issue arises when and only when the sample Mean [[$\overline{x}$] = 428.9667] falls “just” short of the abscissa point-value fixed by decision-maker’s choice of the FPE[p-value[Rate%]] of: 1%. Specifically, the [${\text{z}}_{\text{N}[0,1]}$[α[1%]]] = 2.326385 a value which translates into the following point along the abscissa of: 429.9879 :{419.0 + [2.326385] × [25.8699/$\sqrt{30}$]}. Note that the {Fail to Reject ${\text{H}}_{\text{o}}$ Zone} is <429.9879 i.e., to the LEFT of: 429.9879; and thus, the {Reject ${\text{H}}_{\text{o}}$ Zone} is ≥429.9879 i.e., to the Right of ≥429.9879. As the Mean [[$\overline{\text{x}}$] = 428.9667] is just to the LEFT of 429.9879, it falls into the {Fail to Reject ${\text{H}}_{\text{o}}$ Zone}. Note that this is the same indication that would be produced if we were to have used the p-values. For example, the p-value of {428.9667 v. 419.0} is: 1.7423%% which is not lower that the 1.0% that was the FPE used by the analyst. However, 428.9667 is “Close” to 429.9879; only when this is the case, the decision-maker [DM] usually creates additional FNE[intel] that will be used to make the following binary decision:

• Reject $H_o$ thus Accept $H_a$ that $H_a$ is >$H_o[419.0]$ as the State of Nature, or

• Fail to Reject $H_o$ and thus: Accept $H_o[419.0]$ as the State of Nature.

The β-Risk What-If Context

To create the needed FNE-decision-intel, the decision-maker [DM] Reflects on the following What IF musing: Following, we suggest the usual DM thought processes:

“The FPE[α%]inferential context that I set-up did not provide Clearly Defensible & Actionable decision-intel.” The reason being: The Mean of the Random sample [$\overline{\text{x}}$] was barley in the Fail to Reject[${\text{H}}_{\text{o}}$] Zone. So, this rationalizes consideration of the β-Risk%. Thus, the issue for me to consider is:

• It is possible that due to random sampling that we missed detecting the actual population that could have had a higher Mean than $\overline{\text{x}}$ = 428.9667. If that were to have been the case, then it is possible that the True Mean—which may have been missed due to Random Sampling—would have produced a sampled Mean $\overline{\text{x}}$ ≥ 429.9879 and thus would have been in the Rejection region for ${\text{H}}_{\text{o}}$.

If all these conjectures are reasonable, then there is a FNE[β-Risk%]-protocol that can be used to create the inferential-intel that would aid in probing the above FPE-inferential result. Thus, it would be logical to examine the [95%[$\overline{\text{x}}$]CI] which is centered at $\overline{\text{x}}$ = 428.9667 as this 95%CI contains the best inferential-intel as to possible Alternative Population MEANs[${\mathrm{\mu }}_{\text{APop}}$].

To better focus on the selection of an Alternative Population MEAN[${\mathrm{\mu }}_{\text{APop}}$], it seems that ONLY the set of logical Alternative Populations should be used. These Alternative Populations are readily accessible. They may be selected from the following FNE-Screening partition of the [95%[$\overline{\text{x}}$]CI]:

Use ONLY Alternative Population MEAN[${\mathrm{\mu }}_{\text{APop}}$] selected from the following partition of the [95%[$\overline{\text{x}}$]CI]:

• |[Greater than The Sampled Mean[[$\overline{\text{x}}$]] as the LOWER Limit] through

• [The UPPER Limit or the Right-Hand Side [RHS]Limit of the 95%CI.]}

For the NFMix-example, the Alternative Population Mean[${\mathrm{\mu }}_{\text{APop}}$]-values should be drawn from the following partition of the [95% [$\overline{\text{x}}$]CI]. {>428.9667 through 438.2241}. We recommend that five Mean[${\mathrm{\mu }}_{\text{APop}}$]-values be created from this continuous interval as follows:

• Select the Mean[428.9667] and the RHS-End Point of the 95%[$\overline{\text{x}}$]CI[438.2241]]. This gives [428.9667 & 438.2241]; then create five MEAN[${\mathrm{\mu }}_{\text{APop}}$]-values by calculating an Interval[δ]. Applying the δ to the RHS-End Point of the 95%[$\overline{\text{x}}$]CI] down to the Sampled Mean—gives: [δ = 2.31434]: [(438.2241−428.9667)/4]. In this case, the following five ${\mathrm{\mu }}_{\text{APop}}$ are created: (438.2241, -, -, 431.2810, 428.9667). However, we are not able to use 428.9667 as this is the Actual MEAN[${\mathrm{\mu }}_{\text{APop}}$]-value; thus, we have opted for 431.2801 as the lowest MEAN[${\mathrm{\mu }}_{\text{APop}}$]-value for creating The β-Risk% profile. Finally, we recalibrated the δ-measure as: [[δ = 1.73576]: [(438.2241−431.2810)/4].] this will create the MEAN[${\mathrm{\mu }}_{\text{APop}}$]-values as found in Table IV. This is the only logical set of Alternative Population MEAN[${\mathrm{\mu }}_{\text{APop}}$]-values that can create a valid inferential-input to the Decision-Making process for arriving at the decision to:

Table IV. β-Risk for Selected Germane FPE[α%] Population [$\text{MEAN}[{\mathit{\mu }}_{\mathbf{\text{APop}}}]$s

${\mu }_{APop}$	Diff v. 2.32635	β-Risk	Codex from Table III
438.224	1.743764606	4.06%	Clearly accept $H_a$
436.488	1.37626691	8.44%	Clearly accept $H_a$
434.753	1.008769215	15.65%	Clearly accept $H_a$
433.017	0.64127152	26.07%	Usually accept $H_a$
431.281	0.273773825	39.21%	Consider: {$H_o$ or $H_a$}

Table III. β-Risk Decision-Guidance Codex

β-error risk%**	β: [1% : ≤25%]	β*: [25% : <40%]	β*: [≥60% : <75%]	β: [≥75% : ≈100%]
β-risk% action	Clearly accept $H_a$	Usually accept $H_a$	Usually accept $H_o$	Clearly accept $H_o$
Related justification Re: β-risk%	High degree of support	Reasonable degree of support	Reasonable degree of support	High degree of support

• Reject ${\text{H}}_{\text{o}}$ thus Accept ${\text{H}}_{\text{a}}$ that ${\text{H}}_{\text{a}}$ is >${\text{H}}_{\text{o}}[419.0]$, or

• Fail to Reject ${\text{H}}_{\text{o}}$ and thus: Accept ${\text{H}}_{\text{o}}[419.0]$.

Decoding the Decision Intel re: The FNE or The β-Risk% or The β-Error or The Type II-Risk

We have listed the various ways that the FNE is labeled. THE Only correct one, in context of this research report, is the β-Risk%. Using the example of the [NFMix], the β-Risk% is conversationally defined for The Example of the NFMix Intervention above as:

The β-Risk% IS: The Percentage Risk TO BE ASSUMED BY THE DECISION-MAKER in Failing to Reject ${\text{H}}_{\text{o}}$.

This INDICATES that the Actual State of Nature is assumed to be: The NFMix was effective. HOWEVER, due to the inferential model used, there is a %RISK that the analyst you will FAIL to reject ${\text{H}}_{\text{o}}$ i.e., The Null of NO Effect and INCORRECLY believe that the was NO Effect of the NFMix when this may NOT be the TRUE State of Nature.

The SIMPLE Version: If the β-Risk% is low, say 5%, this indicates that ONLY 5% of the time the INFERENTIAL model created and used would indicate that the NFMix was NOT effective when it was REALLY was Effective. So, a low the β-Risk% suggests the analyst would likely accept this risk of 5% and reject the ${\text{H}}_{\text{o}}$ of NO Effect in favor of ${\text{H}}_{\text{a}}$—That the NFMix IS, in Fact, Effective.

To probe the FPE context, the analyst usually generates an Iterative Profile to examine the β-Risk%s. Before we present the Iterative Risk% Profile, we need to offer the details of the computation of the β-Risk%.

The Calculation and Interpretation of the β-Risk%

Assuming that the alternative population MEAN[${\mathrm{\mu }}_{\text{APop}}$] is judged to be 438.2241 lbs, and the selected FPE is: [α[1%]] thus, the Normal [${\text{z}}_{\text{N}[0,1]}$] value is: 2.326385, the β-Risk% calculated using the Tamhane and Dunlop (2000, p. 211) formulation where MEAN[${\mathrm{\mu }}_{\text{APop}}$] = 419.0:

• The β-Risk = ABS [2.326385−[(438.2241−419.0)/ (25.8699/$\sqrt{30}$)]] = ${\text{z}}_{\text{N}[0,1]}$ = 1.74377 R[1]

• The directional probability of ${\text{z}}_{\text{N}[0,1]}$ = 1.74377 is: 4.060%

• Finally, the β-Risk% will be 4.06%

Discussion

For R[1], mathematically the larger that the MEAN[${\mathrm{\mu }}_{\text{APop}}$] becomes the smaller will become the related β-Risk%. The reason for this is that there are always TWO distributions in play for R[1]: The Population [${\mathrm{\mu }}_{\text{Ho}:419.0}$]—the Fixed Population & The Population [MEAN[${\mathrm{\mu }}_{\text{APop}}$]]—the Variable Population—i.e., that “moves around” in the probability-space. When the MEAN[${\mathrm{\mu }}_{\text{APop}}$] = 438.2241, that PDF distribution shifts to the Right of Population [${\mathrm{\mu }}_{419.0}$]. The probability implication of this shift is that the amount/percentage of Probability Mass of Population [MEAN[${\mathrm{\mu }}_{\text{APop}}$] = 438.2241] that is IN the Fail to Rejection-zone of Population [${\mathrm{\mu }}_{\text{Ho}:419.0}$] is relatively low due to this shift; actually, the percentage of the Left-Tail of Population [MEAN[${\mathrm{\mu }}_{\text{APop}}$] = 438.2241] that falls in the Fail to Reject-Zone of Population [${\mathrm{\mu }}_{\text{Ho}:419}$] is 4.06% as noted above. These are the facts that rationalize the Nature of the β-Risk% Analytics.

The What-IF Conundrum

The critical point of any analysis of the β-Risk% is to realize that the β-Risk% is a function of various parameters, most of which are sensitive to sampling randomness; and, that the Driver of the β-Risk%-function is Population [MEAN[${\mathrm{\mu }}_{\text{APop}}$]]. Unfortunately, the Population [MEAN[${\mathrm{\mu }}_{\text{APop}}$]] is a value that is a What-If Guesstimate. A colleague quipped:

If you want a particular value of the β-Risk%, no problem—give me a consulting contract and five minutes and I will give you the Population [MEAN[${\mathrm{\mu }}_{\text{APop}}$]] that produces the β-Risk% that you need.

In our consultations. we always require a well-reasoned and vetted-justification for fixing the value of Population [MEAN[${\mathrm{\mu }}_{\text{APop}}$]]. Otherwise, the whole β-Risk% [Population [MEAN[${\mathrm{\mu }}_{\text{APop}}$]]]-exercise is a Math-Stat Gaming exercise. For example, assume that a DM wishes to Reject ${\text{H}}_{\text{o}}$ of 419 lbs in favor of accepting that ${\text{H}}_{\text{a}}$—suggesting that the MEAN[weight] is >419 lbs. To make sure this is case, the analyst selects as the Population [MEAN[${\mathrm{\mu }}_{\text{APop}}$]] = 445 lbs. In this Gaming case, the β-Risk% will be 0.2%; this is strong evidence that there is essentially NO risk in rejecting ${\text{H}}_{\text{o}}$; thus arguing that the NFMix had a dramatic effect on the weight gain. The reason for this is that the other DMs are not exactly sure if 445 lbs is a reasonable test population—of course it is Not—as it was a value selected to create a very favorable rejection possibility. Epilogue: Later it is discovered that the son of the DM that selected 445 lbs as the β-Risk test-value was the CEO of the firm that manufactures the NEW Feed Mix.

Acrimony aside, there are a few very important guidelines that aid in moving in the direction of a meaningful β-Risk% analysis. These are:

1. Compute the [95% [$\overline{\text{x}}$]CI] and restrict [MEAN[${\mathrm{\mu }}_{\text{APop}}$] to the following interval:

• [Range [$\overline{\text{x}}$ < MEAN[${\mathrm{\mu }}_{\text{APop}}$] ≤ [Upper Limit[[95%[$\overline{\text{x}}$]CI]]]]],

2. Expect that the β-Risk%[Upper Limit[[95%[$\overline{\text{x}}$]CI]]] will be on the order of ≈ 5%. If this not the case, this is usually an indication of a computation error and so a simple vetting indication.

3. The Ex-Post Analysis Phase: If the β-Risk% is used to form a decision plan: Record all the intel that went into the FPE & FNE analyses and at some point record the actual results re: the decision actually made and implemented. We suggest that the coding be: Binary: Successful [Score 1] and Not Successful [Score 0]. After a reasonable number of scored trials, then an Ex-Post vetting analysis can be used to judge the efficacity of the β-Risk% Analysis for future calibrations. This is our suggestion from two-frequentist-prone analysts.

Closing Homily

As the False Negative- Type II- or or β-Risk% become smaller, the decision indication is more and more clear to Reject ${\text{H}}_{\text{o}}$ in favor of ${\text{H}}_{\text{a}}$. We have polled our colleagues and offer the synthesis of their suggestions as in Table III.

As an illustration of Table III, assume that we are using the VBA-platform and we start at the RHS of the [95%[$\overline{x}$]CI] as noted in Table II which is: 438.2241 and work down—i.e., shifting the Population [$MEAN[{\mu }_{APop}]$438.2241 $lbs$] back towards the Population [${\mu }_{419}$] and ending at Population [${\mu }_{431.2810}$]. In this case, Table IV would be produced.

The summary of Table IV is simple:

1. Vetting Indication At the RHS: β-Risk%[UL[[95%[$\overline{\text{x}}$] CI]] point [438.224 lbs] there is a β-Risk%[4.06%] on the order of: ≈ 5%; this is excellent vetting-intel, and

2. Summary of the Scoring Codex of Table III Most all of the “recommended” actions are in the Reject ${\text{H}}_{\text{o}}$ in favor of Accepting ${\text{H}}_{\text{a}}$—This is the usual case in our experience!

Alert

We have not broached the topic of the Meaning of [1−[β-Risk%]] sometimes labeled, in most of the FNE-literature, as the Power of the Test[β-Risk%]—The Worst case of mis-labeling in the history of Linguistics. The reason for this rather harsh critique is: The word Power pertains and is restricted to the detection parameters desired and parameters of the structure of the Power Model. Power can be an a priori measure or an ex-post measure typically relative to: [The FPE, or The detection range for the ${\sigma }_{Pop}$, or The required likelihood of determining a Significant Result over the parameter(s) of interest], or the Effect Size so as to solve for the Sample Size for the given statistical model selected for the analysis. Cas Fermé! Power is NOT in any way related to the What-If Effect produced in the calculation of the β-Risk%. In the following sections of this research report, we shall not re-take up the details and critique of the Nature of the β-Error Risk% but rather just produce and display the results.

The Construction of the Population Version of the VBA-Platform

Overview

In this example, a Normal Population may be developed and vetted from a simple and logical re-conceptualization of the actual probability context. Specifically, if we have a Bernoulli set of events, then the correct probability model is the Binomial Probability Point Density Function. These Binomial-probabilities use as parameters: (i) The inherent success-rate, (i) The related failure rate: [1−[success-rate]], and (iii) the number of trials [n]. This information may also be used to form a useful but approximate continuous-probability model. Specifically, a proposed two-parameter approximate continuous PDF may be formed as:

$$\begin{array}{r}{f}_{PDF}[\mu =p_s,\sigma =[p_s\times (1-p_s)]\end{array}$$

where: $p_s$ = S/n [S = Number of Successes] or Inherent Population Rate, $p_s$ = ${\pi }_{\mu }$, and σ = [$p_s$ × (1−$p_s$)].

The interesting aspect of borrowing the Parameters from the Binomial PPDF is that the ${\text{f}}_{PDF}$ “morphs” into the Normal [$\mu =p_s$], σ = [$p_s$ × (1−$p_s$)] iff the following joint-conditions are satisfied: {[n × $p_s$] & [n × (1− $p_s$)]} > 5 (Boxet al., 1978) This simple conditional-morphing of the Point-Binomial to the Normal facilitates the creation of FPE & when needed The FNE[β-Risk%] intel. Following is an illustrative example.

Assume the Ott-Livestock Example

In this case, rather than to measure the actual relative-weight gain of the Livestock on the Intervention[NFMix] relative to the other Animals in the Herd, the analyst is interested in the percentage of Livestock in the Two Groups [The Herd[The Control] v. The Animals on the Intervention[NFMix] protocol] that have gained at least 45 lbs.

The Pre-Testing Intervention Population, there was a random selection of 150 Animals from the Livestock Herd all of which were tagged for RFI-screening. All the animals in the Herd [including the RFI-tagged animals] were weighed on three occasions: (i) The Monday of the week before the Intervention[NFMix], (ii) Randomly, over the Tuesday through the Saturday of the week before the Intervention[NFMix], and (iii) the Sunday before the Intervention[NFMix]. The weighted-average of these three weights was computed and used as the Base-Line Weight for the Herd [including the RFI-tagged animals]. Then, on Monday, the Intervention[NFMix] was activated and continued for three months.

For the Post- Intervention Event, the Actual Weight Gain for each of the 150 NFMix animals was measured and compared to their weight at the start of the Intervention[NFMix]. The same measurement were made for the for the Herd Animals not on the Intervention[NFMix]. For the 150 NFMix Group: 35% of these test animals gained ≥ 45 lbs; for the rest of the animals in the Population Herd, 30% of these “Control Animals” [assumed to surrogate for the Population], gained ≥ 45 lbs.

Testing the BBH-conditions: Rationalizing the Normal

The Population Percentage against which, the Animals on the Intervention[NFMix] will be tested, is: 30%. Thus, the ${\text{f}}_{PDF}$ will surrogate for the Normal iff: {[n × $p_s$] & [n × (1−$p_s$)]} > 5. In this sampling case, we have: {[150 × 30%] = 45 > 5 & [150 × (100%−30%)] = 105 > 5. As these BHH-conditions are satisfied, we can use ${\text{f}}_{PDF}$ for our inferential analysis as: Normal[μ[30%], σ[21%]].

Inputs to the VBA-Platform [Constructed Population]

Following the VBA-platform [Constructed Population] is launched:

• [$1_{\text{VBA}}$] The Mean Percentage of the Herd Livestock that were fed the Traditional Feed Mix and that gained ≥ 45 lbs was [${\mathrm{\mu }}_{\text{Trad}}$] = 30%.

• [$2_{\text{VBA}}$] The Mean Percentage for the 150 Livestock Fed the NFMix that gained ≥ 45 lbs was [${\overline{\mathrm{\pi }}}_{\text{NFMix}}$] = 35%.

• [$3_{\text{VBA}}$] Sample Size [n]: 150 These are the Livestock that were fed only the NFMix.

• [$4_{\text{VBA}}$] Directional Testing False Positive Error [FPE]-Expectation was: [${\mathrm{\alpha }}_{\text{Decision}}$]: 5.0% [0.05]

The VBA Computation Set

${\text{VBA}}_{\text{Comp}}$ Standard Deviation of Herd-Population that only were fed the Traditional Mix Post re: the percentage that gained ≥ 45 lbs [${\mathrm{\sigma }}_{\text{Pop}}$] will be computed by the VBA-platform and detailed following.

${\text{VBA}}_{\text{Comp}}$ Proposed Population Mean as the FNE-Concern Population [$\text{MEAN}[{\mathrm{\mu }}_{\text{APop}}]$] will be developed subsequently and displayed in Table V.

Table V. Input and VBA Results

Profile of the FNE	Sample [n = 150]
95% mean confidence interval	{27.67% : 42.33%}
Testing expectation	5% N[0,1]] z = 1.644869
Actual result [35%] v. [$H_o$]: 30%]	z = 1.336306 9.1% N[0,1]
FNE implication	Possible as 9.1% > 5.0%

With this INPUT the VBA-Displays the profile in Table V.

Discussion

The VBA-program computes the Standard Deviation of the Population [${\sigma }_{\mu }$] as:

$$\begin{array}{r}\left[{\mathit{\sigma }}_{\mathit{\mu }}\right]=(30\mathrm{\%}\times (1-30\mathrm{\%}))=21.0\mathrm{\%}\end{array}$$

Continuing: The $z_{cal}$, as calculated by the VBA-platform: where [$\overline{x}$ = 35%], is:

$$\begin{array}{r}z\mathrm{\_}\left(N[0,1]\right)=\left[35\mathrm{\%}–30\mathrm{\%}\right]/[(\sqrt{2}1\mathrm{\%}/150)]=1.336306\end{array}$$

The N[0,1]-probability of a RHS[>z] of 1.3363 is: 9.07%.

As 1.336306 is <1.644869 [The [α[5%]] desired False Positive Error testing cut-point for analysis], but is “sort of close” to the actual p-value, it is not unreasonable for the analyst to examine the Nature of the False Negative Error—the β-Risk% before making the decision on the ONLY two choices in the decision-context:

I. Failing to Reject the H_o, thus indicating that there was No inferential evidence that more than 30% of the Animals of the Intervention experience a weight increase over ≥ 45 lbs, or

II. Reject the H_o thus accepting H_a that the Intervention resulted in the state of Nature where more than 30% of the Animals of the Intervention experience a weight increase over ≥ 45 lbs.

The β-Risk% Decision

For Table VI, as is the recommendation noted above, we computed the δ[increment] that was: −1.37506%. Thus, we started at the Upper Limit[[95%[$\overline{\text{x}}$]CI]] which is: 42.33% and worked down—i.e., shifting the [$\text{MEAN}[{\mathrm{\mu }}_{\text{APop}}$]] = [$42.33\mathrm{\%}$] back to the value [$\text{MEAN}[{\mathrm{\mu }}_{\text{APop}}]]$ = [36.83%]. In this case, Table VI would be produced.

Table VI. β-Risk for Selected Germane FPE[α%] Population [$MEAN[{\mu }_{APop}]$s

${\mu }_{APop}$	Diff v. 1.644869	β-risk	Codex of Table III
42.33%	1.65143	4.93%	Clearly accept $H_a$
40.96%	1.28393	9.96%	Clearly accept $H_a$
39.58%	0.91643	17.97%	Clearly accept $H_a$
38.21%	0.54893	29.15%	Usually accept: $H_a$
36.83%	0.18143	42.80%	Consider: {$H_o$ or $H_a$}

Computational Illustrations

Assume that the analyst enters [$MEAN[{\mu }_{\mathit{A}\mathit{P}\mathit{o}\mathit{p}}]$] = 42.33% as the β-Risk% test against value. The illustrative computations are:

The β-Risk = ABS[1.644869]−[[ABS[42.33%−30%]/ [$\sqrt{21\mathrm{\%}/150}$]]] = $z_{N[0,1]}$ = 1.65144

The β-Risk% directional probability of $z_{N[0,1]}$ = 1.65144 is: 4.93%

Discussion Note

The Summary of Table VI is simple:

1. Vetting Indication At the RHS: β-Risk%[UL[[95%[$\overline{\text{x}}$] CI]] point is: [42.33% lbs] there is a β-Risk%[4.93%] on the order of: ≈ 5%; this is excellent vetting-intel, and

2. Summary of the Scoring Codex of Table VI Most all of the “recommended” actions are in the Reject ${\text{H}}_{\text{o}}$ in favor of Accepting ${\text{H}}_{\text{a}}$—This is the usual case in our experience!

Summary and Outlook

Summary

The decision to entertain considering evaluating the FNE so as to develop intel on the β-Risk is contingent on:

The magnitude of the Actual FPE[p-value] of the test of [The Mean[$\overline{\text{x}}$]] vis-à-vis The a priori FPE[α] specified by the analyst. The FPE[α] partitions the probability abscissa into two zones:

${\text{H}}_{\text{o}}$ is the Null indicating no mean effect of the intervention. Rejecting ${\text{H}}_{\text{o}}$ and opting to accept ${\text{H}}_{\text{a}}$ that indicates that there is inferential evidence that the likely state of nature is that there is an intervention effect.

Simply If the Actual FPE[p-value] is ≤ than the p-value of the FPE[α] specified by the analyst, then the analyst is likely to reject ${\text{H}}_{\text{o}}$ in favor of a meaningful inferential Effect—i.e., accept ${\text{H}}_{\text{a}}$ that the intervention produced an expected result. In this case, there is no logical inferential reason to compute a FNE.

The FNE & The β-Risk% ONLY are interesting intel if:

• Condition [1]: The Actual FPE[p-value] is >the p-value of the FPE[α] specified by the analyst, and

• Condition [2]: It is “close” to the p-value of the FPE[α] specified by the analyst.

For the second example for the Ott-Intervention[NFMix], the selected FPE[p-value] was 5%; whereas, the actual FPE[p-value] computed for the inferential test of the NFMix was 9.07% and was > than 5% but was “sort of close”—i.e., close enough so that a reasonably prudent Analyst probably would have created the FNE-intel in making a decision.

Simply if actual z-value computed from the inferential intervention has a FPE[p-value] that is > but “close” to that of the Selected FPE[α], then and only then does consideration of the β-Risk% come into play in the decision-making analytics.

To aid in the pursuit of a sensible decision re: The β-Risk%, we offered the collective options of our colleagues on when to Accept the ${\text{H}}_{\text{o}}$ or ${\text{H}}_{\text{a}}$ as expressed in Table III. True, there seems to be a Lacuna in Table III where it may not be possible to offer unconditional advice on Accepting: ${\text{H}}_{\text{o}}$ or ${\text{H}}_{\text{a}}$; specifically, in the range the range interval [≥40% : 60%<]. This indicates that: as in most Analytical endeavors there are Zones of Uncertainty—this is part of any sensible analytical decision suggestion-set.

The reason for requiring that DM use ONLY the FNE[β-Risk%] Population alternatives that are “reasonable possibilities” as they are abstracted from the FPE- intel of the [95%[$\overline{\text{x}}$]CI], is that we find/hope that this effectively eliminates “Gaming the FNE[β-Risk]” by DM who sometimes offer pseudo-FNE[β-Risk%]-intel—such as Population[${\mathrm{\mu }}_{\text{APop}}$] alternatives far outside the RHS of the [95%[$\overline{\text{x}}$]CI] to serve a personal-agenda often not in the best interest of the creating useful decision-making intel from the firm’s perspective. To aid in this aspect, the VBA-DSS has the following two-features:

I. VBA Generated Termination Alert If the p-value computed for the inferential test is LOWER than or Equal to [≤] that of the FPE[α] selected by the analyst, the VBA produces the following Alert:

“The FPE:[p-value] calculated for the Experimental design IS [≤] to that selected by the Analyst. In this case, there is no NEED to calculate the FNE. Thus, the VBA Program will terminate.”

II. Collegial Suggestion-Set of Table III The other decision-aide of the VBA-Program is that the Table III-codex of suggestions from our colleagues will be displayed next to the five β-Risk% designations profiled in Table A of the VBA-Program.

The VBA-DSS is an open-access module that is offered without any restrictions on its use only as an e-mail download. Communicate with either of the authors in this regard.

Outlook the Ex-Post Follow-up Phase as a Final Critical-Issue

It is often not part of the Inferential Process, to follow-up on the actual decision that is made as the result of the {FPE[α-Risk] & FNE[β-Risk%] analysis}.

The final stage in any Inferential process is: THE EX-POST FOLLOW-UP. Specifically, at some point the DMs, after reviewing the FPE[α-Risk] & The FNE[β-Risk%]-intel, make the decision to: {Reject: ${\text{H}}_{\text{o}}$ [Accept ${\text{H}}_{\text{a}}$] or Fail to Reject: ${\text{H}}_{\text{o}}$}. At some other point in the future, there is usually an indication if this decision was correct or not correct. Thus, at this future point, the indication of whether that decision was correct or not correct must be recorded in the ex-post archive for that decision. Finally, the ex-post archive can be used to create a frequency profile of the successes and failures of using the {FPE[α-Risk] & FNE[β-Risk]-analytics in fine-tuning their future decisions. Reality check in our consultations, usually all the DM agree that this ex-post phase is an excellent idea. However, in no cases in our experience has the ex-post follow-up became a reality!