*Proceedings of the Seventeenth Symposium on
Explosives and Pyrotechnics, Essington, PA, April 1999.*

All-Fire/No-Fire Test and Analysis Methods

Barry T. Neyer | James Gageby |

EG&G Optoelectronics | JVG Associates |

Miamisburg, OH 45343-0529 | Palm Desert, CA 92211-1399 |

Contact Address

Barry T. Neyer

PerkinElmer Optoelectronics

1100 Vanguard Blvd

Miamisburg, OH 45342

(937) 865-5586

(937) 865-5170 (Fax)

Barry.Neyer@PerkinElmer.com

A new international standard for explosive systems and devices has been developed. It combines key attributes of multi-national standards, specifications, regulations, manuals, publications and journals into a uniform set of criteria that can be used to safely and reliably apply explosive systems onto space vehicles. This paper presents some of the details of this standard, with the major emphasis on the All-Fire and No-Fire test methods.

Technical Papers of Dr. Barry T. Neyer

A new international standard has been developed. It combines key attributes of multi-national standards, specifications, regulations, manuals, publications and journals into a uniform set of criteria that can be used to safely and reliably apply explosive systems onto space vehicles.

The new standard is known as ISO 14304, Criteria for Explosive Systems and Devices Used on Space Vehicles. It provides improved understandings of all areas of the field of explosive systems, also known as pyrotechnics. The criteria outlined in this new international standard are a composite of those verified by previous use in space vehicle applications.

Described are essential design characteristics, suggested manufacturing controls, and methods for certifying performance, acceptance, qualification and useful life. It is intended to be universal sets of tools for explosive system manufacturers and users during all phases of development, certification and application.

Criteria are defined here as standards, rules or tests by which items can be judged. They are measures of value that have proven to be effective in numerous space vehicle applications. These same criteria may also be used in applications unrelated to space vehicles. A goal of this effort was to produce a quality document that can be used by all.

Explosive systems include components and assemblies that provide stimuli for initiation and propagation of explosive trains used to activate explosive devices. Explosive trains and devices are components or assemblies containing or operated by explosive materials. The latter are, by design, "one-shot" components that cannot be tested completely before use. Performance confidence of "one-shot" components can only be obtained by destructive tests of like samples from common production lots.

Key attributes of this new international standard:

- Explosive/pyrotechnic system criteria are offered, that is, performance criteria from ignition stimulus through component activation at the application interface is outlined. No other standard offers system criteria to this level.
- Criteria for advanced technologies in the explosives/pyrotechnics field is included.
- Test and analysis methodologies of the 1940’s, 1960’s and 1990’s are compared and discussions of their application for assessing performance, safety, reliability and confidence given.
- Performance margin criteria for ignition systems, first elements, explosive train components, explosively actuated devices and their interfaces are addressed.
- A comprehensive list of definitions of terms commonly used in the explosive systems/pyrotechnics space vehicle field is provided.

The group responsible for this new international standard is the International Organization for Standardization (ISO), Technical Committee 20 (Aircraft and Space Vehicles), Subcommittee 14 (Space Systems Operations), Working Group 1 (Design Engineering), otherwise known as ISO/TC20/SC14/WG1.

Note that the acronym ISO was used in lieu
IOS to preclude language confusion (English – IOS, French
– OIN). It is a derivative of the Greek *isos, *meaning
equal.

The main body of ISO presently has more
than 166 technical committees working on standardization of a
multitude of items and subjects. The organization is based in
Geneva and was initiated in 1947 via a United Nations resolution
establishing universal compatibility of commonly used items. An
overview of the ISO is given at website __http://w__ww.iso.ch/

Compliance to international standards created by ISO/TC20/SC14 is voluntary. Their intent is to enhance scientific cooperation and promote trade. They do not require individual countries to change or discard their existing specifications or standards. Rather, individual countries are encouraged to exchange information that can be included in ISO standards. ISO/TC20/SC14 goals are given at website www.aiaa.org/information/professional/standards/tc20sc14.html.

Within ISO/TC20/SC14, voting member nations include Canada, China, France, Germany, Italy, Israel, Japan, Russia, the United Kingdom and the United States. Non-voting member nations include Argentina, Brazil, India, Poland, Slovakia and Spain.

The American National Standards Institute (ANSI) is the primary U.S. member body overseeing ISO/TC20/SC14 efforts. They are assisted by the American Institute of Aeronautics and Astronautics (AIAA).

The ANSI/AIAA intent is to have
ISO/TC20/SC14 ISO standards relating to Space Vehicles dominate
the 21^{st} century, bypassing National Commercial and
Government Standards of the 1970’s through the 1990’s.

ISO 14304 has nearly completed the ISO preparatory stage and will progress to the committee stage shortly. It is anticipated that final release will occur in mid-2001. Copies of the full ISO 14304 proposed standard can be downloaded from http://home.att.net/~j.v.gageby/iso14304.html or http://www.aiaa.org/tc/ecs/.

The rest of this paper presents the details of Annex B, which describes the All-Fire /No-fire Test and Analysis Methods.

**ANNEX B, All-Fire/No-Fire Test and
Analysis Methods**

**1. **__Scope__**. **This
Appendix offers test and analysis methods for estimating first
element functional all-fire and non-functional no-fire input
energy ratings. These are applicable to electrical, optical and
mechanical first elements. Reliability and safety issues
necessitate use of methodologies that can assure consistency in
derivation of these estimates. Accepted test and analysis methods
for estimating these parameters include the Bruceton method, and
other advanced methods such as Langlie and Neyer D-Optimal, all
of which are described here. The Bruceton method was developed in
the 1950’s and has proven its value. Advanced methods
described here have been evaluated through experience in use and
may provide improved knowledge of estimates of key parameters
while reducing costs to obtain them.

**2. **__Applicable Documents__**.
**Product specifications of first elements tested. Detailed
descriptions of data acquisition measurement methods and
instrumentation components used in tests and, the following
technical reports and publications.

**Technical Reports**

NAVORD Report 2101 - Statistical Methods Appropriate for Evaluation of Fuze Explosive Train Safety and Reliability, U.S. Naval Ordnance Laboratory, White Oak, MD, (1953).

Report U-1792 - A Reliability Test Method for One-Shot Items, Langlie, H. J. (1965), Technical, Aeronautical Division of Ford Motor Company.

Report MLM-3736 - An Analysis of Sensitivity Tests, EG&G Mound Applied Technologies, (1992).

__Publications__

Journal of the American Statistical Association - A Method for Obtaining and Analyzing Sensitivity Data, Dixon, J. D., and Mood, A. M. (1948), Vol. 43, 109-126

Technometrics - A D-Optimality Based
Sensitivity Test, February 1994, Volume 36, Number 1, pages
61-70, Neyer, B. T. (1994)

(See http://www.neyersoftware.com/Papers/D-Optimal/D-OptimalAbstract.htm)

**3.
**__Test and Analysis Methods__**. **The following describes test and analysis
techniques most commonly used for estimating and evaluating first
element all-fire and no-fire input energy ratings. Although test
and analysis are independent functions, each unique test method
is historically associated with a unique analysis technique.
These methods and techniques are commonly referred to as
sensitivity tests and analysis. First elements with outputs that
are independent of input magnitude require the specialized test
and analysis methods described here.

**3.1 **__Objectives__**. **Objectives of sensitivity test and analysis
methodologies used should be assurance that estimates derived are
as accurate and precise as possible. The parameter to be
estimated is the mean stimulus level at which some fraction of
the samples of a specific first element design will always
ignite, in the case of an all-fire test, or not ignite, in the
case of a no-fire test. There are no methodologies capable of
exact determinations of this parameter. There are no
non-destructive methods available to obtain the data needed. The
tests described here are usually considered as destructive in
nature and therefore, the test articles should not be re-used in
the end item. The analysis portion of the method computes
estimated all-fire or no-fire rating. These estimated ratings are
computed at a specific reliability and confidence and are
applicable to only the specific first element design tested.
These estimated ratings are computed at a specific reliability
and confidence and are applicable to only the specific first
element design tested.

**3.2 **__Limitations__**. **All methods used are small sample based.
Therefore error in the estimates may occur. Care must be
exercised when choosing test stimulus levels during the tests. If
empirical data on the specific design is not available to assist
selection of test levels before start of the test then additional
samples should be allocated to perform pre-test evaluations. All
of the methods used here assume the distribution of the threshold
stimulus levels is normal. It is simple to generalize this
assumption and require that some function, such as a logarithm of
the threshold levels are normally distributed.

**3.3 **__Reliability and Confidence Levels__**. **Reliability and confidence level values
conventionally used in sensitivity tests and analysis are 0.999
and 95%, respectively. This is literally interpreted to mean that
95% of the time no more than 1 in 1000 first elements will fail
to function at the estimated all-fire rating. Therefore, the user
should assure that the ignition stimulus delivered to the first
element in the end item application not be limited to the
all-fire rating, as noted in WD/ISO14304, paragraph 4.4.1. Users
of these computed values should be made aware that adding margin
to estimated all-fire and no-fire values is standard practice. As
noted in paragraph 4.4.1 of WD/ISO14304 an ignition system should
use input stimulus 1.25 times greater than the estimated all-fire
threshold of the interfacing first element. For example, when
using an EED having an all-fire estimate of 3.25 amperes, the
ignition system should be designed to have a minimum output of
4.06 amperes. Explosive system reliability assessments should
therefore use the minimum stimulus values that the ignition
system delivers to the first element to assess realistic system
level reliability.

**3.4 **__Test Conduct__**. **Tests
should be performed in an ambient temperature environment unless
conditions anticipated in the end item application dictate a need
to do otherwise. Heat sinks used should simulate thermal
properties of the end item application, to the extent practical.
Once started, the test should continue uninterrupted until
completed. Analysis can be performed at any time during or after
completion of the test portion of the task. For EED and LID first
elements, all-fire tests should use an ignition stimulus pulse
duration equivalent to that used in the end item application, but
should be no greater than 30 milliseconds. No-fire tests are not
required for mechanical first elements.

**3.4.1 **__Bruceton Test__**.
**At least forty-five (45) first elements should be allocated
for each test. The first sample is pulsed at a defined stimulus
level and duration. If that sample fires the next test sample is
pulsed at a stimulus level that is reduced by a defined
increment, or step, lower than the first. If the first sample had
not fired the next sample would have been pulsed with a stimulus
increased by the same defined increment. The test continues in
this process until at least forty (40) samples are expended. Each
sample is pulsed only once during these tests.

The total number of incremental steps of fire and no fire data points should be greater than three (3), but not more than six (6). Tests where the numbers of increment steps are outside this range should be considered invalid. To prevent this, care should be exercised in selecting the magnitude of the initial stimulus used and in the amount of the defined increment, or step between succeeding pulses before starting the test. Experience with similar first element designs and/or pre-test firings can be used to estimate these values. Five (5) samples of the allocated group can be used in initial searches for reasonable starting points. If determined to be valid data, these five may be combined with the total sample tested.

**3.4.2 **__Langlie Test__**.
**The main goal in developing the Langlie method was to
overcome the dependence of the efficiency of the Bruceton test on
the choice of the step size. Analysis and experience had shown
that the defined increment, or step size of the Bruceton test had
to be correct to within a factor of two for reliable results.

To perform a Langlie test, the experimenter
must specify lower and upper stress limits. The first test is
conducted at a level midway between these limits. The general
rule for obtaining the (*n*+1)^{st} stress level,
having completed *n* trials, is to work backward in the test
sequence, starting at the *n*^{th} trial, until a
previous trial (call it the *p*^{th} trial) is found
such that there are as many successes as failures in the *p*^{th}
through *n*^{th} trials. The (*n*+1)^{st}
stress level is then obtained by averaging the *n*^{th}
stress level with the *p*^{th} stress level. If
there exists no previous stress level satisfying the requirement
stated above, then the (*n*+1)^{st} stress level is
obtained by averaging the *n*^{th} stress level with
the lower or upper stress limits of the test interval according
to whether the *n*^{th} result was a failure or
success.

The Langlie test has been shown to be less susceptible to variations in efficiency caused by inaccurate test design. The efficiency of the test is somewhat dependent on the choice of lower and upper stress limits. Most users specify limits that are extremely wide to avoid the situation where the limits are too close together, or do not contain the region of interest. The method is most efficient if the upper and lower limits are ± 4 standard deviations from the mean.

One problem with the test method, however, is that the method concentrates the test levels too close to the mean, resulting in inefficient determination of the standard deviation of the population.

**3.4.3**__ Neyer D-Optimal Test__**.**
This test was designed to extract the maximum amount of
statistical information from the test sample. Unlike the other
test methods, this method requires detailed computer calculations
to determine the test levels. The Neyer D-Optimal test uses the
results of all the previous tests to compute the next test level.

There are three parts to this test. The first part is designed to "close-in" on the region of interest, to within a few standard deviations of the mean, as quickly as possible. The second part of the test is designed to determine unique estimates of the parameters efficiently. The third part continuously refines the estimates once unique estimates have been established.

This test requires the user to specify three parameters, i.e., lower and upper limits, and an estimate of the standard deviation. The first two parameters are used only for the first few tests (usually two (2) tests) to obtain at least one fire and one fail to fire. The estimate of the standard deviation is used only until overlap of the data occurs. Thus, the efficiency of the test is essentially independent of the parameters used in the test design.

**3.5 **__Comparison of Test Methods__**.**
There is no unambiguous method of ranking the test methods. A
good test method should yield estimates of the parameters of the
population that are accurate and precise. All of the test methods
yield accurate parameters on average. Thus, the best way to
characterize the tests is by their precision. The purpose of most
sensitivity tests is to determine an all-fire or a no-fire level.
These levels are usually defined as that level at which at least
0.999 of the first elements fire (all-fire) or at which at least
0.999 of the first elements fail to fire (no-fire). With the
assumption of normality, the all-fire and no-fire levels can be
converted into a simple function of the mean, m, and the standard
deviation, s,
of the population. The 0.999 all-fire level is m +3.09 s, and the 0.001 no-fire
level is m
-3.09 s.
Thus, precise determination of the all-fire or no-fire level
requires precise determination of the mean, and especially the
standard deviation.

There are several ways to compare the ability of the various test methods to precisely determine estimates of the standard deviation. One method would be to determine the variation of the estimates of the standard deviation as a function of sample size and test method. This variation depends not only on the test method, but also on the selection of the parameters of the population before beginning the test.

The efficiency of the Bruceton test is strongly dependent on the choice of step size. The efficiency of the Langlie test is somewhat dependent on the spacing between the upper and lower test levels. The Neyer D-Optimal test is essentially independent of the choice of parameters.

Figure 1 shows the variation of the estimates of the standard deviation as a function of the sample size for the three test methods under the assumption that the standard deviation is well known before start of testing. If the first elements are well characterized from previous tests, the standard deviation may be known to approximately a factor of two. The figure assumes that the parameters of the test were optimized for the population.

The figure also shows that variation of the estimate of the standard deviation has a strong dependence on the test method chosen. For example, a 20 shot Bruceton test yields a relative variance of 66%, while the Langlie test yields a variance of 28% and the Neyer D-Optimal yields a variance of 20%. The publication by Neyer (1994), noted in section 2, provides more in-depth details of the analysis used to produce this graph. This graph is only valid for the case where the experimenters know the correct test parameters. The variation of the Bruceton test in particular could be much larger if the estimate of the step size is wrong by more than a factor of two.

**Figure 1: Comparison of
the Variation in Estimates of the Standard Deviation**

Another method of judging the utility of the various test methods is to determine the extreme values of the estimates of a parameter. The greatest concern in conducting and analyzing sensitivity tests is the tendency of the method to produce estimates of the parameters that are far removed from the true parameters.

Figure 2 shows the 5% and 95% values of the standard deviation as a function of sample size for the three test methods. Also shown in the figure are the corresponding curves for the F Test whose significance is described in the text below.

**Figure 2: 5% and
95%Estimates of the Relative Standard Deviation**

The figure illustrates several important points. First, it is extremely difficult to establish the value of the standard deviation to a degree of precision desired with a limited sample size. For example, in a Neyer D-Optimal test with a sample size of 150, 5% of the estimates of the standard deviation will be more that 20% lower than the true value, and 5% of the estimates will be more than 20% higher than the true values. For the same sample size for the Langlie test, 5% would be 50% lower, and 5% would be 45% higher. For the Bruceton test the corresponding results are 100% lower, and 30% higher.

For the typical sample size used in threshold tests, e.g., 20 - 50, it is impossible to estimate the standard deviation, and thus the all-fire and no-fire levels, with great certainty. Thus, in addition to the estimation of the parameters of the population, it is also imperative that the appropriate analysis be performed to estimate the confidence of the estimate of the parameters. Confidence estimation is discussed in the next section.

The F Test curves shown in Figure 2 indicate how much less information is available for sensitivity tests compared to standard statistical tests. The F Test is used in standard statistical testing to calculate the fraction of estimates of the standard deviation that are higher or lower than a given value. If it were possible to measure the exact threshold of individual first elements, then the estimates of the standard deviation would be governed by the F Test. Inspection of the curves shows that a sensitivity test requires a sample size many times greater than the sample size of a classical statistical test to achieve the same range of values for the standard deviation.

The final point illustrated by the figures is that the ability to determine reasonable estimates of the parameters is extremely dependent on the test method chosen to conduct the test. Both Figure 1 and Figure 2 illustrate the importance of choosing an efficient test method when conducting sensitivity tests.

**3.6**** Analysis Methods**.
More methods of analyzing the results of sensitivity tests have
been proposed than have test methods. The method chosen to
analyze the data of the test is at least as important as the test
method. While many analysis methods can be used to analyze the
results of any test method, other analysis methods are designed
to analyze only one test design. All of the analysis methods
provide relatively unbiased estimates of the parameters of the
population, i.e. the estimate of the mean,

The variance function method assumes that
the variances of *M* and *S* can be estimated by simple
functions of the sample size and the standard deviation. These
functions are generally dependent on the initial conditions,
sample size, and the test design, i.e., Bruceton, Langlie, Neyer
D-Optimal. Some groups use the T test to compute confidence
intervals for the mean and Chi Squared or F tests to compute
confidence intervals for the standard deviation. However, these
generalized statistical methods should not be used. The
assumptions that are used to construct the general statistical
tests are violated in the case of sensitivity tests. Figure 2 shows the curves for both the F test as well as
curves for the various sensitivity tests. The figures clearly
show that the F test can not be used to analyze sensitivity
tests.

The simulation method uses test results to
determine the variance of the parameters after the test has been
completed. This method can provide reliable estimates of the
variances* *as long as the simulation is carried out with
parameterization relevant to the population. If simulation is
used to estimate the variation of the parameters, the parameters
for the simulation must span a wide area around the estimates of
the test data. The number of simulation runs must be sufficient
(over 1000) to ensure that the results are statistically valid.

The asymptotic method is used by some computer programs, such as ASENT discussed in section 3.6.2, and in the calculations of the variance in the Bruceton method.

Simulation discussed in some of the
referenced papers shows that the variance of both *M* and *S*
scales approximately with s^{2}. Because s^{2} is not independently known. All of the
previously mentioned techniques base their estimates on the
maximum likelihood estimate of s, which is *S*. If the successes and failures
do not overlap, *S* = 0 and these methods fail to produce
estimates for confidence regions for both *M* and *S*.
The likelihood ratio method discussed in section 3.6.3 can produce reliable confidence interval estimates
in all cases, including this degenerate case.

Almost all of the analysis methods used to
date produce false confidence. That is, what is reported as a 95%
confidence level is in actuality more like a 60% confidence level*.
*Thus, there should be agreement between the first element
user and the test facility as to which analysis method is used,
and the method should be one that has been shown to produce
realistic confidence levels.

Records of test data, computations and results should be retained as permanent parts the first element documentation package.

**3.6.1 **** Bruceton Analysis**.
The Bruceton test was developed before the advent of electronic
computers. It was designed so that simple paper and pencil
calculations could be used to determine the mean, the standard
deviation, as well as estimates of their variance. Today, more
advanced analysis methods are available to analyze this data. The
traditional Bruceton analysis method can still be used, but only
when the number of test levels are between 4 and 6, and the
sample size is not less than 40. In all cases it is preferable to
use advanced analysis methods, such as the ones described in the
following two sections. When the Bruceton Analysis method is
applicable, the Asymptotic method of the next section yields the
same results.

**3.6.2
**** Asymptotic Analysis**. Advanced methods include computer software known
as ASENT which is in use at many test facilities. Although this
analysis software is usually associated with the Langlie test
method, it can analyze the results of tests conducted according
to any test method. The analysis method computes the maximum
likelihood estimates of the parameters. It computes estimates of
the variance of the parameters by computing the curvature of the
likelihood function. This analysis method gives the correct
results asymptotically. It will not analyze the results of a test
where the successes and failures do not overlap. It gives
reliable results if the sample size is greater than 200.

**3.6.3 **** Likelihood
Ratio**. Another advanced
method is the likelihood ratio test. This method is used in
software known as MuSig, as described in Report MLM-3736 of
section 2. This software is in use at many laboratories around
the world. Although this is the analysis method usually
associated with the Neyer D-Optimal method, it can analyze the
results of tests conducted using any test method. The analysis
method computes the maximum likelihood estimates of the
parameters. It computes estimates of the variance of the
parameters by using the likelihood ratio test. This analysis
method gives the correct results asymptotically. It will analyze
the results of any test, even if the successes and failures do
not overlap. It gives reliable results if the sample size is
greater than 20.

**3.7 **** Comparison of Analysis
Methods**. The two most widely used general analysis
methods can be compared in a number of ways. The most meaningful
way to compare the methods is to determine what fraction of the
time the true parameters are outside of the specified confidence
region. A properly computed 95% confidence region, for example,
should contain the true parameters approximately 95% of the time.

Figure 3 shows the fraction of parameters outside a given confidence region for both the asymptotic analysis used by ASENT and the likelihood ratio analysis used by MuSig. This figure is for a sample size of 30 for the Bruceton, Langlie, and Neyer D-Optimal tests. The solid line in the figure is what a perfect analysis method would produce. For the group of lines using squares to denote plot points the upper line is the Langlie method, the next lower is the Neyer D-Optimal method and the next is the Bruceton. For the group using circles, the upper is Langlie, the next Neyer D-Optimal and the lower Bruceton.

**Figure 3: Comparison of
Confidence Likelihood Ratio versus ASENT**

The figure clearly shows that both of the analysis methods produce false confidence. For example, for a nominal 95% confidence region, the likelihood ratio test has the parameters outside of the confidence region approximately 8% of the time. While this is more than the 5% expected for a true 95% confidence region it is close to the requested confidence. Note that it would be prudent for the user of this information to specify a slightly more restrictive confidence (such as 97%) to achieve the required 95% confidence region.

The asymptotic 95% confidence region however, has the parameters outside of the confidence region approximately 20% of the time. To achieve a true 95% confidence region using this analysis method would require the computation of a confidence region greater than 99%.

First elements that are considered qualified when analyzed according to one analysis method could be unqualified when analyzed according to a more exact analysis method such as the likelihood ratio test. Thus, the end item user should either specify the analysis method or be consulted by the test facility as to options available. If a true 95% confidence region is required, then only analysis methods capable of producing a realistic confidence region should be used.