In order to develop a tailings risk assessment method comparison, we will start by reviewing a number of existing alternatives to ORE2_Tailings™, discussing what we call the black box objection and examining the ORE2_Tailings procedure, algorithm and results.
Alternatives to ORE2_Tailings
We have grouped possible alternatives into four families discussed below.
Failure Modes and Effects Analysis
Failure modes and effects analysis (FMEA) is not a quantitative risk assessment even if it uses indices like probability = 1, 2 or… n., and neither is it quantitative if the cell limits have arbitrary “probability lookalike” numbers such as 0.01, etc. In FMEA, it is impossible to consider the joint effect (causality) of various failure modes without some mental gymnastics, though one can implicitly include management, care and other contributing aspects in the “binning” exercise. Researchers demonstrated that the coloring of the cell is “disjointed” from reality and public expectations.
Probabilistic Slope Analyses
There is an ample body of literature (for instance Christian & Baecher (2011)) that shows that this approach delivers excessively high probabilities of failure. In addition, it ignores management, care and many other contributing aspects (see Oboni & Oboni 2020, The Factor of Safety and Probability of Failure relationship, TMW 2020).
Silva Lamb Marr Semiempirical Approach
We group two papers under this heading: Silva et al. (2008) (with the updated graphs in its comments) and Altarejos-García et al. (2015). We refer to the approach as SLM due to the author surnames of the 2008 paper, Silva, Lamb and Marr. Engineers mostly use SLM (oftentimes with the outdated graphs) and the difference between SLM and the Altarejos approach is in any case minimal.
Both papers display a very clever set of curves based on a limited set of 75 examples. Among these, some were indeed dam slopes and some were retaining structures or other slopes. The empirical curves link the quality of the slope to the factor of safety (FoS) for stability and deliver the annual probability of failure. The paper does not indicate which deterministic FoS to use among the multiple generally available.
We insist on the term “slope” because SLM does not include any provision for pipelines at crest or ancillary water management typical of dams. Indeed, SLM is intended as a slope analysis despite the keywords indicating dams. SLM cites, however, the possibility of using effective stress or undrained strength. The paper mentions the need to expand the analysis to other failure modes but provides no solution.
Furthermore, the curves allow the user to pick probabilities of failure that are beyond credibility (less than 10-6 or one in a million). These may apply to top notch modern hydraulic structures in mint condition, but certainly not to tailings dams (see Rana et al. (2022)). Indeed, we demonstrated in 2013 that the factual probability of catastrophic failure of nuclear reactors was hovering around 10-4 (or one in ten thousand), and that is despite theoretical values at less than 10-7 (or one in ten million) at design stage.
We have used SLM in some papers where we needed a simple approach for an example, or only in very specific (and rare) cases where the dam can be considered a slope.
In our 2020 book Tailings Dam Management for the Twenty-First Century, we clearly show the limits of SLM for dams. Furthermore, we illustrated the need to anchor the results of this type of approach to a wide array of cases. We did that with ORE2_Tailings using 100 years of history as discussed later in this text.
Chowan et al. 2021
Chowan et al. (2021) arbitrarily modify some aspects of SLM. However, they do not offer any justified anchoring to reality beyond the perception of the parties involved. The approach also encounters the same difficulties and misleading elements of SLM. For instance, it uses one “magic” FoS for a dam (see Table 7 of the paper). It is unclear if they apply that factor to a single dam or an entire tailings storage facility (TSF).
They offer examples of application, but the only “validation” is the comparison to the 2014 Mount Polley failure after the fact, a case we also covered in our book.
Furthermore, their approach only peripherally and implicitly alludes to water management, but falls short of including its actual engineering design criteria.
Blended FTA Approaches
These use failure tree analysis (FTA) to analyze each failure mode and either probabilistic slope analyses or SLM to obtain a blended probability of failure. This is a way to bypass the limitations discussed above.
The probabilities in the FTA are also subject to many black box biases. We explained this approach in our book, in Section 1.2: Linking FoS to Pf: Three Simplified Methods. This was the method a major consulting company used for their QRA. They compared their results with the ORE2_Tailings deployment. We were very pleased to see that the results were astonishingly similar, but ORE2_Tailings was much faster.
None of the above allows for any interdependency to be included in the analyses with other HDS, TSFs or external hazards.
The Black Box Objections to the Five Alternatives
In this section, we go back to each alternative and discuss what we call the “black box objection” that we oftentimes encounter.
FMEA
Despite its apparent transparency and simplicity, there are numerous hidden “tricks” that may bias the results (Thomas et al. 2014, just to quote one). We explained this in our book.
Probabilistic Slope Analyses
One can think probabilistic slope analysis is transparent, but that is not the case. No software vendor explains what happens “inside the algorithm.” Furthermore, if an end user performs a Monte Carlo simulation, the intrinsic assumptions lead to high variability of the results.
Silva Lamb Marr
Silva et al. (2008) based their set of curves on a limited set of examples. The authors state they used expert judgement to obtain “relative estimates” (among that sample) of the slope stability pf. They did not specify the nature and type of the dam material and cross-sections. In addition, not all of the 75 structures were dams. We can’t be certain of whether the dams of the set were hydraulic dams or of the cross-section types.
Chowan et al.
Engineers around the world (including Canada and Latin America) have come to us with objections and questions on this method. We report a sample of these below. It is important to note that the objections denote, in some cases, a hazardous misunderstanding of the method. The cause seems due in part to the imprecise glossary and unclear mathematics used in the paper.
Notes by Users
- It is suggested in the introduction to use a FoS relative to the level of uncertainty. However, no guidance is offered and the example they give shows “one number” FoS with no justification.
- Silva et al. (2008) presents a figure with curves that stops at FoS of +/- 2.2. This makes more sense than their figure which has curves with a FoS of 3. In addition, an inadequately constructed dam remains a high-risk structure, regardless of the FoS. (NB: this last remark is due to our correspondent misunderstanding the aim of the paper).
- The use of words such as: “considered tolerable” tickles me. Because per se it does not mean anything “measurable” in the paper.
- They talk about “excessive seepage.” But that doesn’t mean anything as some dykes are designed to have seepage and others aren’t!
- Table 2 is actually based on what the Quebec government is doing to assess the class of hydraulic dams. The application of such a recipe is debatable. And, I believe, should not be presented as “the solution” but rather as a list of questions to assist in risk assessment.
- Table 3 is not very convincing.
- It is written that “…dam classification can be used to complete this task” (p. 9). How can the classification of a dam help establish Annual Probability of Failure?
- One should not allow wording such as: “…would eliminate the possibility…” in this type of paper.
- Mixing lifespan probability of failure and annual probability of failure seems to me a risky approach. Because, in the end, the TSF must be efficient for an indefinite period. However, here I can be wrong, because my knowledge of statistics is limited.
Blended Approach
The blended approach inherits all the “black box” aspects of the single approaches it uses, described above.
ORE2_Tailings Algorithm
The ORE2_Tailings algorithm converts various sets of data (verbiage and factual data) into a set of proxy variables (numbers). The algorithm combines them mathematically to deliver the annualized probability of failure of the dam “body” in various conditions (e.g. drained, undrained, seismic, liquefaction, etc.) pertinent to the site, studied by the engineers.
In addition, ORE2_Tailings evaluates the causalities of the failure. As a result, it can propose a series of risk-informed control enhancements.
Simultaneously to the dam “body” analysis, the algorithm considers the ancillary water management and pipelines’ potential failures (if applicable) and external erosion potential (river, lake or sea/ocean). This analysis is rather complex as:
- Malfunctioning of the diversions and water balance mismanagement may lead to overtopping.
- Malfunctioning of weirs and spillways may lead to scouring and ultimately failure.
- External water courses, water bodies and sea/ocean may generate toe ablation which can lead to liquefaction, etc.
- Penstock and other elements may lead to hazardous hydraulic conditions.
The presence of active pipes (and possibly traffic) at the crown further complicates this side of the analysis.
Indeed, the two sides (dam “body” and ancillary water management) of the system interact in various ways due to potential for overtopping, downstream erosion, toe ablation and other factors.
The math involved in the algorithm are linear interpolations, series and parallel system reliability equations and statistical functions.
ORE2_Tailings Black Box Objections
As per the black box objection we hear against ORE2_Tailings, we can give the following two replies:
- The methodology is no more a black box than the other commonly accepted misleading methods discussed earlier. However, its results reflect the world-wide portfolio behavior (see below).
- The information we deliver is enough to understand how the algorithm works without delving into the subtleties of its structure. The development of ORE2_Tailigs required decades to formulate and calibrate. Civil and geotechnical engineering is full of examples of methodologies that are known only in principle, such as finite element analyses, sophisticated 3D stability analyses and break analyses. Indeed, no one can reproduce their results “by hand;” the same can be said of ORE2_Tailings.
For instance, when asking consultants to perform dam break analysis, engineers understand the general idea behind the code, but the computer code (often proprietary) is not available for review. This is also the case for ORE2_Tailings.
ORE2_Tailings Experience, Results and Anchoring to Reality
Finally, Riskope’s experience comes from hundreds of HDSs. Thus, we can show the result of ORE2_Tailings on hundreds of dams all over the world, anchored to reality by our benchmarks, as visible in the graph below.
The HDSs we used to prepare the TSF summary in the figure belong to an array of mining companies. They are active/inactive/closed, with upstream, centerline, and downstream/rockfill methods. The owners/operators asked Riskope to deploy ORE2_Tailings. As one can see from the graph, in some cases, ORE2_Tailings confirmed the concerns. However, there were also cases that we could dismiss, even among this worrisome sample of dams.
Note that a great majority of the TSFs in this worldwide portfolio hovered around or within the benchmark lines. Thus, they reflected the results by Rana et al. (2022).
As a matter of fact, the portfolio depicted in the graph predicts a little less than three catastrophic failures per year, if we consider our sample to be representative of the worldwide portfolio. The number of yearly catastrophic failures around the world is slightly above three on long-term average. Hence, we can infer that our sample portfolio is marginally better than the worldwide one.
This rests the discussion of the representativeness of the ORE2_Tailings results. Indeed, it shows they reflect historical reality thanks to years of calibration and observation.
Closing Remark on Tailings Risk Assessment Methods Comparison
We consider the very act of undertaking a rational and serious risk assessment a very important step towards risk reduction.
Clients asking for a study of their dam foster enhanced risk awareness, care and understanding.
However, one should remember the limitations and approximations of each method and ensure that the results are anchored to reality.
