Bank Stress Tests and the Problem of Ignoring Reality

“Too large a proportion of recent "mathematical" economics are mere concoctions, as imprecise as the initial assumptions they rest on, which allow the author to lose sight of the complexities and interdependencies of the real world in a maze of pretentious and unhelpful symbols.”

                                                                                        - John Maynard Keynes

The 2011 European Bank stress tests were largely held in disregard.  They had managed to assume away the implications of a chief risk held by the banks being tested – that countries within the EU could default – culminating in several banks easily passing the tests, only to fail soon thereafter.

The results were released in July 2011, with Dexia and Bankia and the Cypriot banks passing and sometimes easily passing the tests. Dexia failed in October 2011. Bankia survived a little while longer, before being nationalized in May 2012. The Cypriot banks never triggered any kind of concerns among the key monitoring agencies, the EU, EBA, IMF or BIS, well, not before the Cypriot banking collapse.

The editorial board at Bloomberg View just put out a piece on why the US Fed's bank tests lack credibility.  Same problem, here: a lack of basis in reality:

"...the simulation [being run] is a far cry from what happens in a real crisis. It doesn't fully capture how contagion can afflict many of a bank's counterparties at once, magnifying losses many times over. It also assumes that a thin minimum layer of equity capital -- just $4 per $100 in assets -- would be enough to maintain the market's confidence in a bank's solvency. These flaws make a passing grade almost meaningless."

Reality is very different, and modeling behavior in a stressed environment is necessarily a different process from modeling a normal environment, as what was previously uncorrelated or even inversely correlated can suddenly become correlated ... as the economic principles break down and legal rules change.

We're not saying this is easy – but there's little comfort to be gained in performing a test if that test fails to capture the harsh reality that, in times of crisis, our (joint) behavior itself will compromise the predictive value of the theoretical process we're modeling.  

Perhaps a picture will say it best: