A Telling Tale of Two Tables

Just how difficult is it to measure ratings performance, and how useful are the measures, really?

There are certain difficulties we all know about – having to rely on the ratings data being given to you by the parties whose performance you’re measuring. So there is naaturally the potential for the sample to be biased or slanted, and that’s very difficult to uncover. (We’ve discussed this hurdle at great length here.)

The next issue we’ve written about is the inability to separate the defaulting assets from those that didn’t default. In our regulatory submission, we called for transparency as to what happened to each security BEFORE its ratings was withdrawn (see Transparency of Ratings Performance).

Let’s have a look at a stunning example today that brings together a few of the challenges, and leaves one with a few unanswered questions as to the meaningfulness of ratings performance, as it's being currently displayed, and the incentives rating agencies have to update their own ratings.

Here’s a Bloomberg screenshot for the rating actions on deal Stack 2006-1, tranche P.


Starting from the bottom up, it seems Moody’s first rated this bond in 2006 whereas Standard & Poor’s first rated it in 2007. If we try to verify this on S&P’s website for history, we come to realize how difficult it can be to verify:


So let’s suppose everything about the Bloomberg chart is accurate.

As verified on their website, Moody’s rated this bond Aaa in August ’06 and acted no further on this bond until it withdrew the rating in June 2010. (They don’t note whether the bond paid off in full, or whether it defaulted.)

S&P, meanwhile, shows its original AAA rating of 2007 being downgraded in 2008 (to BBB- and then to B-) and in 2009 to CC and again in 2010 to D, which means it defaulted (according to S&P).

For Moody’s, the first year for which the bond remains in the sample for a full year is 2007; thus, it wouldn’t be included in 2006 performance data. For S&P, the first complete year is 2008.

So if we consider how Moody’s would demonstrate its ratings performance on this bond, it would say:

Year 2007: Aaa remains Aaa
Year 2008: Aaa remains Aaa
Year 2009: Aaa remains Aaa
Year 2010: Aaa is withdrawn (WR)

No downgrades took place, according to Moody’s … while at the same time S&P shows it as having defaulted:

Year 2008: AAA downgraded to B-
Year 2009: B- downgraded to CC
Year 2010: CC downgraded to D

Here’s what their respective performance would look like, if one were to apply their procedures (at least as far as we understand them):