Wednesday, April 11, 2012

Credit Rating Agency Models and Open Source

When S&P downgraded the US from AAA to AA+, the US Treasury accused the rating agency of making a $2 trillion mathematical error. S&P initially denied this accusation, but adjusted some of its estimates in a subsequent press release. Economist John Taylor defended S&P, contending that its calculations were based on a defensible set of assumptions, and thus could not be categorized as a mistake. S&P’s model, which projected future debt-to-GDP ratios, has not been made public. As a result, it is difficult for outside observers to decide whom to believe: the rater or the rated.

There are at least three ways a model’s results can be wrong: if the model’s code itself doesn’t function as intended; if the known inputs are incorrectly entered, and if the assumptions are misapplied. In cases as important as the evaluation of US sovereign debt, we think rating agencies and the investing public would be better off if the relevant models were publicly available. Some may argue that the inputs to the models are proprietary or that they reflect qualitative assumptions valuable to the ratings agencies – i.e., that they are a “secret sauce.” But, even if rating agencies want to keep their assumptions proprietary, making the models themselves available would decrease the likelihood of rating errors arising from software defects.

Keeping one’s internal processes internal is the traditional way. Manufacturers assume that consumers don’t want to see how the sausages are made. In the internet era, it is now much easier to produce the intellectual equivalent of sausages in public – and, as it happens, many consumers are interested in the production process and even want to get involved. Wikipedia provides an excellent example of the open, collaborative production of intellectual content: articles are edited in public and the results are often subject to dispute. Writers get almost instantaneous peer review and the outcome is often rapid iteration moving toward the truth. In their books, Wikinomics and Macrowikinomics, Dan Tapscott and Anthony Williams suggest that Wikipedia’s mass collaboration style is the wave of the future for many industries – including computer software.

Many rating methodologies, especially in the area of structured finance, rely upon computer software. At the height of the last cycle, tools that implemented rating methodologies such as Moody’s CDOROMTM, were popular with both issuers and investors wondering how agencies might look at a given transaction. While the algorithms used by these programs are often well documented, the computer source code is usually not released into the public domain.

Over the last two decades, the software industry has seen a growing trend toward open source technology, in which all of a system’s underlying program code is made public. The best known example of open source system is Linux, a computer operating system used by most servers on the internet. Other examples of popular open source programs include Mozilla’s Firefox web browser, the WordPress content management system and the MySQL database.

In financial services, the Quantlib project has created a comprehensive open source framework for quantitative finance. The library, which has been available for more than 11 years, includes a wide array of engines for pricing options and other derivatives.

Open source allows users to see how programs work and with the help of developers, fully customize software to meet their specific needs. Open source communities such as those hosted on GitHub and SourceForge, enable users and programmers from all over the world to participate in the process of debugging and enhancing the software.

So how about credit rating methodologies? Open source seems especially appropriate for rating models. Rating agencies realize relatively little revenue from selling rating models; they are more likely to be used to facilitate revenue generation through issuer-paid ratings.

Open source enables a larger community to identify and fix bugs. If rating model source code were in the public domain, investors and issuers would have a greater chance to spot issues. Rating agencies would be prevented from covering up modeling errors by surreptitiously changing their methodologies. In 2008, The Financial Times reported that Moody’s errantly awarded Aaa credit ratings to a number of Constant Proportion Debt Obligations (CPDOs) due to a software glitch. The error was fixed, but the incorrectly rated securities were not immediately downgraded according to the FT report. Had the rating software been open source, it would not have been much more difficult to conceal this error, and it would have offered the possibility for a positive feedback loop – an investor or other interested party could have found and fixed the bug on Moody’s behalf.

Not only do open source rating models promote quality, they may also reduce litigation. The SEC issued Moody’s a Wells Notice in respect of the above mentioned CPDO issue, and may well have brought suit. (A Wells Notice is a notification of intent to recommend that the US government pursue enforcement proceedings, and is sent by regulators to a company or a person.) Investors have brought suit against the rating agencies to the extent they felt the ratings were inappropriate, for model-related errors or otherwise. By unveiling the black box, the rating agencies would be taking an active approach in buffering against litigation, and enjoy the material defense that, “yes we may have erred, but you were afforded the opportunity to catch our error – and didn’t.”

Unlike the CPDO model employed by Moody’s, the S&P US sovereign "model" likely took the form of a simple spreadsheet containing adjusted forecasts from the Congressional Budget Office. In contrast to the structured and corporate sectors, there are relatively few computer models for estimating sovereign and municipal default probabilities. While little modeling software is available for this sector, accurate modeling of government credit can be seen as a public good. Bond investors, policy makers and citizens themselves could all benefit from more systematic analysis of government solvency.

Open source communities are a private response to public goods problems: individuals collaborate to provide tools that might otherwise appear in the realm of licensed software. Thus open source government default models populated with crowd-sourced data maybe the best way to fill an apparent gap in the bond analytics market.

On May 2nd, PF2 will contribute an open source Public Sector Credit Framework, which is aimed at filling this analytical gap, while demonstrating how future rating models can be distributed and improved in an iterative, transparent manner. If you wish to participate in beta testing or learn more about this technology please contact us at, or call +1 212-797-0215.

Contributed by PF2 consultant Marc Joffe. Marc previously researched and co-authored Kroll Bond Rating Agency’s Municipal Default Study. This posting is the second in a series of posts leading up to May 2nd. The prior piece can be accessed by clicking here.


Anonymous said...

OK, but some (most?) asset classes are rated using inside information provided to the ratings agency by the issuer. Given this, seems unlikely that an open source model is going to help much.

Marc Joffe said...

Thanks for this comment. You make a great point: a rating model may not be able to offset the benefit of having proprietary information.

The framework we are introducing is targeted at government debt issuers. In most Western countries, all of this disclosure is public so rating agencies don't have any proprietary information. I would also note that for corporates and even structured, very large amounts of public data are available - more than enough to run a credit model.

Eija Kulmala said...

I agree that this open source model give a lot of credit and it will very helpful.

14mumta said...
This comment has been removed by the author.
Anonymous said...

i m also agree

Carl said...

Open source can be helpful in rating models but in some point it must be also limited. Asset classes must be excluded in my own opinion. Thanks for sharing an informative post.

Sarah Ahmed said...

Why S&P downgrades credit rating credit rating when it creates many problems regarding the overall economy and mostly in the capital market.

Credit said...

I liked what you said about Over the last two decades, the software industry has seen a growing trend toward open source technology
Follow this kind of articles because they are very good and useful, congratulations.

Dexter jazz said...

The information about the Debt is pretty amazing and the article is also pretty amazing. I want to read this information with my friends. Thanks for sharing this article.
Debt Collection Singapore

Thomas Edison said...

congratulations guys, quality information you have given!!! review

Albert einstien said...

I love all details that you give in your articles.
company no win no fee

James Bond Bond said...

Your blogs and accretion attracts me to come back once more on your site.
best credit repair services

Albert einstien said...

Waooow!! Nice blog, this will be greatly helpful.
credit repair

Eva Karea said...

Your articles make whole sense of every topic.
lexington law reviews

julia bandth said...

This is an informative blog by which I have got that info which I really wanted to get. personalcashadvance

Albert einstien said...

I want to say thank to you people for this great and helpful info. Thanks!!!
pay advance loan

hely fell said...

It's been good to see your blog when I always look for such type of blogs. It’s great to discover the post here.
credit repair companies

Albert einstien said...

Amiable articles and the blogs really helped me a lot, thanks for the valuable information.
easy faxless

Albert einstien said...

It’s a classic great for me to go to this blog site, it offers helpful suggestions
payday advance

Jennifer Anderson said...

This is really pretty cool place I like it because it has everything I want more on this blog soon.

credit repair reviews

ashlie jon said...

It's been good to see your blog when I always look for such type of blogs. It’s great to discover the post here.
options income engine review

Albert einstien said...

The information you have shared with us is really so great, nice effort!! free credit reports from all 3 bureaus

grady william said...

You have discusses incredible points that sounds good, keep up the great work. professional networking