Wednesday, April 11, 2012

Credit Rating Agency Models and Open Source

When S&P downgraded the US from AAA to AA+, the US Treasury accused the rating agency of making a $2 trillion mathematical error. S&P initially denied this accusation, but adjusted some of its estimates in a subsequent press release. Economist John Taylor defended S&P, contending that its calculations were based on a defensible set of assumptions, and thus could not be categorized as a mistake. S&P’s model, which projected future debt-to-GDP ratios, has not been made public. As a result, it is difficult for outside observers to decide whom to believe: the rater or the rated.

There are at least three ways a model’s results can be wrong: if the model’s code itself doesn’t function as intended; if the known inputs are incorrectly entered, and if the assumptions are misapplied. In cases as important as the evaluation of US sovereign debt, we think rating agencies and the investing public would be better off if the relevant models were publicly available. Some may argue that the inputs to the models are proprietary or that they reflect qualitative assumptions valuable to the ratings agencies – i.e., that they are a “secret sauce.” But, even if rating agencies want to keep their assumptions proprietary, making the models themselves available would decrease the likelihood of rating errors arising from software defects.

Keeping one’s internal processes internal is the traditional way. Manufacturers assume that consumers don’t want to see how the sausages are made. In the internet era, it is now much easier to produce the intellectual equivalent of sausages in public – and, as it happens, many consumers are interested in the production process and even want to get involved. Wikipedia provides an excellent example of the open, collaborative production of intellectual content: articles are edited in public and the results are often subject to dispute. Writers get almost instantaneous peer review and the outcome is often rapid iteration moving toward the truth. In their books, Wikinomics and Macrowikinomics, Dan Tapscott and Anthony Williams suggest that Wikipedia’s mass collaboration style is the wave of the future for many industries – including computer software.

Many rating methodologies, especially in the area of structured finance, rely upon computer software. At the height of the last cycle, tools that implemented rating methodologies such as Moody’s CDOROMTM, were popular with both issuers and investors wondering how agencies might look at a given transaction. While the algorithms used by these programs are often well documented, the computer source code is usually not released into the public domain.

Over the last two decades, the software industry has seen a growing trend toward open source technology, in which all of a system’s underlying program code is made public. The best known example of open source system is Linux, a computer operating system used by most servers on the internet. Other examples of popular open source programs include Mozilla’s Firefox web browser, the WordPress content management system and the MySQL database.

In financial services, the Quantlib project has created a comprehensive open source framework for quantitative finance. The library, which has been available for more than 11 years, includes a wide array of engines for pricing options and other derivatives.

Open source allows users to see how programs work and with the help of developers, fully customize software to meet their specific needs. Open source communities such as those hosted on GitHub and SourceForge, enable users and programmers from all over the world to participate in the process of debugging and enhancing the software.

So how about credit rating methodologies? Open source seems especially appropriate for rating models. Rating agencies realize relatively little revenue from selling rating models; they are more likely to be used to facilitate revenue generation through issuer-paid ratings.

Open source enables a larger community to identify and fix bugs. If rating model source code were in the public domain, investors and issuers would have a greater chance to spot issues. Rating agencies would be prevented from covering up modeling errors by surreptitiously changing their methodologies. In 2008, The Financial Times reported that Moody’s errantly awarded Aaa credit ratings to a number of Constant Proportion Debt Obligations (CPDOs) due to a software glitch. The error was fixed, but the incorrectly rated securities were not immediately downgraded according to the FT report. Had the rating software been open source, it would not have been much more difficult to conceal this error, and it would have offered the possibility for a positive feedback loop – an investor or other interested party could have found and fixed the bug on Moody’s behalf.

Not only do open source rating models promote quality, they may also reduce litigation. The SEC issued Moody’s a Wells Notice in respect of the above mentioned CPDO issue, and may well have brought suit. (A Wells Notice is a notification of intent to recommend that the US government pursue enforcement proceedings, and is sent by regulators to a company or a person.) Investors have brought suit against the rating agencies to the extent they felt the ratings were inappropriate, for model-related errors or otherwise. By unveiling the black box, the rating agencies would be taking an active approach in buffering against litigation, and enjoy the material defense that, “yes we may have erred, but you were afforded the opportunity to catch our error – and didn’t.”

Unlike the CPDO model employed by Moody’s, the S&P US sovereign "model" likely took the form of a simple spreadsheet containing adjusted forecasts from the Congressional Budget Office. In contrast to the structured and corporate sectors, there are relatively few computer models for estimating sovereign and municipal default probabilities. While little modeling software is available for this sector, accurate modeling of government credit can be seen as a public good. Bond investors, policy makers and citizens themselves could all benefit from more systematic analysis of government solvency.

Open source communities are a private response to public goods problems: individuals collaborate to provide tools that might otherwise appear in the realm of licensed software. Thus open source government default models populated with crowd-sourced data maybe the best way to fill an apparent gap in the bond analytics market.

On May 2nd, PF2 will contribute an open source Public Sector Credit Framework, which is aimed at filling this analytical gap, while demonstrating how future rating models can be distributed and improved in an iterative, transparent manner. If you wish to participate in beta testing or learn more about this technology please contact us at info@pf2se.com, or call +1 212-797-0215.

--------------------------------------------
Contributed by PF2 consultant Marc Joffe. Marc previously researched and co-authored Kroll Bond Rating Agency’s Municipal Default Study. This posting is the second in a series of posts leading up to May 2nd. The prior piece can be accessed by clicking here.

45 comments:

Anonymous said...

OK, but some (most?) asset classes are rated using inside information provided to the ratings agency by the issuer. Given this, seems unlikely that an open source model is going to help much.

Marc Joffe said...

Thanks for this comment. You make a great point: a rating model may not be able to offset the benefit of having proprietary information.

The framework we are introducing is targeted at government debt issuers. In most Western countries, all of this disclosure is public so rating agencies don't have any proprietary information. I would also note that for corporates and even structured, very large amounts of public data are available - more than enough to run a credit model.

Eija Kulmala said...

I agree that this open source model give a lot of credit and it will very helpful.

Pak Learners said...
This comment has been removed by the author.
Anonymous said...

i m also agree

Carl said...

Open source can be helpful in rating models but in some point it must be also limited. Asset classes must be excluded in my own opinion. Thanks for sharing an informative post.

Unknown said...

Why S&P downgrades credit rating credit rating when it creates many problems regarding the overall economy and mostly in the capital market.

Credit said...

I liked what you said about Over the last two decades, the software industry has seen a growing trend toward open source technology
Follow this kind of articles because they are very good and useful, congratulations.

yatiken said...

Thank you for this post.its very informative...
Yatiken Software

Unknown said...

oro gold
School was designed to help answer your skin care questions.One thing we are really excited to introduce it OROpedia...
the OROGOLD Encyclopedia. Not only will it include a comprehensive list of OROGOLD ingredient but we will keep you updated on the latest scientific updates in the
skin care and cosmetics industry.

Unknown said...

Thanks a lot guys! What a great blog this is! It’s precious in favor of my knowledge. WWW.LISTACADEMYBONUS.NET

Unknown said...
This comment has been removed by a blog administrator.
Anonymous said...

I have spent a good time here at your blog reading. Must Keep the ball rolling buddies!quick cash loans

Anonymous said...

For a long time me & my friend were searching for informative blogs, but now I am on the right place guys, you have made a room in my heart! insurance rates

Unknown said...

Your website is terribly informative and your articles are wonderful.orogold

Unknown said...

Your website is terribly informative and your articles are wonderful.car insurance rates

Unknown said...

Only some of the posts are like that otherwise I don’t like most of the post. This is completely overwhelming. payday loans direct lenders

Unknown said...

I wonder why other professionals don’t notice your website much m glad I found this. Perlindungan Asuransi Kesehatan - Commonwealth Life

Unknown said...

If you should be opting for finest contents like me, just visit this blog site daily because it provides the feature contents, thanks.Undangan Pernikahan | Interior Rumah | Tas Murah

Unknown said...

If you should be opting for finest contents like me, just visit this blog site daily because it provides the feature contents, thanks.Undangan Pernikahan | Interior Rumah | Tas Murah

Anonymous said...

Hello there, just became aware of your blog through Google, and found that it is truly informative. I am gonna watch out for brussels. I will appreciate if you continue this in future. Numerous people will be benefited from your writing. Cheers!

pesan undangan unik
undangan
undangan kipas
souvenir pernikahan unik
contoh undangan pernikahan
undangan pernikahan
undangan nikah

Steven Buttler said...

Your articles make whole sense of every topic.payday loans

Unknown said...

Of course, what a superb site with wonderful posts, enlightening issues has been discussed I like it. payday direct lenders

Unknown said...

You have discusses incredible points that sounds good, keep up the great work. pay day loans

yadongbiz said...


Wow!!! It was really an Informational Article which provide me with much Insightful Information. I would like to first thank the author for such an Insightful Content. keep it up! thankyou Feel free to visit my website; 야설

japanyadongcom said...

Very informative and well-written post! The quite interesting and nice topic chosen for the post. thankyou Feel free to visit my website;
한국야동


koreayadongcom said...


I savour, lead to I found exactly what I used to be having a look for. You’ve ended my four day long hunt! God Bless you man. Have a nice day. Feel free to visit my website; 국산야동


chinayadongnet said...


I am really impressed with your blog article, such great & useful information you mentioned here. I have read all your posts and all are very informative. Thanks for sharing and keep it up like this. Feel free to visit my website; 일본야동



yahanvideonet said...

Thanks for sharing with us this important Content. I feel strongly about it and really enjoyed learning more about this topic. Feel free to visit my website; 일본야동

kckuldeepseo said...

Your article so unique and usefull and thanx for sharing this content.
Sexologist in Delhi

pachinkosite.info said...

Hi! this is often nice article you shared with great information. Thanks for giving such an exquisite informative information. 파칭코

blackjacksite.top said...

Your article is extremely attractive and interesting, hopefully more people will know and visit your blog. 바카라사이트

sportstototopcom said...

Wow! Such an amazing and helpful post this is. I really really love it. It's so good and so awesome. 토토사이트

Anonymous said...

الرجبي

Cast India said...

Amazing article, I strongly advise everyone to read as much as possible

Shikha Singh said...

Awesome article! I want people to know just how good this information is in your article.

PerYourHealth.com said...

Peryourhealth has an automated system that takes care of everything from collecting the patient’s information to sending the bill to the insurance provider for authorization and finally paying it out to the patient.
List of Website @
https://peryourhealthplan.com/
https://per-your-health.online/
https://sites.google.com/view/www-peryourhealth--com/home

hastenchemical said...

fly ash Oklahoma serves as a valuable resource for sustainable construction practices. As a byproduct of coal combustion, it is commonly used as a supplementary cementitious material in concrete production. By enhancing the strength, durability, and workability of concrete, fly ash contributes to the construction of resilient infrastructure across Oklahoma's diverse landscapes.

Cmolds Creativity said...

Cmolds is a leading app design and development company, creating innovative solutions that cater to your business needs, driving growth and user engagement. Elevate your digital presence with Cmolds' expertise in crafting exceptional mobile applications.

martin said...

Thank you for being an amazing part of my online journey. Your encouragement inspires me to keep sharing and creating.
Tipos de Divorcio en Estado de Nueva York

john stephen said...

Abogado Conducción Imprudente Nueva Jersey
"Credit Rating Agency Models and Open Source" is a comprehensive book that provides an in-depth understanding of credit rating models and their potential benefits. It offers insights into the evolving landscape of credit rating agencies, emphasizing transparency and accountability. The book includes practical examples and case studies, making it accessible to professionals and enthusiasts in the field. Overall, it is a highly informative resource for understanding the intersection of credit ratings and open-source technology.

abigailuna said...

The book delves into the intricate world of credit assessments, revealing the opaque processes of credit rating agencies and incorporating open-source methodologies, providing transparency and deeper understanding for finance enthusiasts and professionals. Mejor Abogado Divorcios en Nueva York

Stelena675 said...

Experience unparalleled luxury with our Cabo vacation rentals, offering exquisite accommodations that redefine opulence. Elevate your getaway with our exclusive cabo house rentals with chef services, where every moment is a blend of comfort, style, and culinary delight, ensuring an unforgettable escape in the heart of paradise.

Anonymous said...

I appreciate you sharing this helpful information. Please continue posting.Abogado de DUI Condado de Prince William

Monster Rabbit said...


Experience the power of Epimedium sex honey with Monster Rabbit, the leading provider in UAE. Our premium quality product is crafted to enhance your intimate experiences and boost vitality. Discover the ultimate pleasure with Monster Rabbit's Epimedium sex honey in UAE.