‘It is a feature of many systems of thought, and not only primitive ones, that they possess a self-confirming character. Once their initial premises are accepted, no subsequent discovery will shake the believer’s faith, for he can explain it away in terms of the existing system. Neither will his convictions be weakened by the failure of some accepted ritual to accomplish its desired end, for this too can be accounted for. Such systems of belief possess a resilience which makes them virtually immune to external argument.’
Keith Thomas, ‘Religion and the Decline of Magic’
In this essay I am going to outline Standard & Poor’s (S&P) approach to rating sovereign risk and offer my thoughts on potential limitations and weaknesses. I will suggest two alternative and I think complementary approaches to rating sovereign risk that in addition to S&P’s current approach would I believe provide a more comprehensive assessment of sovereign risk.
The essay is divided into two parts. In part one I will attempt an unadulterated explanation of S&P’s current approach and the rationale for doing it this way. In part two, I will offer my criticisms of their approach and suggestions on how to improve it as well include ideas on what would convince me that I’m wrong.
Part One: S&P’s approach to Sovereign Credit Ratings
Sovereign credit ratings are opinions on the future ability and willingness of sovereign governments to service their debt obligations to the market on time and in full. On time and in full is important because no attempt is made to try and predict the exact nature or extent of default. The reasoning is that default is an extreme event (with an average of roughly one a year in the last fifteen years on S&P rated countries – which is pretty much everyone at this point). Default is so extreme that predictions on if it happens, rather than specifically how, are sufficient. Willingness to pay is the crucial quality that separates sovereigns from the usual companies and organizations S&P rates because companies have clear and immediate legal repercussions for not servicing their debt whereas sovereigns face much less clearly defined economic and political costs.
Although it is not explicitly stated, forward-looking estimates should be at least a year and it was explained to me that non-investment grade have a two-year time line and investment grade has 4-5 years. Technically the ratings are not absolute because they are not tied to any specific underlying metrics. And in fact, pre-1975 the ratings were primarily done through peer comparisons before a more formal framework was put in place. However, it is not correct to say they are purely rankings or that they are fit to a curve because although of course they seek to be, in each time period, internally consistent and offer an accurate measure of relative credit worthiness, they should also be (at least since 1975 when modern ratings methods were put in place) fairly consistent over time and different classes or organizations. Ratings are offered for both local currency and foreign currency debt; the latter is of greater interest because it offers easier international comparison. There is also the more mechanical reason that foreign is calculated first and local is usually just an uptick on the foreign view.
Because overall creditworthiness is a function of both political and economic risks S&P’s rating approach is necessarily both quantitative and qualitative. Qualitative approaches are particularly necessary when assessing willingness to pay. In all there are five key criteria that are considered when rating sovereign debt:
Economic structure and growth prospects; Political institutions and considerations; Government budget considerations (fiscal); Monetary flexibility; External liquidity.
Economic structure and growth concerns the underlying economy and ultimately the tax base that the government of a country can draw from. Stronger underlying economies make for more resilient governments. Political institutions and considerations concerns both stability and transparency issues. Generally the more stable and transparent (which often correlates well with western democracies) the more reliably you can expect countries to pay off their debt. Government budget considerations assess factors relating to the government’s balance sheet. Monetary flexibility assesses the effectiveness and availability of monetary tools whilst external liquidity assesses the impact of balance of payments constraints.
These five factors are rated on a scale of one to six where one is the strongest six the weakest. These factors are combined into two averages. One is a rating of the overall health of the sovereign which takes an average of the economic structure and growth prospects score with the political institutions and considerations score to make the Institutional and governance effectiveness and overall profile score. The remaining three scores for fiscal, monetary and external are also averaged to create the flexibility and performance profile which represents the country’s ability to react to shocks. By averaging this way the result is slightly lower weights for the external, fiscal and monetary scores. These two profiles are then mapped onto a grid where bands of diagonal equivalence formulaically determine the final credit rating. The specific weights and the boundaries of the bands seem to have been chosen at the current levels primarily for legacy and arithmetical convenience but there is no obvious first-glance reason to suggest the weights are significantly off. Clearly the obvious advantage of having a systematic approach to weighing the different factors is that it makes the ratings comparable across countries even though arguably there may be some country to country variations in the relative importance of each factor. All in all, there are 18 different ratings ranging from the best AAA to the worst CC. In addition to each rating an outlook is published which can be positive, negative or stable. There are also, for very extreme events, credit watch outlooks if there is scope for a rating to turn on an upcoming event such as an appeal to a court. There are no first principle reasons for why the number of different ratings are set at 18 specifically but the main thinking behind the relatively high number is that, often times, organization classes tend to clump around a certain set of ratings. Therefore by providing a relatively high number of ratings there is more scope to offer differentiation within each organization class (whether sovereign, university, company, supranationals etc).
Part Two: Alternative and Complementary approaches to Sovereign Credit Ratings
In this section I wanted to fight the temptation of accepting the general approach and just nitpicking within it in favour of trying to ask if there are any fundamentally different approaches that could be made to rating sovereigns. My conclusion was I think there are and that rather than replace they could potentially supplement and complement S&P’s current approach.
I have been fortunate enough to sit on a large variety of credit committees including sovereigns, banking and even a university. The experience was immensely valuable and really brought the ratings criteria to life and my general impression was that I was very impressed with both the level of knowledge that each of the analysts had about not only the sovereigns they covered themselves but also the highly intelligent and diverse set of questions they asked each other. Of course, all the discussions are strictly confidential and I shall respect that here but nonetheless I will start this section by making generalized impressions about their approach, which I accept are only my perceptions, but nonetheless despite this might still perhaps have some grain of truth to them.
The problem of using the model that higher general health means a bigger buffer against disease approach
My overall sense of S&P’s approach to rating sovereigns is that although perhaps the exogenous shock that pushes a country over the edge is largely unpredictable there are long-run build ups of poor fiscal standing, exposure to foreign markets, increased political instability which are completely predictable. In fact it was explained to me that there are no ‘fat tail’ defaults; everything is in the realm of prediction. Therefore the general approach is to try to assess the state of the sovereign with the idea that sovereigns in stronger positions have greater buffers to endure the inevitable shocks that come along. By analogy, if a sovereign was a person and default represented death they try and assess a person’s general health and therefore ability to withstand the inevitable barrage of diseases that the person will come in contact with in the course of a lifetime.
My primary issue with this approach is that the exogenous shocks are treated as completely unpredictable. Returning to my analogy, S&P’s approach is to keep track of all the factors that might affect a person’s health from cholesterol, to blood pressure, to liver function etc. and then from this develop a generalized view of their health (and therefore the size of their generalized buffer) against a generalized disease. Instead, I think there should be some thinking on specific diseases and each person’s unique exposure to and risk of each disease.. With the human analogy you would take the main causes of death, from cancer, to heart attack and stroke, and try to assess each person’s individual risk of each separate cause of death. This is important I think because I expect there is probably a high degree of heterogeneity among sovereigns in not only causes of default but also therefore exposure to the different types of default. Person A may have no chance of dying from lung cancer but have very risk of dying from a heart attack. To then take an average of the high risk of a heart attack and the low risk of lung cancer and say the person has an average risk of dying I think does not fully communicate the true risk of death. I suspect S&P would argue that causes of default are not independent and you would probably need several to occur simultaneously rather than just one. Nonetheless having sat in the credit committees I cannot help but feel that the ratings, especially with the extreme attention paid to adjusting them up and down the 17 notches, are more an assessment of the general health of the economy rather than a forward looking prediction about actual default. Using the human analogy I feel that a person who eats 5 fruits a day would probably get a better health rating, using the S&P methodology but the actual relationship of the number of fruits you eat a day and whether you are going to die soon is not clearly established and essentially there is an underlying assumption that better health, no matter how marginal, means lower risk of death.
The problem with short-run trend from equilibrium + shock analysis
The second problem I think is the short-term nature of the ratings. Long-term ratings on sovereigns are important to investors because sovereign bonds, for example, are often five even ten years in length. I concede that no official position is taken on the actual timeline of the ratings but the general agreement is that should be around the three to five years for investment grade and two to three for non-investment grade and certainly have scope of greater than a year. Having listened to the rating committee discussions though, the feeling is that they are six month or perhaps very generously one year predictions. In the admittedly small number of meetings I have attended rarely did anyone ever try to make predictions beyond six months except to say, for example, that there is political risk and who the hell knows what is going to happen and that that uncertainty should be reflected in a notch down in the ratings as a sort of safety margin. One exception to this would be clear inflection points around elections etc. but in general I heard no attempt to make predictions on worldwide trends and how they make effect a specific sovereign. The obvious rationale for this is that any predictions beyond one year would quickly become highly speculative so instead the approach is to give a, in my opinion, very accurate picture of the country’s position today with buffers for uncertainties. Recently in an attempt to add stability to the system new regulations (although I’m not sure if this is for all countries or just for Europe) were introduced to only allow ratings agencies, outside of exceptional circumstances, to change their ratings every six months and at predictable times so that the market has stable expectations. The resulting approach, I think, is that the ratings end up tracking and describing the risk of default rather than actually predicting it. In fact, in reading the research most trend predictions would take the form of change in A has been caused by B. Perhaps it would be beneficial to firstly establish more concretely (probably through statistics) the B explanation and secondly if it held up to go further with the predictions by predicting changes in A with predictions about changes in B.
Of course, the question is how can you predict something which inherently, is probably, quite unpredictable. One of the advantages I think of the six month to one year approach is that you can basically use short-run trend from equilibrium + shock analysis. Over short time scales trends have a very high predictive power especially when coupled with expectations of short-run shocks and their likely effects on bumping up or bumping down the trend. The resulting analysis therefore feels much less based on underlying macroeconomic theory or a view on fundamental dynamics but rather just a short-run extrapolation into the future. The problem, of course, is that sovereign risk is not so short-term, especially if you are signed up to a bond say, that is of ten year or longer time scales.
Now I should caveat all these comments with the reminder that the realm of macroeconomic predictions is a graveyard of great economic thinkers and for that reason I think the short-run trend from equilibrium + shocks approach that S&P takes should actually be the primary ratings method. But rather than implying to the market that these are actually mid to long-run predictions I think it should be more explicitly stated it is a short-run view on the health of the sovereign which in normal circumstances should be very highly correlated with long-run default risk.
With that short-run approach as the meat and vegetables of the rating I think there could then be scope for longer-run more speculative predictions. These would, as I outlined in my piece (available on my blog) on Frodo Risk, involve trying to categorize default risk into different and distinct narratives. And just like you would separately evaluate the risk of cancer, heart attack etc. rather than try to rate a general risk of death you would, for sovereigns, separately analyze the long-run risk of the different default patterns. You might still want to still have one overall rating because ultimately the market does not really care on how a default comes about just if it does but I think starting with a consideration of the distinct causes and then taking some kind of average is superior to a purely general approach. Thus the research on distinct causes of default could involve speculative predictions about global trends and how these might affect the different sovereign countries. Currently S&P’s approach is to assume these global risks are largely unpredictable, which given that no one in positions of influence in academia, business or politics predicted the Great Recession of 2008 is perhaps a reasonable assumption, and treat them like exogenous shocks. Nonetheless even if the long-run analysis just involved a tree-diagram of different scenarios and how each would one effect risk of default I still think that would be valuable resource for investors.
I also think there could be scope to try to use macroeconomic (and perhaps econometric) models to make long-term predictions for potential exogenous shocks and how sovereigns will be, in all likelihood differently, affected. Finally, I understand the aversion to econometric modelling and its related tools, particularly when it comes to the actual ratings process, but I think a small team of motivated econometricians could uncover a wealth of relationships and rules of thumb that could help anchor the sovereign analysts’ considerable knowledge and intuition to fundamentals. It might also ensure that there are no biases or false intuitions. This would be particularly valuable in examining assumptions about what matters in terms of actual default risk. I would concede that the relatively small sample size and number of defaults, at least when compared to say company ratings, would somewhat restrict the scope and power of statistical methods but nonetheless I still think it would be immensely valuable. I have no strong feelings about having 17 notches although I suppose if there was some first principles approach that could be used in deciding the classifications that might be valuable also.
What would convince me that I am wrong?
On the general health point I suppose that if it was shown that you generally need all the averaged factors S&P uses to go bad to have a default and that there is a very strong relationship between general health (as a buffer) and risk of default I think then perhaps the current approach would be justified and sufficient. Also if defaults either followed completely different patterns every time or the same pattern every time, I think you could argue that there would be limited value in trying to consider, using my analogy, different causes of death separately.
On the comments I made about S&P’s short-run trend from equilibrium plus shocks approach I suppose the only reason for not supplementing it is that any long-run predictions would be so speculative and in all likelihood wrong that they would have great risk of confusing rather than informing. Another thing that would change my mind is if it were shown that short-run default risk was highly correlated and therefore itself a predictor of medium to long-run default risk. Given the stability of the investment grade ratings and comparative volatility in the non-investment grade ratings this seems to be true for high ratings but perhaps not true for lower ratings. Again the question of can we do better is not clear but I think if we cannot then there might be a case for being more honest about the long-term reliability of the ratings.
This is only day four of my work experience and just like with my Justin Yifu Lin ‘Comparative Advantage Following’ essay I concede the absurdity of offering my musings but currently it is not obvious to me why I am wrong so I would greatly appreciate any comments that you have. My emotions over the four days have varied immensely. Initially I felt S&P’s approach was completely wrong but after seeing it in action and being immensely impressed with the breadth and depth of the analyst’s knowledge I felt like it was completely right. Having just written the essay though I have, at least for now, settled on the feeling that although S&P’s current approach to rating sovereigns is broadly correct it might be valuably supplemented by both econometric analysis and attempts at predicting the long-run shocks. I would add as my final comment that it is always worth thinking what would the reaction be if something went horribly wrong? How would the press and public react if it was discovered that (quite reasonably I might add) ratings were done in this way. I think there might be a backlash where even though rating agencies can’t be expected to predict all defaults they will nonetheless be blamed for not being able to do so.