How to read a country report without being misled by it

Any company assessing Ukraine will encounter half a dozen published assessments, and they will not agree. One index shows steady improvement, another shows stagnation, a rating agency is negative while an investment bank is constructive.

None of them is lying. They are measuring different things, and knowing which is which converts a confusing set of documents into a usable one.

The World Bank Doing Business methodology

The most cited and the most misread. It measured the regulatory burden on a hypothetical standardised firm — a domestic limited company of a specific size, in the largest city, in a specific sector — using a defined set of procedures.

It measured what the law says, not what happens. A country could improve its score substantially by passing legislation that was then unevenly applied, and Ukraine's climb in these rankings during the 2010s partly reflected exactly that.

The indicator is therefore genuinely informative about the formal regulatory framework and close to uninformative about the experience of actually operating. Both facts are worth holding at once.

Competitiveness indices

These blend hard data with an executive opinion survey. The survey component means they capture sentiment, which moves with the news cycle and with recent experience rather than with structural change.

That makes them useful as a mood indicator and unreliable as a measure of anything that changed slowly.

Sovereign credit ratings

These assess one thing precisely: the probability that the government will service its foreign-currency debt. They are excellent at that and they say little about whether a factory investment will succeed.

A country can be a poor sovereign credit and a reasonable place to manufacture. Confusing the two is one of the most common analytical errors made about emerging markets.

Corruption perception measures

Perception, as the name says. They aggregate expert and business surveys, which means they lag actual change in both directions — improvements take years to register, and a single prominent scandal can move the score more than a structural reform.

Investment bank research

Usually the most current and the most specific, and written for portfolio investors rather than operators. The macro analysis is generally sound; the implications drawn from it are about bond and equity positions, not about whether to open a plant.

What to do with all of it

Read them for the underlying data rather than the ranking. The useful content of a Doing Business report is the number of days to register property, not the position in the table.

Then weight each source by what it actually observes. For a manufacturing decision, the labour cost data, the electricity tariff, the logistics performance index and the customs clearance times matter enormously and the headline competitiveness ranking matters almost not at all.

And then, unavoidably, talk to companies already operating there. Every published index is a summary of the past; the people running a business in the market this quarter know things that no methodology captures, and an afternoon of those conversations is worth more than the entire stack of reports.

That is not an argument against the reports. It is an argument for using them as the frame rather than the conclusion.

Related in this archive

• Growing the market rather than dividing it: when competitors are worth more than they cost
• Industrial parks: the incentive regime, and whether it works
• The western ports and why they matter more than the map suggests
• Ukraine Annual Review 2016: the year the new economy took shape

None of these reports is wrong; they simply answer different questions, and the reader has usually asked a third one. I have never used any of them alone to make an investment decision. The method that works is this: read the report, then telephone three people working in the same sector — where the two differ, the telephone is right.