No harmony in the Harmonized System (HS)
I’ve been working with the Harmonized System (HS) now for one year and a half, as part of my work at Capia, as part of a project that aims to understand the intricacies of the fishing business. During this time I’ve been in touch with HS implementations of USA, Norway, Iceland, Faroe Islands, Russia and EU (NOAA, SSB, Hagstova, Hagstofa, Gosstat, Eurostat).
What is was supposed to be HS?
An internationally standardized system of names and numbers for classifying traded products.
Wow. Sounds like a very good idea. All these people that are trying to make some sense of the overall image of trade will be super happy. But the sad truth is that this system was designed by Chuck Norris and it will make you cry like an onion. I have been actually holding on for a long time all these complains and I don’t like that I have to complain all time about systems that are sometimes described as as perfect as possible. A lot of people are defending these systems as being very well designed and definitely much better than having nothing or whatever even worst system they had before. But I am not like this. I deeply despise the lack of logic, the shortness of view when thinking about possible usage of a system and the complacency within a system. Lack of logic tends to perpetuate errors and to create crippled monsters.
To all this I just want to add that sometimes the bureaucy tends to create unusable system just because it can, people have inertia in changing systems which allows them to keep a position and control. This is the most difficult fight.
Differences between different countries
This is supposed to be an internationally standardized system, which in my non native English it means that it should be standard like in the word standard, like in ISO, like in international standard. I understand that you have local conditions and that the americans are not yet accustomed with the kilogram, and the french are putting the comma as a thousand separators. But how different can you be and still be in a standard? The standard itself should give either a direct translation either a indirect function to transform/compare A to B, when A has been defined in one country and B in other country of the same HS standard. But there is no such thing.
- HS codes have 1 to 10 digits.
- Codes are synchronized between participating country up to 4 digits only.
- Some countries have 8 and some 10 digits, and EU sometimes is using 8 and sometimes 10 depending on the website (example: European Comission uses 10 digits. )
- 206 countries – means around 150 different systems – EU has a standard for 27 and probably some small countries are using subsets from others
- There is no function to ensure full synchronicity between the codes.
- This restricts usage only for big categories and the VERY big picture information
These are tilapia related product codes. Tilapia is a very common fish on the european tables, even thought sometimes probably you are paying for something else.
Norway SSB has 6 codes, mostly quite new introduced in January 2012:
- 03027100 – Fresh or chilled tilapia “Oreochromis spp.”
- 03032300 – Frozen tilapia “Oreochromis spp.”
- 03043100 – Fresh or chilled fillets of tilapia “Oreochromis s…
- 03046100 – Frozen fillets of tilapia “Oreochromis spp.”
- 03054400 – Smoked tilapia “Oreochromis spp.”..
- 03056400 – Tilapia “Oreochromis spp.”, catfish “Pangasius spp…
- 0303790040 – Tilapia frozen
- 0303792094 – Tilapia frozen
- 0304104061 – Tilapia fresh
- 0304190043 – Tilapia fresh
- 0304206042 – Tilapia frozen fillets
- 0304296042 -Tilapia frozen fillets
- 03026915 – Fresh or chilled tilapia (Oreochromis spp.)
- 03042905 – Frozen fillets of tilapia (Oreochromis spp.)
Inside the same system the codes for the same product differ from year to year.
- 0303790040 – Tilapia frozen from: 2007-01-01 to 3999-12-31
- 0303792094 – Tilapia frozen from: 1992-01-01 to 2007-12-31
As you saw tilapia, is only covered by two codes directly (and 2 more where tilapia is negated) in the EU HS.
- 03042905 – Frozen fillets of tilapia (Oreochromis spp.)
- 03026915 – Fresh or chilled tilapia (Oreochromis spp.)
Let me tell you what will happen. It will be declared as saltwater fish and disappear from tilapia related statistics and data. It will vanish. Until somebody from EUROSTAT, by manually looking at import/export papers, will decide that frozen whole tilapia is an important enough category. And this can take 1 year, 5 years or an infinity because it depends on outside factors trying to modify a system which thinks is complete.
Lack of detail concept, no precision
- 03026981 – FRESH OR CHILLED MONKFISH “LOPHIUS SPP.”
This is a monkfish image from fishbase.org
Why is this example relevant? Because the monkfish tail is the only thing that you can eat. But the head is two thirds of the body and probably 70% of the weight. The price of whole fish is very different from the headed. Fresh, frozen and fillets is not acceptable as level of detail. How do you expect for people to use your data with this amount of precision?
Business is complicated and each business has its own things that only people from that industry know. When creating a system that wants to harmonize smaller systems you have to decide what factors to consider and how deep in details you want to get into. You also want to choose a standard set of criteria to do your classification. Let us look at the following two examples from EU, although I repeat that the same problems appear everywhere.
- 03024098 – FRESH OR CHILLED HERRINGS ‘CLUPEA HARENGUS, CLUPEA PALLASII’, FROM 16 JUNE TO 31 DECEMBER
- 03027000 – FRESH OR CHILLED FISH LIVERS AND ROES
- 03021120 – FRESH OR CHILLED TROUT OF THE SPECIES “ONCORHYNCHUS MYKISS”, WITH HEADS ON AND GILLS ON, GUTTED, WEIGHING > 1,2 KG EACH, OR WITH HEADS OFF, GILLED AND GUTTED, WEIGHING > 1 KG EACH
- 03032900 -FROZEN SALMONIDAE (EXCL. PACIFIC SALMON, ATLANTIC SALMON, DANUBE SALMON AND TROUT)
What I see is a MIX OF CRITERIA. Fresh/frozen seem to be the main criteria, and it certainly seems much more important to know this than knowing what the specie the product is. The specie sometimes is omitted entirely, sometimes is very precise at subspecie level, sometimes we talk about group of species more or less precisely defined. Sometimes we care a lot about how the fish is prepared, like in the example with “heads off, gilled and gutted” and sometimes we couldn’t care less. Sometimes other specific business attributes appear, like specific weight, but not in a consistent manner. Only a handful of products are precisely defined. Sometimes arbitrarily concepts appear, like “FROM 16 JUNE TO 31 DECEMBER”, which you don’t know if it applies to the trade data itself or to the definition, for which you need really deep understanding of the industry. This kind of inconsistency is very difficult to address even when you are trying to normalize the system.
Defining through negation
This one is my favorite. I’ve already have written about it here. Defining through negation should be punishable by prison. All people reading this cannot be defined as “not monkeys”. In the programming world you can define by negation when the opposing category is exactly the opposite of what you are defining. But when creating categories or defining finite elements of one group you cannot choose to arbitrarily define using negation.
- 03022990 – FRESH OR CHILLED FLAT FISH “PLEURONECTIDAE, BOTHIDAE, CYNOGLOSSIDAE, SOLEIDAE, SCOPHTHALMIDAE AND CATHARIDAE” (EXCL. LESSER OR GREENLAND HALIBUT, ATLANTIC HALIBUT, PACIFIC HALIBUT, PLAICE, SOLE AND MEGRIM)
Try to picture this definition using a Venn diagram (bubbles). When you can be precise, be precise and for the rest use the parent categories which should act as containers for everything that is not defined.
About a quarter of EUROSTAT definitition have negations inside.
- 03037560 – Frozen porbeagle shark (Lamna nasus)
- 03042965 – Frozen fillets of porbeagle shark (Lamna nasus)
These are the codes for the frozen porbeagle related products. None of this is a parent to another although one is detailed and one is general. One is a general ambiguous category that can be used as a containter for anything frozen. At first glance you will think that the fillets are included in generic category, but this is not the case. In this case the first product probably is whole fish and you’ll have to do this kind of assumptions.
Dude, where is my logic?
The biggest problem that I see is the design. They started the Harmonized System design by splitting the stuff in by several criterias.
- 0301 – fish, live
- 0302 – fish, fresh or chilled (no fillets or other meat)
- 0303 – fish, frozen (no fish fillets or other fish meat)
- 0304 – fish fillets & other fish meat, fresh, chill or frozen
- 0305 – fish, dried, salted etc, smoked etc, ed fish meal
- 0306 – crustaceans, live, fresh etc, and cooked etc.
- 0307 – molluscs & aquatic invertebrates nesoi, live etc
First criteria is the specie, then storage type as second criteria, and a third as type of product. The problem is that they were thinking in terms of fresh/frozen not in terms of storage. They were thinking on fish/crustaceeans instead of specie and of fillets/mean instead of type of product. This is a big error, and this happens when people without mathematical concepts are doing mathematical stuff. Actually the problem happens at all levels. In order for me to get all seafood related products I have to search chapter 3 and 1603, 1604, 1605. And there at the bigger level you see another organization:
- 01-05 – Animal & Animal Products
- 06-15 – Vegetable Products
- 16-24 – Foodstuffs
- 25-27 – Mineral Products
- 28-38 – Chemicals & Allied Industries
- 39-40 – Plastics / Rubbers
- 41-43 – Raw Hides, Skins, Leather, & Furs
- 44-49 – Wood & Wood Products
- 50-63 – Textiles
- 64-67 – Footwear / Headgear
- 68-71 – Stone / Glass
- 72-83 – Metals
- 84-85 – Machinery / Electrical
- 86-89 – Transportation
- 90-97 – Miscellaneous
- 98-99 – Service
Wait. These seem very clear and distinct categories. Yes… they are more or less distinct, but they have no logic. What is the criteria I ask? Is is the raw material that the product is created, like for the raw products? Is it the type of use that will have, like in the foodstuff or the transportation? In building a list you need a criteria that will help you organize the stuff. In this list there is no criteria. Just a bunch of unrelated, no concept container for products.
Classical categorization comes to us first from Plato, who introduces the approach of grouping objects based on their similar properties. Miscategorization can be a logical fallacy in which diverse and dissimilar objects, concepts, entities, etc. are grouped together based upon illogical common denominators, or common denominators that virtually any concept, object or entity have in common.
Read more on the Source, this is why is there: Wikipedia.
Purpose (or the lack of)
According to wikipedia the system was suppose to be used as a basis to:
- Customs tariffs
- Collection of international trade statistics
- Rules of origin
- Collection of internal taxes
- Trade negotiations (e.g., the World Trade Organization schedules of tariff concessions)
- Transport tariffs and statistics
- Monitoring of controlled goods (e.g., wastes, narcotics, chemical weapons, ozone layer depleting substances, endangered species)
- Areas of Customs controls and procedures, including risk assessment, information technology and compliance.
Basically the system was build by minds that did not knew all this. As it is now I see no way in which this system can be used for all these legitimate purposes. The implementing countries are actually using this system for taxation purposes. I don’t know how well this works, but this is actually the only practical use that it has – to substitute custom tables for collecting taxes. And I am not very sure is very good at that either, as I can find a lot of loopholes and such ambiguous descriptions allow probably a lot of fiscal evasion. Yeah, companies do that usually in case you have not noticed. This is a semi-parasitic use, with the excuse of not having to support two systems, the customs are using the HS and sometimes they push modifications based on their problems into the HS. Most internal state or private associations, choose not to use the system as it lacks detail and they develop their own standards and taxonomies, most of the time with no connection with HS.
As for I was concern the system by itself has no real statistical use. The only use can be on localized and very global issues like how much frozen fish a country exports. Long term analysis is not possible and mixing information from two different system or doing comparative analysis – which is a standard – is not possible either. Statisticians are loosing more time extracting and massaging the data than using it. There are no tools to interconnect data from different sources (until us of course).
Not relevant to business
I’ve heard it already from clients. The trade statistics are good for researchers and PhD students that do overview analysis with an academic end. For business they are RUBBISH. First of all because they are late. Most of them come with 45 days or more delay. Why? Because the systems are created in the nineties at tops. They have no concept of live data or partial data. Let me give you some good news. We can build today systems with sub-second delay. And I mean global systems, like Nasdaq or other types of trading, GPS data, maritime data and so on. Volume is not an issue anymore. But freshness is!
The second problem is the detail. Businesses don’t want an average price for fillets, the loins are triple in price compared with other things that are put in the same fillets category.I am curious how useful is the system to the states and the people who created it. So far I have not met a single person who uses trade data in business successfully.
The third is problem is complexity. Who has time to monitor all these complexities, all these changes, all these different systems and formats? Nobody in the business can put so much time/money/effort into it. A manager will have no time for such a thing, a secretary will not understand anything, and the budget does not allow extra people only to understand data and extract meaning from it.
I see this as a biggest fail of the Harmonized System.
Ambiguity all over
- 03027000 – FRESH OR CHILLED FISH LIVERS AND ROES
All fish does have liver and roes, what specie is this? In other places liver and roes are treated separately and as well in the market. You dont mix and sell them together. They have different prices and so on. Caviar – sturgeon roes- have definitely another price than cod liver. And there are a lot of examples like this. What is the meaning of having this category? Who can analyze what is inside. There is data for it, so this is no obscure code that nobody will use: 32315 entries since 1988 for a total of 210574 MT (metric tons), that is huge for roes. That is the weight of 28 Eiffel towers in fish eggs and liver. How can I know how much liver and how much row are inside 210 thousands tons?
Lack of documentation and technical issues
The HS just is. You are not allowed to question or to judge it. It is above you and everybody else. If you don’t like it too bad, there is nothing else.
They are not selling books of publishing a documentation because by doing that you will understand how bad and illogical the system is. For a normal person is IMPOSSIBLE to take a look at the data. Usually data comes in raw files, with separate entries for country taxonomies, product taxonomies. The data is hidden in some super websites that probably costs a lot of money to the taxpayers. If you want to see it you have to do it using custom made tools that only allow you specific formats and specific angles to the data. Each country has its own way of hiding data. I want to congratulate personally the business sales manager who will open a 200 MB raw file of data for one month of EU trade and that will successfully JOIN the needed taxonomies. Probably the guy should get the “Business/programmer of the year” award.
In a world that speaks a lot of “OPEN DATA’, there are a lot of problems in finding the data, information about the data. Easy access systems or open API’s dont exist. What does exist: 7zip archives, excel and a lot of custom build solutions.
Most stupid example
These are a couple of many that make you say: WTF?
- 16041994 – HAKE “MERLUCCIUS SPP., UROPHYCIS SPP.”, PREPARED OR PRESERVED, WHOLE OR IN PIECES (EXCL. FINELY MINCED AND FILLETS, RAW, MERELY COATED WITH BATTER OR BREADCRUMBS, WHETHER OR NOT PRE-FRIED IN OIL,FROZEN)
- 03056990 – FISH, SALTED OR IN BRINE, BUT NEITHER DRIED NOR SMOKED (EXCL. HERRINGS, COD, ANCHOVIES, FISH OF THE SPECIES BOREOGADUS SAIDA, LESSER OR GREENLAND HALIBUT, PACIFIC HALIBUT, ATLANTIC HALIBUT, PACIFIC SALMON, ATLANTIC SALMON, DANUBE SALMON AND FILLETS IN GENERAL
I decided not to put any names or correspondence that I had with the people from these institutions. Is not their fault, they are part of THE SYSTEM, a system that has a huge INERTIA and no purpose. People inside a system don’t question the system and certainly don’t want change of a system which guarantees and maintains their place. So why should they bother? I understand them.
Is everything bad?
Well, there are good intentions like this one
- 03032120 – FROZEN TROUT OF THE SPECIES “ONCORHYNCHUS MYKISS”, WITH HEADS ON AND GILLS ON, GUTTED, WEIGHING > 1,2 KG EACH, OR WITH HEADS OFF, GILLED AND GUTTED, WEIGHING > 1 KG EACH
which is closer to a normalized definition, by being specific, by defining only one specie and a preparation type and a weight, so it looks more complete than others. But all good intention are still only marking the path to hell and they blew it by doing a logical OR. They are trying to solve the round weight equivalent problem but they never heard about it so they don’t know. The round weight equivalent is a problem in the fishing industry as the fish can be cut and prepared in a variety of ways, loosing more or less mass from the raw fish. This problem has been solved a long ago by creating some tables that specify conversion rates for different species and types of cuts. This thing solved people just send the data transformed into a round weight equivalent. And that’s that. Is so simple, but you have to think in abstract not just copy whatever the industry is sending at you.
My criteria for a future RHS – THE REAL HARMONIZED SYSTEM
The definitions should be:
- harmonized at a low level between countries
- useful for everybody
- accessible and very low tech
I will conclude by saying that the Harmonized System harmonizes the Trade in exactly the same measure that American Football rules are harmonized to European Football rules.
PS: the discussion about the data itself and its insights deserves another article… soon!