What is was supposed to be HS?

An internationally standardized system of names and numbers for classifying traded products.

Wow. Sounds like a very good idea. All these people that are trying to make some sense of the overall image of trade will be super happy. But the sad truth is that this system was designed by Chuck Norris and it will make you cry like an onion. I have been actually holding on for a long time all these complains and I don’t like that I have to complain all time about systems  that are sometimes described as as perfect as possible. A lot of people are defending these systems as being very well designed and definitely much better than having nothing or whatever even worst system they had before.  But I am not like this. I deeply despise the lack of logic, the shortness of view when thinking about possible usage of a system and the complacency within a system.  Lack of logic tends to perpetuate errors and to create crippled monsters. The shortness of view on the possibilities of a system drastically limit the system; the

To all this I just want to add that sometimes the bureaucy tends to create unusable system just because it can, people have inertia in changing systems which allows them to keep a position and control. This is the most difficult fight.

Lets start!

Differences between different countries
This is supposed to be an internationally standardized system, which in my non native English it means that it should be standard like in the word standard, like in ISO, like in international standard. I understand that you have local conditions and that the americans are not yet accustomed with the kilogram, and the french are putting the comma as a thousand separators. But how different can you be and still be in a standard? The standard itself should give either a direct translation either a indirect function to transform/compare A to B, when A has been defined in one country and B in other country of the same HS standard.  But there is no such thing.

FACTS:

• HS codes have 1 to 10 digits.
• Codes are synchronized between participating country up to 4 digits only.
• Some countries have 8 and some 10 digits, and EU sometimes is using 8 and sometimes 10 depending on the website (example: European Comission uses 10 digits. )
• 206 countries – means around 150 different systems – EU has a standard for 27 and probably some small countries are using subsets from others
• There is no function to ensure full synchronicity between the codes.
• This restricts usage only for big categories and the VERY big picture information

EXAMPLE

These are tilapia related product codes. Tilapia is a very common fish on the european tables, even thought sometimes probably you are paying for something else.

Norway SSB has 6 codes, mostly quite new introduced in January 2012:

• 03027100 – Fresh or chilled tilapia “Oreochromis spp.”
• 03032300 - Frozen tilapia “Oreochromis spp.”
• 03043100 - Fresh or chilled fillets of tilapia “Oreochromis s…
• 03046100 - Frozen fillets of tilapia “Oreochromis spp.”
• 03054400 - Smoked tilapia “Oreochromis spp.”..
• 03056400 - Tilapia “Oreochromis spp.”, catfish “Pangasius spp…

Mixing concepts

Business is complicated and each business has its own things that only people from that industry know. When creating a system that wants to harmonize smaller systems you have to decide what factors to consider and how deep in details you want to get into. You also want to choose a standard set of criteria to do your classification. Let us look at the following two examples from EU, although I repeat that the same problems appear everywhere.

• 03024098 - FRESH OR CHILLED HERRINGS ‘CLUPEA HARENGUS, CLUPEA PALLASII’, FROM 16 JUNE TO 31 DECEMBER
• 03027000 - FRESH OR CHILLED FISH LIVERS AND ROES
• 03021120 - FRESH OR CHILLED TROUT OF THE SPECIES “ONCORHYNCHUS MYKISS”, WITH HEADS ON AND GILLS ON, GUTTED, WEIGHING > 1,2 KG EACH, OR WITH HEADS OFF, GILLED AND GUTTED, WEIGHING > 1 KG EACH
• 03032900 -FROZEN SALMONIDAE (EXCL. PACIFIC SALMON, ATLANTIC SALMON, DANUBE SALMON AND TROUT)

What I see is a MIX OF CRITERIA. Fresh/frozen seem to be the main criteria, and it certainly seems much more important to know this than knowing what the specie the product is. The specie sometimes is omitted entirely, sometimes is very precise at subspecie level, sometimes we talk about group of species more or less precisely defined. Sometimes we care a lot about how the fish is prepared, like in the example with “heads off, gilled and gutted” and sometimes we couldn’t care less. Sometimes other specific business attributes appear, like specific weight, but not in a consistent manner. Only a handful of products are precisely defined. Sometimes arbitrarily concepts appear, like “FROM 16 JUNE TO 31 DECEMBER”, which you don’t know if it applies to the trade data itself or to the definition, for which you need really deep understanding of the industry. This kind of inconsistency is very difficult to address even when you are trying to normalize the system.

Defining through negation
This one is my favorite. I’ve already have written about it here. Defining through negation should be punishable by prison. All people reading this cannot be defined as “not monkeys”. In the programming world you can define by negation when the opposing category is exactly the opposite of what you are defining. But when creating categories or defining finite elements of one group you cannot choose to arbitrarily define using negation.

EXAMPLE

• 03022990 - FRESH OR CHILLED FLAT FISH “PLEURONECTIDAE, BOTHIDAE, CYNOGLOSSIDAE, SOLEIDAE, SCOPHTHALMIDAE AND CATHARIDAE” (EXCL. LESSER OR GREENLAND HALIBUT, ATLANTIC HALIBUT, PACIFIC HALIBUT, PLAICE, SOLE AND MEGRIM)

Try to picture this definition using a Venn diagram (bubbles). When you can be precise, be precise and for the rest use the parent categories which should act as containers for everything that is not defined.

About a quarter of EUROSTAT definitition have negations inside.

Overlapping categories

• 03037560 - Frozen porbeagle shark (Lamna nasus)
• 03042965 – Frozen fillets of porbeagle shark (Lamna nasus)

These are the codes for the frozen porbeagle related products. None of this is a parent to another although one is detailed and one is general. One is a general ambiguous category that can be used as a containter for anything frozen. At first glance you will think that the fillets are included in generic category, but this is not the case. In this case the first product probably is whole fish and you’ll have to do this kind of assumptions.

Dude, where is my logic?
The biggest problem that I see is the design. They started the Harmonized System design by splitting the stuff in by several criterias.

• 0301 - fish, live
• 0302 - fish, fresh or chilled (no fillets or other meat)
• 0303  - fish, frozen (no fish fillets or other fish meat)
• 0304  - fish fillets & other fish meat, fresh, chill or frozen
• 0305  - fish, dried, salted etc, smoked etc, ed fish meal
• 0306 - crustaceans, live, fresh etc, and cooked etc.
• 0307  - molluscs & aquatic invertebrates nesoi, live etc

First criteria is the specie, then storage type as second criteria, and a third as type of product. The problem is that they were thinking in terms of fresh/frozen not in terms of storage. They were thinking on fish/crustaceeans instead of specie and of fillets/mean instead of type of product. This is a big error, and this happens when people without mathematical concepts are doing mathematical stuff. Actually the problem happens at all levels. In order for me to get all seafood related products I have to search chapter 3 and 1603, 1604, 1605. And there at the bigger level you see another organization:

• 01-05 - Animal & Animal Products
• 06-15 - Vegetable Products
• 16-24 - Foodstuffs
• 25-27 - Mineral Products
• 28-38 - Chemicals & Allied Industries
• 39-40 - Plastics / Rubbers
• 41-43 - Raw Hides, Skins, Leather, & Furs
• 44-49 - Wood & Wood Products
• 50-63 - Textiles
• 64-67 - Footwear / Headgear
• 68-71 - Stone / Glass
• 72-83 - Metals
• 84-85 - Machinery / Electrical
• 86-89 - Transportation
• 90-97  - Miscellaneous
• 98-99 - Service

Wait. These seem very clear and distinct categories. Yes… they are more or less distinct, but they have no logic. What is the criteria I ask? Is is the raw material that the product is created, like for the raw products? Is it the type of use that will have, like in the foodstuff or the transportation? In building a list you need a criteria that will help you organize the stuff. In this list there is no criteria. Just a bunch of unrelated, no concept container for products.

Classical categorization comes to us first from Plato, who introduces the approach of grouping objects based on their similar properties. Miscategorization can be a logical fallacy in which diverse and dissimilar objects, concepts, entities, etc. are grouped together based upon illogical common denominators, or common denominators that virtually any concept, object or entity have in common.

Read more on the Source, this is why is there: Wikipedia.

Purpose (or the lack of)
According to wikipedia the system was suppose to be used as a basis to:

• Customs tariffs
• Collection of international trade statistics
• Rules of origin
• Collection of internal taxes
• Trade negotiations (e.g., the World Trade Organization schedules of tariff concessions)
• Transport tariffs and statistics
• Monitoring of controlled goods (e.g., wastes, narcotics, chemical weapons, ozone layer depleting substances, endangered species)
• Areas of Customs controls and procedures, including risk assessment, information technology and compliance.

Basically the system was build by minds that did not knew all this. As it is now I see no way in which this system can be used for all these legitimate purposes. The implementing countries are actually using this system for taxation purposes. I don’t know how well this works, but this is actually the only practical use that it has – to substitute custom tables for collecting taxes. And I am not very sure is very good at that either, as I can find a lot of loopholes  and such ambiguous descriptions allow probably a lot of fiscal evasion. Yeah, companies do that usually in case you have not noticed.  This is a semi-parasitic use, with the excuse of not having to support two systems, the customs are using the HS and sometimes they push modifications based on their problems into the HS. Most internal state or private associations, choose not to use the system as it lacks detail and they develop their own standards and taxonomies, most of the time with no connection with HS.

As for I was concern the system by itself has no real statistical use. The only use can be on localized and very global issues like how much frozen fish a country exports. Long term analysis is not possible and mixing information from two different system or doing comparative analysis – which is a standard – is not possible either. Statisticians are loosing more time extracting and massaging the data than using it. There are no tools to interconnect data from different sources (until us of course).

Not relevant to business
I’ve heard it already from clients. The trade statistics are good for researchers and PhD students that do overview analysis with an academic end. For business they are RUBBISH.  First of all because they are late. Most of them come with 45 days or more delay. Why? Because the systems are created in the nineties at tops. They have no concept of live data or partial data. Let me give you some good news. We can build today systems with sub-second delay. And I mean global systems, like Nasdaq or other types of trading, GPS data, maritime data and so on. Volume is not an issue anymore. But freshness is!

The second problem is the detail. Businesses don’t want an average price for fillets, the loins are triple in price compared with other things that are put in the same fillets category.I am curious how useful is the system to the states and the people who created it. So far I have not met a single person who uses trade data in business successfully.

The third is problem is complexity. Who has time to monitor all these complexities, all these changes, all these different systems and formats? Nobody in the business can put so much time/money/effort into it. A manager will have no time for such a thing, a secretary will not understand anything, and the budget does not allow extra people only to understand data and extract meaning from it.

I see this as a biggest fail of the Harmonized System.

Ambiguity all over

• 03027000 - FRESH OR CHILLED FISH LIVERS AND ROES

All fish does have liver and roes, what specie is this? In other places liver and roes are treated separately and as well in the market. You dont mix and sell them together. They have different prices and so on. Caviar – sturgeon roes- have definitely another price than cod liver. And there are a lot of examples like this.  What is the meaning of having this category? Who can analyze what is inside. There is data for it, so this is no obscure code that nobody will use: 32315 entries since 1988 for a total of 210574 MT (metric tons), that is huge for roes.  That is the weight of 28 Eiffel towers in fish eggs and liver. How can I know how much liver and how much row are inside 210 thousands tons?

Lack of documentation and technical issues
The HS just is. You are not allowed to question or  to judge it. It is above you and everybody else. If you don’t like it too bad, there is nothing else.

They are not selling books of publishing a documentation because by doing that you will understand how bad and illogical the system is. For a normal person is IMPOSSIBLE to take a look at the data. Usually data comes in raw files, with separate entries for country taxonomies, product taxonomies. The data is hidden in some super websites that probably costs a lot of money to the taxpayers. If you want to see it you have to do it using custom made tools that only allow you specific formats and specific angles to the data. Each country has its own way of hiding data.  I want to congratulate personally the business sales manager who will open a 200 MB raw file of data for one month of EU trade and that will successfully  JOIN the needed taxonomies. Probably the guy should get the “Business/programmer of the year” award.

In a world that speaks a lot of ”OPEN DATA’, there are a lot of problems in finding the data, information about the data. Easy access systems or open API’s dont exist. What does exist: 7zip archives, excel and a lot of custom build solutions.

Most stupid example
These are a couple of many that make you say: WTF?

• 16041994 - HAKE “MERLUCCIUS SPP., UROPHYCIS SPP.”, PREPARED OR PRESERVED, WHOLE OR IN PIECES (EXCL. FINELY MINCED AND FILLETS, RAW, MERELY COATED WITH BATTER OR BREADCRUMBS, WHETHER OR NOT PRE-FRIED IN OIL,FROZEN)
• 03056990  - FISH, SALTED OR IN BRINE, BUT NEITHER DRIED NOR SMOKED (EXCL. HERRINGS, COD, ANCHOVIES, FISH OF THE SPECIES BOREOGADUS SAIDA, LESSER OR GREENLAND HALIBUT, PACIFIC HALIBUT, ATLANTIC HALIBUT, PACIFIC SALMON, ATLANTIC SALMON, DANUBE SALMON AND FILLETS IN GENERAL

The people
I decided not to put any names or correspondence that I had with the people from these institutions. Is not their fault, they are part of THE SYSTEM, a system that has a huge INERTIA and no purpose. People inside a system don’t question the system and certainly don’t want change of a system which guarantees and maintains their place. So why should they bother? I understand them.

Is everything bad?
Well, there are good intentions like this one

• 03032120  - FROZEN TROUT OF THE SPECIES “ONCORHYNCHUS MYKISS”, WITH HEADS ON AND GILLS ON, GUTTED, WEIGHING > 1,2 KG EACH, OR WITH HEADS OFF, GILLED AND GUTTED, WEIGHING > 1 KG EACH

which is closer to a normalized definition, by being specific, by defining only one specie and a preparation type and a weight, so it looks more complete than others.  But all good intention are still only marking the path to hell and they blew it by doing a logical OR. They are trying to solve the round weight equivalent problem but they never heard about it so they don’t know. The round weight equivalent is a problem in the fishing industry as the fish can be cut and prepared in a variety of ways, loosing more or less mass from the raw fish. This problem has been solved a long ago by creating some tables that specify conversion rates for different species and types of cuts. This thing solved people just send the data transformed into a round weight equivalent. And that’s that. Is so simple, but you have to think in abstract not just copy whatever the industry is sending at you.
My criteria for a future RHS – THE REAL HARMONIZED SYSTEM
The definitions should be:

• precise
• complete
• unique
• harmonized at a low level between countries
• useful for everybody
• accessible and very low tech

I will conclude by saying that the Harmonized System harmonizes the Trade in exactly the same measure that American Football rules are harmonized to European Football rules.

PS: the discussion about the data itself and its insights  deserves another article… soon!

Natural selection is a key element to evolution, being a nonrandom process by which biologic traits become either more or less common in a population as a function of differential reproduction of their bearers. A rabbit that runs faster than others may be more likely to escape from predators, and algae that are more efficient at extracting energy from sunlight will grow faster. But these days, at least for humans the things are not like this anymore.  Advances in medicine and more health care and in social behavior result in longer lives and very low infant mortality rates, a very known fact.  Thought the current gene pool seems healthy, by allowing all (or most) mutations to survive some really different and maybe unhealthy branches of humans will develop. XMEN? I see them coming! So far the biggest group that benefited the tranquility are the mediocre. The hunter-gatherer had to work hard in order to make a catch or a harvest. Things are getting softer each day.  Is mediocrity going to asphyxiate the creativity? A big mediocre society does implement mechanisms that empower its own representatives. The politics today are run by mediocre intelligence, people who are really representing the damaged and brain dead middle class.

My questions are:

1. how do we continue to improve our specie? should we intervene?
2. we need all that diversity? we need subspecies for Homo Sapiens Sapiens? is the cast system pointing towards this?


03056200 - COD 'GADUS MORHUA, GADUS OGAC, GADUS MACROCEPHALUS', SALTED OR IN BRINE ONLY (EXCL. FILLETS)


How is it possible to define something by negation? This is against basic logic. Not fillets can mean thousands of stuff.

If I would say:

I am not dog.

Can you tell if I am a insect or a mammal? And mammals actually do include dogs.

EUROSTAT taxonomy is a joke of and useful only to taxation purposes, thought I will look very carefully before sending my goods under a specific code, as I may found at least 10 more equally unclear that may be taxed lower.

And SSB, Hagstova and Hagstofa are copying this crazy way of doing definitions. John Venn would crucify them all.


htmlspecialchars() expects parameter 1 to be string, object given in /var/www/myproject/library/Zend/View/Abstract.php on line 897



// species is a Zend_Db_Table_Rowset
// this will warn
$form = new My_Form(array('species' => $species));
// this will go ok
$form = new My_Form(array('species' => $species->toArray()));
// if this is generating further rendering problems you'll have to create a simple array with only key value pairs. :(


• autocleans your code
• autoprotects your code

Just apply gently.

:)

2. Update R - I updated R from inside with update.packages() (! or similar)

3. RApache install

• we need the prefork only for apreq2 libraries,  so even if you have Apache working do this.


apt-get install r-base-dev apache2-mpm-prefork apache2-prefork-dev
wget http://biostat.mc.vanderbilt.edu/rapache/files/rapache-latest.tar.gz
rapachedir=`tar tzf rapache-latest.tar.gz | head -1`
tar xzvf rapache-latest.tar.gz
cd $rapachedir
./configure
make
make install

4. Create a file for R config

nano /etc/apache2/mods-available/r.conf

ROutputErrors
SetHandler r-info
# hangs if the modules are not available
#REvalOnStartup "library(DBI); library(RMySQL)"
#RSourceOnStartup "/var/www/r/startup.R"
SetHandler r-script RHandler sys.source


5. Create a module for apache

nano r.load

with the following content

LoadModule R_module /usr/lib/apache2/modules/mod_R.so

6. Add module to Apache

a2nmod r

7. Reload Apache

/etc/init.d/apache2 reload

8. Check your installation at

http://localhost/RApacheInfo

9. Test it

http://localhost/r/startup.R?a=1&b=2

startup.R file content


get = str(GET)
d = LETTERS[1:4]
print(get)
print (d)


Should print

List of 2
$ a: chr "1"
$ b: chr "2"
NULL
[1] "A" "B" "C" "D"

INTERNATIONALIZATION (or the lack of)

First problem that I’ve came in contact with is the lack of internationalization support. This goes for web, customer relation, email communications and personal in the bank. While most Norwegians speak English, the opposite is not true. Most of SpareBank international clients are not speaking Norwegian. A lot of people come to Norway to work, and learning Norwegian should not be a requisite for opening an account. All the website is in norwegian, all the forms and manuals are the same. Since September some parts of the website are in English, but very little and very bad. Is quite strange and I could say grossly impolite to respond in Norwegian if I am writing emails to you in English.  But it happens a lot. And the response I get:

If you use Google translater you can copy paste the solution
ant tanslate from norwegian to english.

Per Kr. Eilertsen
Seniorrådgiver

17.03.3011

No edits were made. Are you serious? This is your corporate answer? Millions of NOK go into PR campaigns for this?

There is a list of item that I bought in my internet banking. 80% of them are named, and I have English as the selected option:

Varekjøp

All other items that I paid using my card all the things are in Norwegian.

ACCESSIBILITY (or the lack of)
There are people of all types that are using your services and the internet. Myself I am using Linux as my operating system. Linux is free and it has growing number of users. Linux/Mozilla is flawless compared with Windows in terms of security and in terms of displaying webpages.

When you are looking in your google Analytics you will tell me that not a lot of people is using it. Of course not if you are not offering a valid web application.

I have no problem accessing any other internet banking account from my linux machine. On SNN  though this is almost impossible. The messages are cryptic. The Java plugin that you are using is behaving each time different in terms of responses, or errors. Sometimes it hangs indefinately, sometimes it works once and never again.

And is your applet and not the plugin which works in other configurations.

JAVA was a choice for your plugin beacause its CROSS-platform capabilities. But I am sure you already knew that.

After a lot of mailing back (in Norwegian) and forth I get recommended to change the JAVA plugin with one which in your support is the “correct version”. I just want to say that the “correct version” is not the correct version. The procedure to install it is not either straightforward, either explained anyware. This is the link that your customer support is sending me to:

http://origowiki.sparebank1.no/origowiki/index.php/BankID_og_Linux

It does not work for me. Maybe is a link on your intranet. But I am sure you already knew that.

USABILITY

When working with internet banking you want to know stuff live. This is the advantage of internet in the first place.
I made a few international transfers. After a dificult procedure, which asks me to safe a sender (why?) I make the payment and everything dissapears. I dont get a confirmation that something happen, I dont see money reserver in my account, I dont have a page showing me the latest transactions… NOTHING. I have to wait for a day or two until the transaction will enter the main history line to realize what really happen. Or I can try again thus duplicating the transaction.

The transaction signing. I understand that is a good security procedure to sign transactions, but you are abusing it, in terms of screens. Put all the codes in one screen. What is the point on having the national number first and the rest in a second step procedure?

The messages are short and unclear.

SECURITY
Contrary to the public opinion by using a JAVA applet you are not making the application more secure. You probably knew this as well.

SUPPORT

And all this is handled by a customer service who:

• writes bank in Norwegian when you are addressing in English
• sends links that don’t work
• responds from emails that are bounce (cti_bags_test@snn.no)
• that are so polite that send you use google translate as a service
• that answer with 24 hours+ delay for each email
• that ask you for your date of birth instead of lsb_release() – it happend on the SNN twitter.
• that don’t respond anymore if they don’t like the problem
• that don’t call you back (Espen Kjølberg) even if the general manager (Olav Eriksen) tells you he will call back (I got a call after publishing this post, more than 2 weeks later)
• that tell you in the bank that you have to wait because everybody is at lunch ( at 12.pm in my lunch time)
• that have different approaches depending on the person (procedures are flexible and adaptable)

The best feature of your internet banking for is the fact that on the login page your people have put a list of errors that are most common. This means that is mostly expected that this errors will happen. Look at BOFA or Santander  or any other big banks and tell me how many errors do they list on the login page?

And above all I get emails from you inviting me to seminars where you will teach me the benefits of internet banking? and how well you do it? This becomes paradoxical…

All this small things cost money in the end, and affect the quality and the price of your services.

Just cut the crap and assume responsibility!

Update from the dialogs (monologs – me talking about how nice is the internet, and the big corporation mimicking a face saving) that followed. Everything from this post was resumed to the Linux issue and solved like this :

The BankID and Linux issues that you are experiencing are most likely due to the fact that you are using openjdk. BankID and the BankID applets which are used by all Norwegian banks require SUNs JRE.

My last email to Mona Strøm Arnøy. and Espen Kjolberg is still without a reply:

I still have a couple of questions for your leaders:

• what are other banks in the world are doing? I mean the banks who are not implementing BankId and still have strong authentication. As I was saying java != strong_auth. Just strong problems.
• so 2.5 million users (forcibly) have enrolled to BankID. What other options your bank provides me as a customer?

10% of the readers of this blog don’t have a Java solution (google analytics). Those users are not able to use internet banking solutions in Norway?

Stupid systems and people without vertebrae everywhere… what can you expect?

UPDATE

Here is the proof that I am not crazy, but just an informed customer, and all Spare Bank users are indeed in danger (written by  Kristian Gjøstee in 2008!!!) . And another interesting paper from 2008! And another one from 2009!