In Defense of Details

People are generally overwhelmed by amounts of detail in programming and their intuition is to hide it. It is so ubiquitous that in the Abstraction_(disambiguation) there reads:

Unfortunately, this results in more things to learn and hide.

An example (and another) in Arduino

An example of this can be found in the Arduino, a very successful electronics prototyping kits used by hobbyists and professionals around the world.

Arduino circuit board contains digital pins that are helpfully labeled. This labeling is a good and a helpful way to quickly reference each pin.

In the arduino library there is a function to read the value coming into each digital pin. Here's an example of reading the pin 4 and storing it into a variable:

value = digitalRead(4);

This adds some portability and attempts to hide a detail. The Arduino is based on the AVR family of microcontrollers. The pin 4 on Arduino UNO is actually the pin 4 in the Port D of the Atmega328. Here is the corresponding AVR code for reading the pin 4:

value = (PIND >> 4) & 1;

The above code shifts the fifth bit first (>>4) and then erases the higher bit values (&1), then it stores (value=) the result into the variable.

The digitalRead functionality is meant to be helpful for a beginner and helpful for portability, but beyond the simplest read-a-switch -applications it will end up only being an extra detail you have to memorise, with no other use.

depending on what your application is doing the pins are not interchangeable in functionality. In that case, if you move the wire from pin 4 to pin 5 or 6, and change the program accordingly, it may break. Also, if the functionality is not on that pin on a different Arduino microcontroller card, the application will not be portable even if it would use this function. On top of that, the digitalRead is in order of 10 to 50 times slower than reading the value directly. Many microcontroller projects are time critical. If you run out of the time to do your thing, the digitalRead is the first function to vanish in the code.

The Atmel datasheet lists the following Alternate Functions for the PD4:

XCK (USART External Clock Input/Output)
T0 (Timer/Counter 0 External Counter Input)
PCINT20 (Pin Change Interrupt 20)

The XCK is unfamiliar to me because I've not used USART. I am sure it is an interesting feature but there is a lot to tell about so I skip it.

The T0 is a functionality that would allow you to set one of the timer/counter units on the microcontroller to listen at this pin, and increment whenever there is a transition from zero to one, or vice-versa, on this pin. I've seen this used once, and the code was really not portable from Arduino UNO to micro, for example. Yet the functionality was necessary.

The alternative functionality exists on the microcontroller because it is convenient. Due to the cost, you cannot provide every functionality on every pin. For this reason they have been spread across the pins with the hope that the user does not need the overlapping functionality in a single application.

The Pin Change Interrupt means that you can call a function when the pin changes. The PCINT -specific interrupts react all-at-once to one interrupt.

There is a PCINT function for every pin on an Arduino UNO (this is not true on the other Arduinos). Coincidentally the Arduino also has a tool to associate a function with a specific pin change. That also sees little to no use for a good reason.

Interrupts are small chunks of programs that run over other programs whenever some external event occurs. During the interrupt the application cannot proceed. Because of this the programmers attempt to keep the interrupts short as possible.

If you end up calling the attachInterrupt() on Arduino, you will introduce an interrupt that is long enough to really delay every possible action that is taking place.

You pay a terrible price, have to remember yet another detail, on top of those details that it does not hide, you gain nothing.

This is a funny and a frustrating theme across the Arduino. The intuition to use abstraction to hide is a plague among the Arduino community, resulting in simply bad code and stupid ideas.

On an Arduino micro, there is a builtin LED behind the pin 13 which is used in simplest applications to simplify a hello-world application, requiring no assembly. Also the Arduino's bootloader is flashing that led every time when you upload a program on the microcontroller.

The builtin LED shares a function "Input capture with Timer/Counter 3". The functionality is always active because it does not interfere with anything. If this pin is set as an input, it will immediately store a value from the timer whenever there is a 0->1 or 1->0 transition in the pin.

The input capture is extremely useful feature for reading various unusual signals. The builtin LED is partially redundant and makes all those input functions much less useful or even unusable.

It is natural

To be overwhelmed by all the details is natural. Get enough of those magic numbers on the Arduino and you're easily overwhelmed.

Becoming overwhelmed by details is entirely natural, yet in computing it results in exactly the opposite result of what you desire. On top of that the complexity you introduce is left for the future generations to "enjoy".

Perhaps this article was written because we received enough of this kind of "technical debt".

The case Arduino digitalRead fix

There's an abstraction to cope with the complexity. It also helps with porting the application. You do the following in the beginning of the program:

// The Pin 4 on Arduino UNO, we are using the T0
// functionality to read the frequency of the incoming
// signal.
#define FREQ_SOURCE_DDR  DDRD
#define FREQ_SOURCE_PORT PORTD
#define FREQ_SOURCE_PIN  PIND
#define FREQ_SOURCE_BIT 4

The #define is a preprocessor command that tells it to replace the first symbol with the sequence of rest. It is a crude way to set an alias.

Though now you know what the pin is used for and why it is used. Because of the explanation you can devise a strategy when porting the code for an another platform that does not have the exact same pinout and use a conditional macro block to adjust the behavior.

Reading the input is simple as:

// clears the bit, so that the pin is used as an input. (this comment is redundant because you are expected to know this)
FREQ_SOURCE_DDR &= ~(1 << FREQ_SOURCE_BIT);

// reads the input into a value (this comment is redundant for the same reason as earlier one)
value = (FREQ_SOURCE_PIN >> FREQ_SOURCE_BIT) & 1;

If you find out you are writing that above text all the time, you can do this (and you should only do it when you do have to write a read command often):

#define FREQ_SOURCE_IN ((FREQ_SOURCE_PIN >> FREQ_SOURCE_BIT) & 1)

So now you can write:

value = FREQ_SOURCE_IN;

Looks complicated? Well, if you are about to write a computer program that does something more than flips a switch, you have to understand every and each operation done here. The above weird looking symbols are Bitwise operations. They are very common in computing, a basic subject you have hard time to miss if you really get into computer programming. It's maybe not the first lecture you should take, but "Bitwise operations" could very well come in the chapter 6 of the "Basics of Computer Programming".

Likewise understanding such simple substitution macros belong in the basics of C/C++ programming.

But wait! Didn't this remove the portability?!

Some may claim that removing the things such as the digitalRead or the notorious attachInterrupt would remove value. After all, if you only use Arduino to read whether your room's door is open, you would make your door opening switch program portable across all Arduinos if you used digitalRead instead of raw port manipulation. Also all Arduino projects do not have timing limitations although many have.

There may be a merit in Arduino's helping wheel libraries. They behave in this manner anyway. If you do not use the digitalRead, then it is not provided. If you can cope up with the simplifications and never end up needing the deeper behavior within Arduino, then you'd be perfectly happy with these beginner-friendly libraries.

What!! I'm not saying that digitalRead or attachInterrupt is a menace that should be erased from Arduino?! Before we can do that conclusion, it has to be weighted.

Here are the good things of the situation:

Here are the bad things:

Making it easy for everyone to comprehend any program thrown in front of you, rather than trying to do it easy for beginners, would not introduce more things to learn.

You can see the bad things start to collect much faster than the good things. Considering we have alternative ways to use the time spent to implementing features, there could be a really good argument there to say that such data hiding abstractions are an anti-feature.

You spend time to make it. People who use it spend time to learn and use it. Some people win a little. Many people lose a little. In total a lot of time and attention is lost. The thing that should be done won't get done. That's the tally of hiding details in implementation and data, in this case.

A school subject

What if you hide the details so well that they cannot be accessed? Is that possible? Encapsulation and information hiding are taught principles in object oriented programming courses. Three intents can be identified:

Hiding information in your software is deemed as an abstraction technique having NO ILL EFFECTS. Moreso, although it is not said, it is thought that once something has been hidden you don't have to tell about it to anybody else.

Documentation is the first place to suffer from the information hiding. Because the internal details should not be visible outside they are not documented anywhere at all. It is not a good situation when something breaks.

A thought play

Lets say that you have a piece of program that should not change. An another program is interfacing with that piece and the interface shouldn't change in order for the application to work.

The interface in your program stays same and is allowed to extend, until a next major version change.

The interfaces are bound to the software that provides them. Changing the interface makes everything break, so you don't change it, ever.

Phillips vs. Torx

For a long while people used to fasten everything with Phillips screws. The screws used to have a problem that the screwdriver slipped and dulled the screwhead. Torx heads didn't have this problem, but they only recently became cheaper and more widespread everywhere.

The change was smooth and nobody really noticed much. There were no crisis over threading or size changes.

Screws and bolts today are highly standardized. The standardization process begun when it became feasible to mass-produce metallic screws.

Standardization

Standard is an established way to design a system that ensures interoperability with all the other parties. It is usually written down into a formal document.

The standardization is a slow process. Before the standardization there will be products and pieces that are different and do not fit together. This way the early problems and flaws hopefully do not end up into a standard.

Any product is free to conform or diverge from a standard. Also any product can conform to multiple similar standards.

Some people require that the standards are made by organizations meant for it. But in practice you can have useful standards that come from a paper or specification written by an individual company.

Standards do not require that internal details are hidden from the user, unless it is for the safety of the user or the machine.

Standards allow experimentation, because the interfaces that conform to standard are identified. Experimental interfaces are allowed to exist along standard interfaces. Eg. once you have one Torx screw, you don't have to replace all the remaining Phillips screws with Torx.

In programming environments you often face kind of version incompatibility dilemmas though. The software does not interoperate across interfaces because the interface is tied to the version of the software.

~~A whole industry gets it wrong.~~ Ah darn. This is taught wrong in the schools. A whole industry must get this wrong.

Root cause

The whole problem here is that we think all wrong about the purpose of abstractions during programming.

Programmers think about the users of their code as inequals, that their users can't peek into the program, understand it or change it.

The problem is that often it's not easy to define the boundaries for hiding things.

Lets take an another real-world example that illustrates this problem.

Microwave ovens

Most real-world items such as fridges, fans, hammers, drills cannot really ensure that the users use them correctly. They often simply do not have the means to add safety or hiding mechanisms without introducing very obvious costs. They only do those things that are feasible to do.

For example, microwave ovens prevent you from starting the oven while the enclosure is open. This feature avoids you from receiving microwave exposure from the oven.

Yet you can do lots of stupid things with microwave ovens by attempting to microwave the following things:

You could attach a camera into an oven and require that any product that is microwaved comes in an enclosure with a product identification in it, which allows the microwave to retrieve the working parameters and ensure that the product belongs into the microwave oven.

Well, the user could still microwave something else along the item, you would have to solve that too somehow. To be for safety, these features should prevent the microwave from being otherwise. Therefore those features combined would ensure maximum inconvenience for the users of your microwave oven:

So if you can't make microwave ovens safe by introducing safety layers, how happens that people do fine with their microwave ovens? The reason is simple.

People know that the microwave oven heating is a violent procedure that heats unevenly, that the microwave doesn't like shiny objects, and that you should use it for foodstuff if you intend to eat the stuff that was inside it. They know what the thing does and whether it does what they want is left for them to decide.

And this is how it should be with computing as well. The details aren't really as bad as the things you would have to do to hide them.

Update 2017-10-12: In the lobste.rs comments Geoff Wozniak correctly points out that writing out "a whole industry gets it wrong" is a naive extrapolation. But oh wait. Why is it taught in schools if a whole damn industry doesn't get it wrong?

Encapsulation and the data hiding implemented that way is treated as one of the fundamentals of OOP, and OOP is taught in schools.

Also, this post really wasn't about "leaky abstractions" as it. That's a different blog post and I'm not sure I agree with that guy. The problem is that these abstractions achieved by hiding add much more work than what they take away. It is not the only problem that they leak. The main problem is that using abstraction for hiding things isn't how it works in the first place. That was the point of writing this post.

Similar posts