In Defense of Details
People are generally overwhelmed by amounts of detail in programming and their intuition is to hide it. It is so ubiquitous that in the Abstraction_(disambiguation) there reads:
- Abstraction (software engineering), a process of hiding details of implementation in programs and data
Unfortunately, this results in more things to learn and hide.
An example (and another) in Arduino
An example of this can be found in the Arduino, a very successful electronics prototyping kits used by hobbyists and professionals around the world.
Arduino circuit board contains digital pins that are helpfully labeled. This labeling is a good and a helpful way to quickly reference each pin.
In the arduino library there is a function to read the value coming into each digital pin. Here's an example of reading the pin 4 and storing it into a variable:
value = digitalRead(4);
This adds some portability and attempts to hide a detail. The Arduino is based on the AVR family of microcontrollers. The pin 4 on Arduino UNO is actually the pin 4 in the Port D of the Atmega328. Here is the corresponding AVR code for reading the pin 4:
value = (PIND >> 4) & 1;
The above code shifts the fifth bit first (
>>4) and then
erases the higher bit values (
&1), then it stores
value=) the result into the variable.
digitalRead functionality is meant to be helpful for a
beginner and helpful for portability, but beyond the simplest
read-a-switch -applications it will end up only being an
extra detail you have to memorise, with no other use.
depending on what your application is doing the pins are not
interchangeable in functionality. In that case, if you move
the wire from pin 4 to pin 5 or 6, and change the program
accordingly, it may break. Also, if the functionality is not
on that pin on a different Arduino microcontroller card, the
application will not be portable even if it would use this
function. On top of that, the
digitalRead is in order of 10
to 50 times slower than
reading the value directly. Many microcontroller projects
are time critical. If you run out of the time to do your
digitalRead is the first function to vanish in
The Atmel datasheet lists the following Alternate Functions for the PD4:
XCK (USART External Clock Input/Output)
T0 (Timer/Counter 0 External Counter Input)
PCINT20 (Pin Change Interrupt 20)
The XCK is unfamiliar to me because I've not used USART. I am sure it is an interesting feature but there is a lot to tell about so I skip it.
T0 is a functionality that would allow you to set one of
the timer/counter units on the microcontroller to listen at
this pin, and increment whenever there is a transition from
zero to one, or vice-versa, on this pin. I've seen this used
once, and the code was really not portable from Arduino UNO
to micro, for example. Yet the functionality was necessary.
The alternative functionality exists on the microcontroller because it is convenient. Due to the cost, you cannot provide every functionality on every pin. For this reason they have been spread across the pins with the hope that the user does not need the overlapping functionality in a single application.
Pin Change Interrupt means that you can call a
function when the pin changes. The
interrupts react all-at-once to one interrupt.
There is a
PCINT function for every pin on an Arduino UNO
(this is not true on the other Arduinos). Coincidentally the
Arduino also has a tool to associate a function with a
specific pin change. That also sees little to no use for a
Interrupts are small chunks of programs that run over other programs whenever some external event occurs. During the interrupt the application cannot proceed. Because of this the programmers attempt to keep the interrupts short as possible.
If you end up calling the
attachInterrupt() on Arduino,
you will introduce an interrupt that is long enough to
really delay every possible action that is taking place.
You pay a terrible price, have to remember yet another detail, on top of those details that it does not hide, you gain nothing.
This is a funny and a frustrating theme across the Arduino. The intuition to use abstraction to hide is a plague among the Arduino community, resulting in simply bad code and stupid ideas.
On an Arduino micro, there is a builtin LED behind the pin 13 which is used in simplest applications to simplify a hello-world application, requiring no assembly. Also the Arduino's bootloader is flashing that led every time when you upload a program on the microcontroller.
The builtin LED shares a function "Input capture with
Timer/Counter 3". The functionality is always active because
it does not interfere with anything. If this pin is set as
an input, it will immediately store a value from the timer
whenever there is a
1->0 transition in the pin.
The input capture is extremely useful feature for reading various unusual signals. The builtin LED is partially redundant and makes all those input functions much less useful or even unusable.
It is natural
To be overwhelmed by all the details is natural. Get enough of those magic numbers on the Arduino and you're easily overwhelmed.
Becoming overwhelmed by details is entirely natural, yet in computing it results in exactly the opposite result of what you desire. On top of that the complexity you introduce is left for the future generations to "enjoy".
Perhaps this article was written because we received enough of this kind of "technical debt".
The case Arduino digitalRead fix
There's an abstraction to cope with the complexity. It also helps with porting the application. You do the following in the beginning of the program:
// The Pin 4 on Arduino UNO, we are using the T0
// functionality to read the frequency of the incoming
#define FREQ_SOURCE_DDR DDRD
#define FREQ_SOURCE_PORT PORTD
#define FREQ_SOURCE_PIN PIND
#define FREQ_SOURCE_BIT 4
#define is a preprocessor command that tells it to
replace the first symbol with the sequence of rest. It is a
crude way to set an alias.
Though now you know what the pin is used for and why it is used. Because of the explanation you can devise a strategy when porting the code for an another platform that does not have the exact same pinout and use a conditional macro block to adjust the behavior.
Reading the input is simple as:
// clears the bit, so that the pin is used as an input. (this comment is redundant because you are expected to know this)
FREQ_SOURCE_DDR &= ~(1 << FREQ_SOURCE_BIT);
// reads the input into a value (this comment is redundant for the same reason as earlier one)
value = (FREQ_SOURCE_PIN >> FREQ_SOURCE_BIT) & 1;
If you find out you are writing that above text all the time, you can do this (and you should only do it when you do have to write a read command often):
#define FREQ_SOURCE_IN ((FREQ_SOURCE_PIN >> FREQ_SOURCE_BIT) & 1)
So now you can write:
value = FREQ_SOURCE_IN;
Looks complicated? Well, if you are about to write a computer program that does something more than flips a switch, you have to understand every and each operation done here. The above weird looking symbols are Bitwise operations. They are very common in computing, a basic subject you have hard time to miss if you really get into computer programming. It's maybe not the first lecture you should take, but "Bitwise operations" could very well come in the chapter 6 of the "Basics of Computer Programming".
Likewise understanding such simple substitution macros belong in the basics of C/C++ programming.
But wait! Didn't this remove the portability?!
Some may claim that removing the things such as the
digitalRead or the notorious
remove value. After all, if you only use Arduino to read
whether your room's door is open, you would make your door
opening switch program portable across all Arduinos if you
digitalRead instead of raw port manipulation. Also
all Arduino projects do not have timing limitations although
There may be a merit in Arduino's helping wheel libraries.
They behave in this manner anyway. If you do not use the
digitalRead, then it is not provided. If you can cope up
with the simplifications and never end up needing the deeper
behavior within Arduino, then you'd be perfectly happy with
these beginner-friendly libraries.
What!! I'm not saying that
attachInterrupt is a menace that should be erased from
Arduino?! Before we can do that conclusion, it has to be
Here are the good things of the situation:
- If your project is really simple, these features will help you avoid learning things in order to carry through your project.
- In electronics projects you have lot of things to worry over about. It can be great to make things really simple in the beginning so that you can mull over other unavoidable details such as the reason why a LED needs a resistor in front.
Here are the bad things:
- If you are not attentive, you will never find out that Arduino can be used in a better manner. The simple function hides all the features.
- There are more things in general to learn before you have
mastered Arduino, because along the
digitalReadyou would still have to learn all the details too.
- The basics you skip now are not necessarily something you can skip indefinitely. It may be a better to give a nose-dive into them rather than try to avoid them.
- You will be surprised, perhaps even appalled, when you find out that the pin choice affects what you can do with the signal. The abstraction "the pin does not matter", is a broken promise in the end.
- If the information was better organized for the pro, it would speed up the work of everyone working with the platform. An interactive pin®ister&feature map online about every Arduino card could be a real timesaver.
Making it easy for everyone to comprehend any program thrown in front of you, rather than trying to do it easy for beginners, would not introduce more things to learn.
You can see the bad things start to collect much faster than the good things. Considering we have alternative ways to use the time spent to implementing features, there could be a really good argument there to say that such data hiding abstractions are an anti-feature.
You spend time to make it. People who use it spend time to learn and use it. Some people win a little. Many people lose a little. In total a lot of time and attention is lost. The thing that should be done won't get done. That's the tally of hiding details in implementation and data, in this case.
A school subject
What if you hide the details so well that they cannot be accessed? Is that possible? Encapsulation and information hiding are taught principles in object oriented programming courses. Three intents can be identified:
- Make it safe to change details because they are not exposed.
- Protect the user from the incorrect use of the system.
- Prevent the user from moving the system into an inconsistent state.
Hiding information in your software is deemed as an abstraction technique having NO ILL EFFECTS. Moreso, although it is not said, it is thought that once something has been hidden you don't have to tell about it to anybody else.
Documentation is the first place to suffer from the information hiding. Because the internal details should not be visible outside they are not documented anywhere at all. It is not a good situation when something breaks.
A thought play
Lets say that you have a piece of program that should not change. An another program is interfacing with that piece and the interface shouldn't change in order for the application to work.
The interface in your program stays same and is allowed to extend, until a next major version change.
The interfaces are bound to the software that provides them. Changing the interface makes everything break, so you don't change it, ever.
Phillips vs. Torx
For a long while people used to fasten everything with Phillips screws. The screws used to have a problem that the screwdriver slipped and dulled the screwhead. Torx heads didn't have this problem, but they only recently became cheaper and more widespread everywhere.
The change was smooth and nobody really noticed much. There were no crisis over threading or size changes.
Screws and bolts today are highly standardized. The standardization process begun when it became feasible to mass-produce metallic screws.
Standard is an established way to design a system that ensures interoperability with all the other parties. It is usually written down into a formal document.
The standardization is a slow process. Before the standardization there will be products and pieces that are different and do not fit together. This way the early problems and flaws hopefully do not end up into a standard.
Any product is free to conform or diverge from a standard. Also any product can conform to multiple similar standards.
Some people require that the standards are made by organizations meant for it. But in practice you can have useful standards that come from a paper or specification written by an individual company.
Standards do not require that internal details are hidden from the user, unless it is for the safety of the user or the machine.
Standards allow experimentation, because the interfaces that conform to standard are identified. Experimental interfaces are allowed to exist along standard interfaces. Eg. once you have one Torx screw, you don't have to replace all the remaining Phillips screws with Torx.
In programming environments you often face kind of version incompatibility dilemmas though. The software does not interoperate across interfaces because the interface is tied to the version of the software.
~~A whole industry gets it wrong.~~ Ah darn. This is taught wrong in the schools. A whole industry must get this wrong.
The whole problem here is that we think all wrong about the purpose of abstractions during programming.
Programmers think about the users of their code as inequals, that their users can't peek into the program, understand it or change it.
The problem is that often it's not easy to define the boundaries for hiding things.
Lets take an another real-world example that illustrates this problem.
Most real-world items such as fridges, fans, hammers, drills cannot really ensure that the users use them correctly. They often simply do not have the means to add safety or hiding mechanisms without introducing very obvious costs. They only do those things that are feasible to do.
For example, microwave ovens prevent you from starting the oven while the enclosure is open. This feature avoids you from receiving microwave exposure from the oven.
Yet you can do lots of stupid things with microwave ovens by attempting to microwave the following things:
- Household items not meant into an oven.
- Shiny objects.
- Any non-food stuff.
You could attach a camera into an oven and require that any product that is microwaved comes in an enclosure with a product identification in it, which allows the microwave to retrieve the working parameters and ensure that the product belongs into the microwave oven.
Well, the user could still microwave something else along the item, you would have to solve that too somehow. To be for safety, these features should prevent the microwave from being otherwise. Therefore those features combined would ensure maximum inconvenience for the users of your microwave oven:
- Your users would be no longer able to choose how much of food they microwave at once.
- They would have to buy products with the identification marks, and could not use the oven to heat foods people are not usually putting into a microwave oven because those would not have the identification in it.
- They could not reheat food because the machine cannot ensure what's in the enclosure if someone has opened it.
So if you can't make microwave ovens safe by introducing safety layers, how happens that people do fine with their microwave ovens? The reason is simple.
People know that the microwave oven heating is a violent procedure that heats unevenly, that the microwave doesn't like shiny objects, and that you should use it for foodstuff if you intend to eat the stuff that was inside it. They know what the thing does and whether it does what they want is left for them to decide.
And this is how it should be with computing as well. The details aren't really as bad as the things you would have to do to hide them.
Update 2017-10-12: In the lobste.rs comments Geoff Wozniak correctly points out that writing out "a whole industry gets it wrong" is a naive extrapolation. But oh wait. Why is it taught in schools if a whole damn industry doesn't get it wrong?
Encapsulation and the data hiding implemented that way is treated as one of the fundamentals of OOP, and OOP is taught in schools.
Also, this post really wasn't about "leaky abstractions" as it. That's a different blog post and I'm not sure I agree with that guy. The problem is that these abstractions achieved by hiding add much more work than what they take away. It is not the only problem that they leak. The main problem is that using abstraction for hiding things isn't how it works in the first place. That was the point of writing this post.