Visual Programming

This is my rough sketch of visual programming

I have had obsession to visual programming for 5 years or more. Once I worked on this kind of project for three months without paying attention to anything else. That much time I've wasted to come up with an alternative to programming with plain text editors, yet I still think I should keep working on. Properly designed visual programming editor would save lot of effort in programming.

It might be unclear what I mean with visual programming. It is a sort of an environment, where you could directly manipulate program structures with keyboard instead of writing a text file and letting computer to parse it. The written structures would appear on the screen in a beautiful, compact notation. The representation would somewhat affect and depend on how the program was written in. Whenever language semantics change, new structures could be introduced with new ways to notate them.

Effects on learning

Visual programming would allow us to have very reliable tools to assist in writing correct syntax. In other hand, what you write with the keyboard would no longer match as literally what you see in the screen. It is difficult to say whether the effect on learning would be positive, but I'm trying to express the potential here.

People require more complex things from computers, than they did before. You may need to adjust what kind of effect a brush produces, or you have to configure a program using a configuration file. All kind of interaction with computer could be thought as a subtle kind of programming. You give commands that evaluate as you give them. You might remember Greenspun's tenth rule, it could be that this same aforism translates to programming editors. Any sufficiently complicated user interface, might in fact be a bug-ridden, inefficient implementation of visual programming.

brushes

Learning the syntax of a programming language is not an incredibly hard problem. People seem to naturally learn any programming language. It happens by introducing big code snippets they can use, then make the snippets smaller until the snippets are semantic structures of the programming language. I've heard about small children taught to program this way. Visual programming would not eliminate this kind of learning, but it could make it more useful. Fast keyboard input required by experts could come in the same system along an easier mouse-centric interface meant for children, disabled and elderly.

There also exist graphical programming tools meant for children or non-programmer people. Scratch and Blender Cycles, the most well known examples. They are both less effective and more cumbersome to use than the tools meant for programmers. Another problem is that they rely on environments that are insulated from the scripting interfaces. Even if you knew how to work with smaller pieces, you're just getting the duplo blocks. Maintaining separate environments for different demographs is difficult, therefore the bias is on the more common demograph.

It's not just wasted effort to maintain separate environments for different programmers. Switching to better tools means that you're crossing a barrier. A visual programming environment could serve all kind of users, giving them more natural path to becoming experienced programmers.

Readability

Better the programmer, more he values readability of his programs. Best of them supply useful documents and references along the neat, easy-to-read sources. This is one thing that visual programming environments would certainly improve. Consider the next example:

Algebraic notation for the code below

In text, the same code would be expressed as:

effect = mass / (radius**2 * distance) + 10
color  = rgba("#8ce219ff")

I suspect what I just shown is not even the best what it could achieve. The idea is that the programming environment would provide good defaults for the layout of the code that could be altered when it doesn't match the context. If the structure layout wouldn't work for the code, it could be notated by a different layout which works. In extreme, programs could supply few layouts on their own and the layout engine would be standardized. This would let you write very readable programs.

Uniform programming environment

How many different notations there are for program block? Five or six? I've seen the following notations:

Sub sandwitch (alpha as String)
End Sub

: sandwitch pop
;

function sandwitch (alpha) {
}

function sandwitch (alpha :String) {
}

sandwitch = (alpha) ->

def sandwitch(alpha):
    pass

void sandwitch(char* alpha) {
}

def sandwitch (alpha)
end

function sandwitch (alpha)
end

(define sandwitch (alpha)
    )

sandwitch alpha =
    None

sandwitch(Alpha) :- true.

It's easier if you have structures. You can specify default ways to describe certain structures, maybe even let the programmers define their own variations they include along their programs. Notational difference between languages is easier to handle than the syntactic one. Also in a graphical environment it would be easier to come up with good notations. A single language could easily contain multiple notations for different uses, which would make it possible to use a single high-order programming language for everything. There would be less programming languages simply because there would be less to argue about and less pressure to create yet another pet language.

Contemporary uses

Tree structures a very common way of organizing things. I already told about the example about drawing brushes above. Did I mention that I've recognised several other situations where that kind of interface would be useful? You could use them to represent musical notes for a synthetizer. They might make spreadsheet more versatile and help with fetching and storing them in a database. Both HTML and CSS are tree structured in their nature, so those would transform into structured editing as well.

Configuration files

Your Linux system has hundreds of configuration files that describe instructions for programs to adjust for the user. They are mostly text which is fine for programmers. The problems arise when non-programming user wants to configure his linux system. Today linux distributions provide separate GUI for configuring common parts of the system. That does not cover the whole linux configs and it duplicates the text based features. It's also really complex and blurred by how to change any configuration programmatically.

So instead of this you could have nice configuration tool based on the visual programming environment. It would be easy enough and controlled process for layman to change the configuration to their preferences while being the exact same system used by programmers to change the configs. To provide the settings panel, /etc and ~/.config could be simply scanned for configuration files.

Every configuration directory could contain an annotation file along the configuration, which tells how to restart a service after config change, and which kind of controls are presented to the user. Text-file based configuration would be left in as an option for those who want them.

It's doable!

I can't be hundred percent certain about it, but I believe good visual programming editors are doable. It will require careful design to avoid complications though. Afterward some standardization is required to have a common layouting and interchange format between different editors.

I've been through maybe, 4 or 5 iterations of designs for the editor. During working on them I've got some idea about what are the limitations and possible dead ends. For example the visual output needs to form a programming language in itself. It should get inspiration from TeX. Input is equally difficult because it needs to match with the output a bit and be convenient to use.

I think there's already designs that work for the input. The commands you type simply transcends the same problems that arise in traditional text based programming. Parsing techniques can be applied. I've got one example here about one parsing algorithm applied to structured input.

Shunting-yard input

Shunting-yard illustration

The Shunting-yard algorithm is one of the simple parsing algorithms to understand. It parses structures out of infix syntax, obeying precedence rules. There is the input expression, output and operator stacks. Operations such as addition and multiplication go into the operator stack.

The operator stack has a requirement that it's sorted by precedence. The lowest operator in the stack is required to have lowest precedence. Whenever an operator comes that has lower predence than the highmost item in the stack, the operators are dropped into output stack. When an operator arrives into the output stack, it will capture two topmost values before going into the output stack.

There's no problem in applying this to structure programming. The caret will simply imitate the behavior of the shunting yard algorithm. You move the caret into the place and type the expressions while the caret determines correct place in the structure for what you just typed. Here's a group of pictures about how it might work:

Shunting-yard input

Have some simple rules to reorder the surroundings of the caret during inserting and removing pieces, and you've got a simple way to edit expressions just like how you'd do it in plain text. Except that it's all stored in structures.

The point

Point of this post was to sell you the idea of structured, visual programming. That explains the diagrams and graphics spam. I did my best, as this is difficult subject and many people are quite happy to the current state of things.

It is possible that there never will be a visual programming platform that ends up into widespread use. Text files are extremely versatile as a format. Half of the world basically shifts text files around to represent structured data. Even if a great environment were designed, it is questionable whether it would ever ends up into widespread use.

How would you copy&paste structured programs around? The small cut of a program should somehow be interchangeable with a whole document file. It should have a very reliable format that stays relevant for decades. Those are solveable things. But what if you want to search the web with a snippet of code? Google would have to support it. What about forums, email? It is an extreme amount of software that deals with text now. It works, why would anyone want to change it?

But I'm still excited about the idea.

Similar posts