Back to Visual Programming and Structure Editors

Structure editors and visual programming could be dead-on-arrival -concepts; There are too many details that need to be solved for it to be practical and the benefits look and feel meager. It seems you would mostly get a peace of mind from coding in structures. But I've obtained reasons that make it worthwhile for me to pursue these ideas once again.

Lisp

Choosing between lisp and other languages is a tradeoff. Lisp is more generic and encourages very solid semantics. You only need to learn few special forms to use lisp. Therefore it is easy to learn. Lisp can evolve to fit new purposes and new users. Unfortunately it is hard to read because there are lot of parentheses. And a badly matching parenthesis spanning multiple lines is hard to detect but may profoundly change meaning of the program.

There have been attempts to produce readable notation for lisp. David A. Wheeler, author of the readable lisp expressions, believes those languages lost the advantages of lisp expressions. Lisp syntax is distinct and valued because it is generic and homoiconic. I believe he's right, because if a language maintains these properties, it probably belongs to the lisp family.

Genericity means that the syntax is detached from semantics. In lisp the semantics come from how the lists are interpreted.

Homoiconicity means that the underlying data structure can be recognised from the syntax. In lisp parentheses form a list. It is clear that everything that is not an atom is a list.

If you use box instead of parentheses to denote a list, you get a visual programming language instead of lisp. It is generic if the data structure would be detached from semantics. It is Homoiconic if the visualization maintains certain level of invariance: Items will stay within boxes and they won't shuffle around in the representation.

If these properties are kept invariant, and if the users choose their semantics carefully, then the editor should be able to handle anything you throw at it. If you remove the unnecessary while you maintain the homoiconicity and genericity, you end up with the following production rules:

atom -> List label, [atom]
      | Marker label
      | Symbol label, text
label -> string | null
text  -> string

label must be in place for semantics. Otherwise you need to denote the representation with symbols for semantics. This is the minimal viable syntax for a visual programming language. It becomes burnensome to attempt to do it with less, or more.

This kind of clearly defined, clear structure is important for the user. It is very difficult to operate the editor without having an intuitive notion of what is it editing. It is also important as a provider of stability and freedom over the remaining design. Problems related to structure editors are difficult.

Oculus Rift

I have heard the DK2 has progressed a lot since the DK1 that I experienced. So many things have improved that I'm expecting the "Civilian Version 1" to be qualified for many things beside gaming. These devices mean for new ways to represent information to user.

For some things such as modelling it is obvious how Oculus Rift might help. But computer programming may benefit from it too. For the first time we may show interactive holograms to the user with affordable hardware.

The first people wanting to develop inside HMD are the developers who create HMD-enabled software. It is frustrating and jarring the workflow to constantly remove and wear the headset when you're experimenting with visuals. People could keep coding just like usual within a headset. But there are more possibilities so everybody should be more open for new alternatives.

Only communicative text is necessary

Many programming languages attempt to minimize the amount of text used to represent structure of the code. Python is considered readable. If you take the structure apart from the text in a factorial, you get 12 tokens of each:

def factorial(k):
    if k <= 1:
        return 1
    else:
        return k * factorial(k-1)

def ( ) : if : return else : return ( )
factorial k k <= 1 1 k * factorial k - 1

You could argue that the * is structure, and technically you'd be correct. But I consider the * as a function. The infix notation is the structure. Lets consider Lisp factorial:

(define (factorial k)
    (cond
        ((<= k 1)
            1)
        (else (* k (factorial (- k 1))))))

( define ( ) ( cond ( ( ) ) ( else ( ( ( ) ) ) ) ) )
factorial k <= k 1 1 * k factorial - k 1

There are 12 tokens, just like there were in python, but nearly doubled amount of structure, 21 tokens! For this example, the structure/data ratio of Lisp is 1.75, whereas for python it is 1.0. Other languages fall somewhere in between.

We refer to the data tokens as the communicative part of the program. The data lacks everything that forms the structure. Ultimately you try to minimize the amount of structure tokens by introducing syntax, but even in the best case you have one structure token for every data token.

Now if we move to structure editors. If we're representing the same factorial example in our structure editor, we'll have 12 data tokens, just like python or lisp had. The data tokens are the kind of text we cannot discard. But we no longer have any structure tokens. It's up to the writer of the code to decide whether he wants to denote structure somehow. As long as the language accepts the communicative part in the order, you may decide how to visualize it.

Programmers already think about code in terms of semantics rather than text

I've noticed I'm not really coding in text when using text editor. Rather, I'm using keystrokes to transform the meaning of the program. It is visible in a workflow recording I made few months ago. The code constantly visits a state where it's structurally valid. If I design an input system that takes this into account, it could let me actually be faster with structure editors than I am with vim, which is a highly optimized system for editing text.

One important detail aside this is that writing text tends to be a bottom-up experience. The data tokens appear before the structure is complete. An enjoyable experience of using structure editor probably requires something similar. You should be able to introduce the data tokens first, and have ways to glue them together with structures later on.

Browsers already solve some code layouting problems

How to layout and represent the code to the user is one of the big obstacles in designing a well working structure editor. I found a nice article about the browser internals. Having read that I believe it's easier to design a layouting engine that is good enough. Modern web browsers do not entirely solve the problem of layouting for code, but they provide a nonzero starting point.

I made a prototype people can try

If someone is not studying and designing structure editors the benefits and deficits will be never known. We may speculate as much as we want, but that isn't better than actually trying it out.

I have tried to implement visual programming numerous times. Having done that often makes me one of the few people who could actually create a functioning structure editor. It seems that in this sense I have to finish what I started.

My latest structure editor prototype is still missing important features. But you can try it and get a feeling of how the editing would work. It is open source. You can fork, modify, send proposals with issues system and follow it's overall progress in github. The editor implements a keyboard controlled modal input scheme.

I've got a upgrade coming to the layouting engine, and the input processing. Both made possible by having this prototype up in the web. I'm planning to bootstrap the editor into a programmable programming environment. But it is not useful prior a nearly complete input processing.