Journeys of Lever
Last week I integrated libuv into Lever. So far the decision appears to be good. LibUV interfaces are clean and the library allows me to concentrate important subjects instead of having to worry about portable async basic I/O.
It takes a while before every I/O function in Lever is async. I will be gradually introducing more and more libuv functions into the runtime as I need them.
I've been studying what to do next. It turns out there are several independent journeys I could do along Lever development.
As lever gains more utilities for I/O, it is important to have good routines for operating with bytes and buffers.
Implementing decoders for GZIP, TTF, JPEG or PNG formats shouldn't be a serious struggle. Interfacing with zlib should go smoothly and efficiently.
- byte array concatenation
- String utf8 encoding, decoding utilities
- String building
- Byte array building
- Boolean operations with bitarrays, bitarray building.
Lever has internal documentation fields, but I haven't written the documentation system or the engine that reads the documentation.
Lever exception and syntax error reporting, pretty printing and visualization tools aren't top-notch yet.
Documentation should be always available via the REPL. For example, readable with an alternate screen.
To provide much of the functionality into the VR, lever needs an OpenGL framework of some sort and support for font rendering.
Some bitmap signed distance field monospace fonts should be enough to bootstrap it. In practice byteshuffling journey must be completed with TTF loader before we can conclude FontGL journey.
Starting point for this one would be in the freetype-gl library.
Regex support is a basic thing Lever is still missing. After looking around the likely way to solve it is to write your own regex engine and JIT optimize it.
The chartparser module should be upgraded with right-recursion optimization and splitting the parsing state into DAG-charts, to allow fancier uses for the chart parser.
Afterwards the grammar language should be improved, as well as the syntax of the whole language.
The compiler could use some improvements to allow use by third party modules. The design of the compiler appears to be otherwise working.
As I've studied PyPy JIT, it shows out to have some consequences that are inobvious. For example, trivial implementations might not be inefficient. Also whether the VM uses registers or a stack to shuffle values doesn't seem to matter at all.
This means that there must be a better way to implement what I've been doing. The rules have changed and I don't know the new rules.
I was adviced that I should read the Partial Evaluation and Program Generation -book.
I also found some pages I will likely read through again:
- RPython JIT documentation
- Overall pypy documentation
- Overall rpython documentation
- PyPy JIT tutorial
- PyPy papers
As Lever is nearing to be a practical language, I soon need to optimize it. To do this some benchmarks need to be written. Then they need to be analysed to provide the correct optimizations.
Most of optimizations for now would be simple ones and well-known. Such as using a map in attribute access, or making the VM frames virtualizables.
Type inferencers and Lever-to-stuff translators. Allowing the writing of these utilities was one of the goals in writing the Lever in the first place.
Likely the first place for this would be to continue with the spirthon module in Lever. Also the first target could be GLSL rather than the SPIR-V.
Crazy math journey
Lever could use its own Computer algebra system and an autodifferentiator. It should be verified that Lever can be used for writing some crazy differential geometry code. Also the existing vector algebra and randomness modules should be revised.