Thinking out loud: a scripting language for Cantabile

Cantabile
21

I’ve skimmed through, and will read in more detail when I’m back at my desk but am very interested in this. On my “act like a regular keyboard” setup, I’ve got lots of very similar bindings spread across 8 buttons (that either switch to a rack / trigger the next patch in this rack / cycle back to the first patch in this rack, all while keeping LEDs in sync on the keyboard).

If I can reduce these to a single (or few) bind with a code block then that would make things so much simpler. Is the number of items in a rack exposed anywhere in the system? That would certainly help me cycle from last to first, as it’s currently a very manual setup.

My background’s JavaScript, VBScript, and Python, but I’m always willing to learn

1 Like
23

Hi Brad,

From my perspective (I guess I’m a power-user, since I teach PL theory and design at the PhD level, and am comfortable with any language), the API is more important than the language. I view Cantabile as a VM since it hosts VSTs, which means that its current power is mainly limited by the API it exposes to the scripts and plug-ins it hosts. For example:

  • An API that allows setting the transport position would allow development of scripts and plug-ins that repeat choruses or pause fermatas live.
  • An API for setting tempo would allow development of code that performs fine tempo adjustments based on MIDI event timings.
  • An API for receiving display-change events from Cantabile would allow development of deep integration plug-ins that reflect Cantabile display elements to control surfaces.

All of these are limitations of the API rather than the specific scripting language. If Cantabile had an API that allowed existing plug-in formats (e.g., VST3) to access Cantabile’s major functionalities, I’d certainly develop for it. Such an API would free users to write code using any language/compiler they like, which would maybe dodge the problem of “can’t please everyone”.

1 Like
24

Reaper uses 2 primary scripting languages Lua and ELL. Lua is used for general scripting and ELL is used to create effects plugin’s. It’s very fast and can access memory etc.

That said, Javascript is lightyears away from where it was when Lua was developed. Javascript and Node have a lot to offer and speed as well. Lua over in Reaper land didn’t support modules which was very painful for developers there. I was actually the one that figured out how to make modules work via a bit of a kludge. From my experience there and as a node developer, I think javascript/node would be superior to LUA in every respect including speed.

1 Like
25

I’ve been experimenting and put together a couple of interesting things.

Geeky tech discussion incoming…

Runtime Environment

Not complete, but I’ve made a good start on a runtime environment for a strongly typed, garbage collected language:

  • A type system that supports native types (bool, integers, floats etc…), strings, typed arrays and typed dictionaries (still need to add user types, aka classes)

  • A single threaded, incremental garbage collector that supports global roots, stack frames and pinned objects.

  • The GC is non-compacting, non-relocating and just manages memory over the top of a regular memory heap (eg: malloc/free).

  • The underlying heap is pluggable so I can use Cantabile’s internal non-blocking heap manager on the audio thread.

  • All the internal structures used by the GC are O(1) for insert/delete

  • A complete GC cycle is typically O(n) where n is the number of allocated objects.

  • Both the mark and sweep phases of the GC are incremental so the entire cost of the garbage collection can be amortized over a long period of time (instead of having to cause a long GC induced, glitch provoking pause).

  • The incremental aspect of the GC is tunable in that it can be told how much work to do each cycle, but I need to come up with an algorithm to tune this as it runs.

  • The GC algorithm is a tri-color garbage collector with write barriers.

  • The entire runtime/GC is written in good old plain C (not even C++). While I wouldn’t want to develop full time with it, C is such an elegantly simple but powerful language that I really enjoy working with when going low level.

All that’s to say, it seems like a garbage collected language running on the audio thread should be possible.

Language Compiler

Although the runtime sounds cool, it is a complete pain to use. Even just writing some simple tests for it was painful.

So I’ve started on a compiler for a C# inspired scripting language. By C# inspired I mean a language with basically the same syntax, but more easily hosted and more aimed at scripting (without requirement for namespaces/classes, without a huge class library but with support for top-level statements). The language will be strictly single threaded with the GC running on the same thread.

So far, I’ve built a tokenizer, a parser and set of classes to represent the abstract tree - mostly copied from an existing, abandoned project.

Next to do is the semantic analysis phase which is where the bulk of the work will be.

Code generation will be done by generating C code and pushing it through Tiny C Compiler for on-the-fly just-in-time in-memory machine code generation. Fun!

The compiler is being developed in C# and will be hostable directly in Cantabile.

Future

What’s clear from the above is there’s a lot of work to be done to build this - but it’s the kind of project I find really interesting and challenging.

So, I’m going to continue playing with this as a side project (so as to not distract from continued Cantabile development) and if it ends up something useful then good, if not then I’m sure I’ll have learned something along the way.

3 Likes
26

Sounds link good side project material, but could yield something very powerful.

I have been reworking some Cantabile songs over the holiday whilst building my new show (to include lights and video) and as ever I remain impressed at the power we already have in Cantabile, especially the new binding expressions, triggering changes on transport position (I now have a “dead state” at the start of songs, which means I can get pedals in the right position without them affecting anything) and so on.

2 Likes
27

I agree with Dave Dore… machine code is fast and needs no overhead routines creating garbage.
The scripting would be easier for us “dabble in code” guys and I trust your “user friendliness” to make it easy for others.

I think it would help me take deeper control of my Synths as well as MIDI.
I’m getting into Python due to the MIDI code libraries & Audio objects.

Pep

1 Like
28

A silly thought: since Cantabile is written in Microsoft .NET, why not VBscript? It’s very easy for the user to understand, but I don’t know if it fits Brad’s side.

2 Likes
29

@cpaolo

That was some 40 years ago, and I doubt my current energy level could take on another scripting education. But, one never knows! :thinking:

2 Likes
30

I used to quite like VBScript when it was an easy option for talking COM components. These days it’s very out of date. Also, FTR, it’s not .NET based - it’s COM which .NET interops with, but a very different beast. I wouldn’t trust it running on the audio thread.

2 Likes
31

Coming in late to this discussion …

I would like to second the @TorstenH ‘MIDI processor’ idea. Cantabile has a good collection of abilities to trigger and alter MIDI events, but there are things that are difficult to implement unless one resorts to a complicated mix of Bindings, States, and Filters. Rather than continuing to add to the Filter/Binding choices, a MIDI event processor that could be programmed in a JS/C++ manner would allow specific functions to be added only when needed.

One thing that I think would be useful is the ability to define global variables that continue to exist as Songs are loaded and unloaded. There are some neat MIDI tricks that can fix dodgy HW problems, but they require previous knowledge of the state of a CC or NRPN to work:

  • Preventing volume, effect, etc. jumps when changing presets on a synth.
  • Sending a limited range of data values based on a larger range. (ex: send only 0, 63, and 127 from an expression pedal.)
2 Likes
32

Some thoughts after having followed the discussion for some time:

Two scenarios for scripting, with different requirements for a scripting language:

  1. real-time processing within the audio cycle - mostly MIDI processing. For that, we would need a very efficient language, ideally without any time-consuming garbage collect requirements. Explicit memory allocation, simple data types only, simple control structures, limited and explicitly declared access to Cantabile variables, etc. And ideally pre-compiled for speed

  2. General, more complex automation tasks (“super-bindings”) to react to MIDI and other Cantabile Events (e.g. state changes). These wouldn’t need to run within the audio thread - more like batch scripts to automate the UI. For these tasks, a different language might be better suited - full on-demand access to Cantabile variables, objects and binding points, more complex data and control structures, etc…

Maybe two different languages?

Cheers

Torsten

3 Likes
33

Coming in late to the discussion here. One thought I had regarding the difficulties and complications of a language with a garbage collector running on the audio engine, would be to consider that if you’re creating your own internal language, you can place your own constraints on it, to suit the application. One possible approach would be to require all memory/variables etc. to be declared up-front in the script, before any actual execution. It’s an approach I’ve seen in some embedded development, where there’s no OS or runtime, and all memory is mapped out in advance at the start by the compiler, onto on-CPU RAM/registers.

I know it might feel cumbersome to those coming from a memory-managed scripting language background, but it would mean scripts could run super-fast with no memory management problems, which is exactly what you’d want for anything running on the audio thread. It may also be not such a big disadvantage for the kinds of things people might want to run in the audio engine, which I’m guessing would be mostly small, tight, relatively simple processing units.

An approach to consider, at least.

2 Likes
34

Thanks all the feedback guys.

Crazy idea (maybe not): what about C as the scripting language for stuff on the real-time thread?

I’ve been playing around with tcc (tiny c compiler) and its ability to compile C code directly to memory and run it is very cool. Some implicitly included header files for MIDI data packets etc… and the audio engine could call this reasonably easily.

Yes, or rather than some weird syntax that requires declaring things up front, link the user’s C code with a runtime that explodes if you do anything bad on the audio thread (like allocating memory, reading a file etc…)

The more I think about it, the more I like the idea of using C

Brad

4 Likes
35

Not crazy at all. Standard C is a good and easy language, and if you say tcc is very cool, C is welcome. image :rofl: :rofl:

2 Likes
36

Great idea. As long as we get a couple of useful automatic #includes and an easy structure to read the incoming midi data buffer, vst parameter automation (sample accurate) and put stuff into the output buffer.

A midi processor should pretty much work like a vst MIDI plugin; as such it will probably need a very simple structure

  • a definition section, defining used variables and buffers, as well as automation parameters exposed. Maybe also defining input and output ports (if you want to have multiple)? It also should define the external (Cantabile) variables used by the script so the environment can provide a “current cache” for these variables
  • an initialization section whenever the script is instantiated to set useful base values for the variables
  • a “queue processing” section that gets called for every MIDI data buffer and that has access to time-stamped MIDI and automation data for the current batch

Unlike ReaJS, which has a “slider” section, which handles vst parameter changes, I’d prefer parameter changes in the “buffer processing” section, so we have a common (time-position-ordered) input queue of MIDI events and automation events. That would make it very easy to work with sample-accurate automation - walk through the input queue item by item and process them in order. So every MIDI event is processed with the most current VST parameter values.

Just thinking out loud, based on my experience building ReaJS scripts…

3 Likes
37

C is fine by me. :slight_smile: it was always intended as a lightweight language that could be quite close to the architecture.

Anyone for assembler? :wink:

1 Like
43

Are we still talking about a scripting language, or are you envisioning a plugin API for user-compiled binary extensions to Cantabile (e.g., DLLs)?

The latter seems to me like the ideal solution, since it would allow C programmers to write super-fast real-time code when needed, while also allowing you (or anyone) to write a plugin that implements a safer, more user-friendly scripting language (e.g., a plugin that interprets or JIT-compiles user-supplied scripts and calls the underlying C API to effect script actions). That would give you the freedom to develop scripting capabilities as an optional plugin without adding that code complexity directly to Cantabile’s core.

1 Like
44

@brad I could definitely live with C as the language. My memory needs are pretty small (20 or less variables), and I’m not looking to print out messages on the UI or anything. I could live with the suggestion of defining all variables at the top of the script; this would both speed execution and eliminate the need for complex memory cleanup.

Here are three things that I found difficult or impossible to program in Cantabile via MIDI:

  1. Thinning data – i.e., sending only the nth event of some type
  2. Creating last-note priority for mono leads
  3. Storing the current value of a pedal or knob
  4. Mapping all notes of any velocity to the same note on a different channel.

I know the 4th one can be done with the current generation of Filters, but it takes 3-4 of them, as well as both a little imagination and careful ordering. It could probably be done with 1-2 lines of C code.

Regards
-BW

45

I like the three things adding up to 4 :joy:

All of this is pretty easy using ReaJS - and I’d expect it to be even easier in a future MIDI processor

1 Like
46

Everything starts at “Hello World”…

4 Likes