Experimental Video Playback Solution (cantabile-media-server)

Hi, @brad

Given this is not meant to a program that quits, unless you press CTRL+C or close the command prompt, I felt the options like --version closing the program did not feel right (compared to command line software that is usually short lived), and what you had before you introduced the options was about right in terms of info reporting and logging.

Having said that. there is really no right or wrong way for this, and as I said above, maybe once you have a stable major release, there is no need for the info to be shown all the time, so your instincts and what you have done to control the logging via options is better long term.

Main thing for me is the main functionality that is maturing nicely.

Hey Derek,

Next version will show version number on normal start:

and running with --version displays version number and quits:

Haven’t finialized/cleaned up other logging yet.

1 Like

Today I started adding support for “layers”

In the following config you can see channel 0 (aka channel 1) is configured to have three layers:

  1. background layer - a static image (static1.jpg)
  2. media layer - a dynamic layer that shows the media selected by the channels program number
  3. camera layer - which is hidden by default

The web browser places all three layers directly over each other. Tomorrow’s job is to add MIDI CC controls to allow hiding and showing individual layers.

A setup like the above lets you:

  • When camera layer is hidden and video player is stopped a static image is shown. (eg: the band logo)
  • When media layer is an image or a playing video it replaces the static background image.
  • When camera layer is shown, the middle video player layer continues to play in-sync in background.

ie: this setup would let you show a static image or a video and then switch seamlessly between it and a camera feed.

Still working on all this and open to feedback.

Brad

2 Likes

Perfect!! The only thing I’d add is that the camera layer should support multiple cameras (e.g. one facing the band or different band members at different times, one facing the audience), but perhaps that’s handled in the camera server. In any event, what I’m hoping to do is map some unused pads on my keyboard to each camera, so I can turn a camera on/off from the keyboard or via a binding to a transport position.

Hi @Ivan,

There’s a couple of ways to handle multiple cameras…

  1. As you suggest, one way would be to control it at the feed source end - eg: using OBS or similar software to switch between different cameras. I think that would probably work best if you only ever need to show one camera feed at a time (ie: you don’t need different video displays showing different camera views).

  2. The other way would be to just create additional layers, one for each camera and then hiding/showing the appropriate layers. The above example shows three layers, but you can create more of you need them.

  3. Finally, I’m thinking of providing a way to have switchable media on multiple layers on the same channel. This requires an alternative mechanism for selecting the media since currently it’s using the MIDI channel’s program number and this would need multiple “programs” per channel. I have a couple of ideas here.

To be honest I’m not totally convinced about the camera support in this kind of setup because I’m not sure if the latency is tolerable and the delay in OBS->mediamtc feeds at startup might make switching between feeds janky. go2rtc is better in this regard but has other incompatibilities.

That said, the camera feed side of things is pretty much done. Once I get this next version out I’ll be keen for feedback.

2 Likes

I’ve just put up version 0.0.6 with a whole bunch of new stuff:

  • ‘MTC’ and ‘master’ time sync. support
  • Drift and latency compensation
  • Multi-layer support
  • CC’s 80 - 89 to control layer visibility
  • CC’s 70 - 73 for selecting alternate program slot numbers
  • Ability to auto-hide video layers when stopped
  • Various other tweaks and improvements

All of the above is better explained in the docs.

Existing configurations should run without change - the only real difference being that ‘master’ time sync mode is the default whereas previous versions behaved in ‘none’ mode.

As always, interested in feedback.

3 Likes

Might be a little while before I get to try it, but I will…

I currently have my Yamaha SY99 in pieces - replacing the 70 front panel switches which are now very sticky. That’s four solder connections per switch, and of course to get to the buttons it’s a complete strip down from the bottom in

1 Like

FYI: 0.0.7 available now fixes some issues.

2 Likes

I’ve been loosely following this thread and have a question (I’m not using video during shows right now, but…). Is this “Cantabile-Media-Server” part of the solution for Multi monitor support in Trello?

image

I’m looking to use multiple notes screens for different band members. These notes could be static jpegs for each song, as long as each jpeg can be shown on a different monitor. I could use the onscreen keyboard functions to display variable, too, but not sure how involved all that would be.

I didn’t have this in mind when I started cantabile-media-server, but during the development it became apparent it could work as a solution for this - partly because the media server works so much better than I expected.

In fact, I’ve already prototyped using the media server as a way to view and control pdf files (with mixed success, but believe I have a way to make it work).

Back in June I posted some thoughts about completely reworking show notes - see here. Originally I was planning on implementing this directly in Cantabile, but I’m now seriously leaning towards moving this into the media server - it gives lots of advantages, not least of which is multi-monitor show notes, remote show notes, different show notes per band member, pdf support and more…

If you just want to show static image show notes, the current media server will work for this. For MIDI controlled scrollable content, that’s still to come (hopefully).

3 Likes

I’ll review that thread and give it a try. First I need to procure some hardware. Thanks.

Using media server would be a nice touch - my only major concern is using up additional CPU cycles on the cantabile machine. I actually almost never display text on the notes page - rather I display notation. I tell my notation editor to format a .pdf for 11" wide x 5" high (which is pretty close) to my screen size, I then convert the .pdf to .jpg’s, and then use states to move from one .jpg to the next. Some lengthy songs can take up to 8 .jpgs. What I don’t understand is how I would create a .pdf long enough to capture the entire song, let alone how Cantabile would know how to move through it (unlike lyrics where you could scroll at a pre-determined rate, repeats in the notation can make autoscroll detrimental, especially where a repeat goes back to the beginning of a song).

Some of the uses I can immediately see are 1) a feed of the song name to the drummer’s tablet so he starts the right song; 2) a feed of scrollable lyrics to the singer.

In my case I get nice multi-monitor support now by having my “development mode” screen on the 1st monitor and my “live mode” screen (which displays notes plus customized controls) on a 2nd monitor and and shifting between the two with a hot key. Availability of those controls is important and wouldn’t appear if the 2nd monitor just displayed broadcasted notes.

Well there’s nothing to try yet… that linked thread was a discussion about possibilities and nothing yet implemented.

I’m not too concerned about this because you can run the media server on a separate machine and/or you could run a media server for video on another machine and one for notes on the local machine. Showing notes via media server is a pretty low load task.

There’s no reason the media server couldn’t do this. Just like how Cantabile can start/stop a video player in a web browser, it could also instruct the browser to show a particular PDF page or jump to a particular show notes position. Whether that’s controlled by states, CCs in a media player, bindings, or something more tightly integrated with Cantabile is yet to be decided.

In fact, converting pdf pages to images is probably what the media server would do internally to show the pdf (that’s because even though most browsers can display pdfs natively there’s no scripting control to allow scrolling etc…)

Yes, this is something that’s currently not possible with current show notes.

Not sure I understand your point here. Are you saying having show notes external to Cantabile’s main window would be a problem? If so one solution might be to integrate a web viewer into Cantabile for the show notes, but that would probably come later.

I guess my main point here is that using a web browser for the front-end display of notes opens up some interesting possibilities - and rather than implement the new show notes twice (once built-in to Cantabile and again for remote viewing in a brower) I’d prefer just do it once.

Regarding > Ivan:

Some of the uses I can immediately see are 1) a feed of the song name to the drummer’s tablet so he starts the right song; 2) a feed of scrollable lyrics to the singer.

Yes, this is something that’s currently not possible with current show notes.

I have a draft of that implemented with cantabile-media-server (not with show notes). What I did was create a programlist.txt with the media files we use, then add 100 to each for a 2nd list meant for the singer, and add another 100 for a 3rd list meant for the drummer. The web page doesn’t have scroll capability, but this works for proof of concept. If anyone is interested in details, I’m following this with my config.json and my programlist.txt (I only implemented two of the songs on all three channels as proof of concept):

config.json:
{
    "baseDir": ".",
    "midiPort": "loopMIDI Port",
    "port": 3000,
    "programList": "programList.txt",
    "syncMode": "master",
    "channels": {
            "1": {
                "layers": [
                    { 
                        "mediaFile": "band_logo.jpg",
                    },
                    { 
                        "useProgramList": true,
                        "hiddenWhenStopped": true,
                    },
                    { 
                        "mediaFile": "webrtc+http://localhost:8889/mystream/whep",
                        "display": "hidden"
                    }
                ]
            },
           "2": {
                "layers": [
                    { 
                        "mediaFile": "band_logo.jpg",
                    },
                    { 
                        "useProgramList": true,
                        "programNumberOffset": 100
                       }
                ]
            },
           "3": {
                "layers": [
                    { 
                        "mediaFile": "band_logo.jpg",
                    },
                    { 
                        "useProgramList": true,
                        "programNumberOffset": 200
                       }
                ]
            }
        }
}

programlist.txt:

# specifies the default for any programs not explicitly stated
default: Blackimage.jpg

# specifies the base program number (0 or 1)
base: 1

# program number to media file mapping
0: Blackimage.jpg
36: Alone.mp4
37: Angels.mp4
38: Bring Me To Life.mp4
39: Destination Departure.mp4
40: Discord.mp4
41: Enjoy The Silence.mp4
42: Ever Dream.mp4
43: Farewell Proud Men.mp4
44: Going Under.mp4
45: Heaven's a Lie.mp4
46: Mother Earth.mp4
47: Nemo.mp4
48: Phantom of the Opera.mp4
49: Poison.mp4
50: Sancta Terra.mp4
51: Sleepwalker's Dream.mp4
52: The Promise.mp4
53: The Howling.mp4
54: Walking in the Air.mp4
99: Outro.mp4
145: Heaven's a Lie lyrics.jpg
153: The Howling lyrics.jpg
245: Next song is Heaven's a Lie.jpg
253: Next song is The Howling.jpg

What I was saying was that when I perform in Live Mode I’m displaying notes, the menu bar, and the controller bar. I’d be totally happy with the notes part replaced by something with more features (scrollability, support for .pdf’s, etc) but not if takes up the full screen and I lose the menu and controller bars - I really need those for live performance control.

Gotcha… so that will probably come later and will require incorporating webview into Cantabile. In other words, what I’m looking at now is getting the tech stack right as in web browser vs native rendering in Cantabile (and leaning towards web browser). Then later smoothing things out to make them more use friendly.

@Ivan fyi when pasting config/json files here, if you place the content between triple backticks, the content will appear as plain text instead of treating # lines as headers etc…

eg: if you enter this:

```
# specifies the default for any programs not explicitly stated
default: Blackimage.jpg

# specifies the base program number (0 or 1)
base: 1

```

it will appear like this

# specifies the default for any programs not explicitly stated
default: Blackimage.jpg

# specifies the base program number (0 or 1)
base: 1

instead of like this:

specifies the default for any programs not explicitly stated

default: Blackimage.jpg

specifies the base program number (0 or 1)

base: 1

A post was merged into an existing topic: Experimental MIDI Controllable PDF Viewer

Feature modification -
Hi Brad - I’m using three video window channels simultaneously, sending the same program number to each (each channel is meant for a different player). I was hoping you might add an option (programMode?) so that a program change to channel 1 gets sent to all channels.

Rather than an option to send program changes to all channels, I think a better idea (and I nearly implemented it but couldn’t think of a use case till you mentioned it) is to allow layers to use the program number of a different channel. This would allow configurations where web view channels 1,2,3 all use the selected program of MIDI channel 1 for example.

This shouldn’t be hard to add… I’ll look into it.

1 Like

Version 0.0.9 now available with PDF support.