RFC: Cantabile and Video Playback... what's needed?

Hey All,

Without promising anything, I’ve been thinking about how to integrate video playback into Cantabile but realized I don’t actually know how you guys would want to use it, so… I’m asking :slight_smile:

The main thing I’m curious about is where the audio fits into this - would you want to stream audio from the video through Cantabile, or would you mostly be using the video for display effects, and the audio isn’t really needed at all?

I’m thinking at a minimum you’d probably want:

  1. Ability for a song to reference a video file (multiple?) and have it displayed somwhere
  2. Ability for the song to control the playback (start/stop/pause) and playback position (seek).
  3. Some sort of sync between Cantabile’s transport position and the video position, but I’m really not sure how that would work (changing the playback speed of video to keep it in sync sounds horribly complicated and not really something I want to get into unless it’s vitally important).

What else? What’s the minimally useful functionality you’d like to see here? What would be the typical use case?

Brad

Hi

I have two use cases

  1. Trigger a video when the transport starts, stop it when transport stops (e.g. in my Pink Floyd band when we were playing back pre-recorded videos - most are available on the PULSE DVD!)
  2. Use of a video generator like Imaginando Visual Synthesizer that is reacting to my playing

So far I have kept video duties on a separate laptop as I do not want video duties impacting audio duties, and for example, in use case 2 I am really taxing the GPU - to the point where I think the laptop I purchased is not quite powerful enough for the job, and I would be reluctant to run it on my GIG PC which is 7 years old and in need of upgrade this year

I would say use case 1 is potentially useful in Cantabile if performance is good and you can direct the video to a different output (I would still want to see Cantabile’s UI on stage). And having a video start right on the mark in Cantabile would be neat as I have never achieved perfect sync on start with the video package I was using that supported remote start over MIDI.

Use Case 2 is a bit more niche, and if I am running a lot of VSTs and video I think it is best to keep them seperate - sync is fine via RTP MIDI

1 Like

Hi @Derek

Yes, scenario 2 definitely seems outside the scope of Cantabile - especially if it’s generating video.

Scenario 1 is more the area I was thinking about - in that case you’re just playing video and no audio from the video file?

Here’s what I was thinking:

  • The basic idea would be to use a web browser for the video playback. This has a number of advantages like already supporting a good range of codecs, streaming the video over network so it can be played on another machine, full screen mode, etc… You could have multiple machines all streaming/displaying the same video.

  • The video itself could be streamed from a simple server either on a dedicated machine or the same machine as Cantabile. I could include video streaming in Cantabile’s network server but I’d be a little concerned about impact on load and it would be nice to have the ability to offload that to a second machine if necessary.

  • The web browser would have a web-socket connection to Cantabile letting Cantabile control the playback.

I think this could be pretty tightly integrated with Cantabile’s existing media player functionality - instead of opening an audio/midi file, you connect to the server hosting the video files (in simplest case that’d be localhost) and choose from an available video file. You then open a browser and point it at something like http://localhost:35007/video and you get a plain window with just the video in it, but with playback controlled by the media player in Cantabile.

Obviously, there’s a few moving parts to make all this work and I’d need to automate it for basic usage, but keep the ability to pull it apart and reconfigure it for more advanced setups.

Does this sound like it might be useful?

1 Like

Hi, @brad

Agree that Use Case 2 is quite different, and well covered by the Imaginando product either standalone on a separate PC or you can run it as a VST and it worked in Cantabile when I experimented with it. It supports SPOUT and NDI protocols.

So Use Case 1 is the focus. It certainly sounds useful to me - stupid Q as i have never tried it, assume you can get a browser streamed video full screen with no “window decorations”?

It may be worth you looking at what Show Cue Systems SCS11 does in terms of general features as that is what I was using synced to Cantabile via MIDI over RTP. It did the job, but as clunky as hell, so if the Floyd Band gets going again (a possibility) then I would sure like to use Cantabile for that instead of purchasing a new SCS11 license!

Happy to beta test.

Another idea that would be useful as well as Video would be to support static pictures, which are always useful for band advertising/promotion before/after the show. E.g. My default song in my Spectral Streams setlist is not a song per se, but the show notes are a checklist of live versus studio settings to check(!) and it triggers Visual Synthesizer to select a preset with no video layers but a static background image.which has my promotional details.

1 Like

PS, I also need the ability to start videos in a free form manner (e.g. from a key press or MIDI CC. Shine on You Crazy Diamond is an example of a song we start free form and the video needs to start when the intro guitar solo starts. The click and everything else starts later in the song where the whole band comes in.

1 Like

Thanks Derek…

As mentioned, no promises here, and mostly I’m just gathering ideas and thoughts at this point and your feedback is much appreciated.

Brad

1 Like

No worries. I know I can meet my needs by other software external to Cantabile, but it would be good to see video integrated.

I also know you need to prioritise on where the main user interest is, but I am perhaps a little surprised that I am the only responder to your query?

Surely (but don’t call me Shirley), I cannot be the only Cantabile user interested in the visual aspects of the show as well as the sounds? Maybe the differentiator might be one of scale? In bands where I have done both, we are not large in terms of reach and scale (i.e. we are not famous, with caboodles of money and resources to employ lighting and video engineers). So I am the keyboard player, light man and video man; why does it always seem to fall to the keyboard player? :slight_smile:

Because he is the mad scientist in the band! :laughing:
Or the wizard (at least when he wears a glittering cape).

Gabriel

Sorry for wasting bandwdith with silly jokes…well, not really… :wink:

3 Likes

More like we are the nerds/geeks with the technical know how!

4 Likes

I know how you feel brother. The “technical know how” upsets the ignorant ones.

OT Corky :crazy_face:

1 Like

A post was merged into an existing topic: Thinking about show notes