Without promising anything, I’ve been thinking about how to integrate video playback into Cantabile but realized I don’t actually know how you guys would want to use it, so… I’m asking
The main thing I’m curious about is where the audio fits into this - would you want to stream audio from the video through Cantabile, or would you mostly be using the video for display effects, and the audio isn’t really needed at all?
I’m thinking at a minimum you’d probably want:
Ability for a song to reference a video file (multiple?) and have it displayed somwhere
Ability for the song to control the playback (start/stop/pause) and playback position (seek).
Some sort of sync between Cantabile’s transport position and the video position, but I’m really not sure how that would work (changing the playback speed of video to keep it in sync sounds horribly complicated and not really something I want to get into unless it’s vitally important).
What else? What’s the minimally useful functionality you’d like to see here? What would be the typical use case?
Trigger a video when the transport starts, stop it when transport stops (e.g. in my Pink Floyd band when we were playing back pre-recorded videos - most are available on the PULSE DVD!)
Use of a video generator like Imaginando Visual Synthesizer that is reacting to my playing
So far I have kept video duties on a separate laptop as I do not want video duties impacting audio duties, and for example, in use case 2 I am really taxing the GPU - to the point where I think the laptop I purchased is not quite powerful enough for the job, and I would be reluctant to run it on my GIG PC which is 7 years old and in need of upgrade this year
I would say use case 1 is potentially useful in Cantabile if performance is good and you can direct the video to a different output (I would still want to see Cantabile’s UI on stage). And having a video start right on the mark in Cantabile would be neat as I have never achieved perfect sync on start with the video package I was using that supported remote start over MIDI.
Use Case 2 is a bit more niche, and if I am running a lot of VSTs and video I think it is best to keep them seperate - sync is fine via RTP MIDI
Yes, scenario 2 definitely seems outside the scope of Cantabile - especially if it’s generating video.
Scenario 1 is more the area I was thinking about - in that case you’re just playing video and no audio from the video file?
Here’s what I was thinking:
The basic idea would be to use a web browser for the video playback. This has a number of advantages like already supporting a good range of codecs, streaming the video over network so it can be played on another machine, full screen mode, etc… You could have multiple machines all streaming/displaying the same video.
The video itself could be streamed from a simple server either on a dedicated machine or the same machine as Cantabile. I could include video streaming in Cantabile’s network server but I’d be a little concerned about impact on load and it would be nice to have the ability to offload that to a second machine if necessary.
The web browser would have a web-socket connection to Cantabile letting Cantabile control the playback.
I think this could be pretty tightly integrated with Cantabile’s existing media player functionality - instead of opening an audio/midi file, you connect to the server hosting the video files (in simplest case that’d be localhost) and choose from an available video file. You then open a browser and point it at something like http://localhost:35007/video and you get a plain window with just the video in it, but with playback controlled by the media player in Cantabile.
Obviously, there’s a few moving parts to make all this work and I’d need to automate it for basic usage, but keep the ability to pull it apart and reconfigure it for more advanced setups.
Agree that Use Case 2 is quite different, and well covered by the Imaginando product either standalone on a separate PC or you can run it as a VST and it worked in Cantabile when I experimented with it. It supports SPOUT and NDI protocols.
So Use Case 1 is the focus. It certainly sounds useful to me - stupid Q as i have never tried it, assume you can get a browser streamed video full screen with no “window decorations”?
It may be worth you looking at what Show Cue Systems SCS11 does in terms of general features as that is what I was using synced to Cantabile via MIDI over RTP. It did the job, but as clunky as hell, so if the Floyd Band gets going again (a possibility) then I would sure like to use Cantabile for that instead of purchasing a new SCS11 license!
Happy to beta test.
Another idea that would be useful as well as Video would be to support static pictures, which are always useful for band advertising/promotion before/after the show. E.g. My default song in my Spectral Streams setlist is not a song per se, but the show notes are a checklist of live versus studio settings to check(!) and it triggers Visual Synthesizer to select a preset with no video layers but a static background image.which has my promotional details.
PS, I also need the ability to start videos in a free form manner (e.g. from a key press or MIDI CC. Shine on You Crazy Diamond is an example of a song we start free form and the video needs to start when the intro guitar solo starts. The click and everything else starts later in the song where the whole band comes in.
No worries. I know I can meet my needs by other software external to Cantabile, but it would be good to see video integrated.
I also know you need to prioritise on where the main user interest is, but I am perhaps a little surprised that I am the only responder to your query?
Surely (but don’t call me Shirley), I cannot be the only Cantabile user interested in the visual aspects of the show as well as the sounds? Maybe the differentiator might be one of scale? In bands where I have done both, we are not large in terms of reach and scale (i.e. we are not famous, with caboodles of money and resources to employ lighting and video engineers). So I am the keyboard player, light man and video man; why does it always seem to fall to the keyboard player?
What I’d be looking for is the following for display on a projection TV -
Ability to start/stop a video file, with one or more video files associated with a song (just as I might trigger an audio file in media player)
Ability to display a band banner
Ability to switch to one (or more) live cameras
Support for the Imaginando product or similar. This is of far less importance as we use that in the studio, make a recording, and then use that recording as a video.
Ability to display a black screen when video isn’t playing (I tried a VST that displays video, but it shows the opening frame until the transport starts rather than displaying a black screen. A workaround is to edit each video to add a black first frame, but that a lot of extra effort)
Does anyone have an interim solution that works? I’ve thought to use vlc since it supports cameras and incoming audio streams, but I don’t know if it would do both at the same time and if I issue multiple commands from Cantabile it just opens up a new instance of vlc for each command, which is not what I want.
No promises, but I’m thinking about knocking together a simple content/control server for video playback that would be controllable via Cantabile.
I understand most of the above requirements except for live camera feed display. What are we talking about here - are these cameras attached directly to a PC, or IP/Wireless cameras. Any further detail in this area would be useful.
I have the same user case #1 from Derek: in one of my groups we play music themes from old TV series, and videos are projected live behind the band , so a basic video start/stop/loop with MIDI CC would be more than enough (audio is not needed for me)
Hi @brad in media world, we tend to use MTC (MIDI Timecode) for video sync. It would be wonderful if Cantabile could support both sending and receiving MTC, in addition to MIDI clock.
Here are a couple of software for video sync, both paid and free, that might serve as inspiration:
