Without promising anything, I’ve been thinking about how to integrate video playback into Cantabile but realized I don’t actually know how you guys would want to use it, so… I’m asking
The main thing I’m curious about is where the audio fits into this - would you want to stream audio from the video through Cantabile, or would you mostly be using the video for display effects, and the audio isn’t really needed at all?
I’m thinking at a minimum you’d probably want:
Ability for a song to reference a video file (multiple?) and have it displayed somwhere
Ability for the song to control the playback (start/stop/pause) and playback position (seek).
Some sort of sync between Cantabile’s transport position and the video position, but I’m really not sure how that would work (changing the playback speed of video to keep it in sync sounds horribly complicated and not really something I want to get into unless it’s vitally important).
What else? What’s the minimally useful functionality you’d like to see here? What would be the typical use case?
Trigger a video when the transport starts, stop it when transport stops (e.g. in my Pink Floyd band when we were playing back pre-recorded videos - most are available on the PULSE DVD!)
Use of a video generator like Imaginando Visual Synthesizer that is reacting to my playing
So far I have kept video duties on a separate laptop as I do not want video duties impacting audio duties, and for example, in use case 2 I am really taxing the GPU - to the point where I think the laptop I purchased is not quite powerful enough for the job, and I would be reluctant to run it on my GIG PC which is 7 years old and in need of upgrade this year
I would say use case 1 is potentially useful in Cantabile if performance is good and you can direct the video to a different output (I would still want to see Cantabile’s UI on stage). And having a video start right on the mark in Cantabile would be neat as I have never achieved perfect sync on start with the video package I was using that supported remote start over MIDI.
Use Case 2 is a bit more niche, and if I am running a lot of VSTs and video I think it is best to keep them seperate - sync is fine via RTP MIDI
Yes, scenario 2 definitely seems outside the scope of Cantabile - especially if it’s generating video.
Scenario 1 is more the area I was thinking about - in that case you’re just playing video and no audio from the video file?
Here’s what I was thinking:
The basic idea would be to use a web browser for the video playback. This has a number of advantages like already supporting a good range of codecs, streaming the video over network so it can be played on another machine, full screen mode, etc… You could have multiple machines all streaming/displaying the same video.
The video itself could be streamed from a simple server either on a dedicated machine or the same machine as Cantabile. I could include video streaming in Cantabile’s network server but I’d be a little concerned about impact on load and it would be nice to have the ability to offload that to a second machine if necessary.
The web browser would have a web-socket connection to Cantabile letting Cantabile control the playback.
I think this could be pretty tightly integrated with Cantabile’s existing media player functionality - instead of opening an audio/midi file, you connect to the server hosting the video files (in simplest case that’d be localhost) and choose from an available video file. You then open a browser and point it at something like http://localhost:35007/video and you get a plain window with just the video in it, but with playback controlled by the media player in Cantabile.
Obviously, there’s a few moving parts to make all this work and I’d need to automate it for basic usage, but keep the ability to pull it apart and reconfigure it for more advanced setups.
Agree that Use Case 2 is quite different, and well covered by the Imaginando product either standalone on a separate PC or you can run it as a VST and it worked in Cantabile when I experimented with it. It supports SPOUT and NDI protocols.
So Use Case 1 is the focus. It certainly sounds useful to me - stupid Q as i have never tried it, assume you can get a browser streamed video full screen with no “window decorations”?
It may be worth you looking at what Show Cue Systems SCS11 does in terms of general features as that is what I was using synced to Cantabile via MIDI over RTP. It did the job, but as clunky as hell, so if the Floyd Band gets going again (a possibility) then I would sure like to use Cantabile for that instead of purchasing a new SCS11 license!
Happy to beta test.
Another idea that would be useful as well as Video would be to support static pictures, which are always useful for band advertising/promotion before/after the show. E.g. My default song in my Spectral Streams setlist is not a song per se, but the show notes are a checklist of live versus studio settings to check(!) and it triggers Visual Synthesizer to select a preset with no video layers but a static background image.which has my promotional details.
PS, I also need the ability to start videos in a free form manner (e.g. from a key press or MIDI CC. Shine on You Crazy Diamond is an example of a song we start free form and the video needs to start when the intro guitar solo starts. The click and everything else starts later in the song where the whole band comes in.
No worries. I know I can meet my needs by other software external to Cantabile, but it would be good to see video integrated.
I also know you need to prioritise on where the main user interest is, but I am perhaps a little surprised that I am the only responder to your query?
Surely (but don’t call me Shirley), I cannot be the only Cantabile user interested in the visual aspects of the show as well as the sounds? Maybe the differentiator might be one of scale? In bands where I have done both, we are not large in terms of reach and scale (i.e. we are not famous, with caboodles of money and resources to employ lighting and video engineers). So I am the keyboard player, light man and video man; why does it always seem to fall to the keyboard player?
What I’d be looking for is the following for display on a projection TV -
Ability to start/stop a video file, with one or more video files associated with a song (just as I might trigger an audio file in media player)
Ability to display a band banner
Ability to switch to one (or more) live cameras
Support for the Imaginando product or similar. This is of far less importance as we use that in the studio, make a recording, and then use that recording as a video.
Ability to display a black screen when video isn’t playing (I tried a VST that displays video, but it shows the opening frame until the transport starts rather than displaying a black screen. A workaround is to edit each video to add a black first frame, but that a lot of extra effort)
Does anyone have an interim solution that works? I’ve thought to use vlc since it supports cameras and incoming audio streams, but I don’t know if it would do both at the same time and if I issue multiple commands from Cantabile it just opens up a new instance of vlc for each command, which is not what I want.
No promises, but I’m thinking about knocking together a simple content/control server for video playback that would be controllable via Cantabile.
I understand most of the above requirements except for live camera feed display. What are we talking about here - are these cameras attached directly to a PC, or IP/Wireless cameras. Any further detail in this area would be useful.
I have the same user case #1 from Derek: in one of my groups we play music themes from old TV series, and videos are projected live behind the band , so a basic video start/stop/loop with MIDI CC would be more than enough (audio is not needed for me)
Hi @brad in media world, we tend to use MTC (MIDI Timecode) for video sync. It would be wonderful if Cantabile could support both sending and receiving MTC, in addition to MIDI clock.
Here are a couple of software for video sync, both paid and free, that might serve as inspiration:
Thanks for the suggestion re MTC. I’ve looked at this before but can’t remember off hand why I never implemented it. I’ll take another look when I get a chance.
I’m thinking of experimenting with a simple web based, MIDI controllable video solution. It’d be completely separate from, but controllable from Cantabile:
Control Server
A server that you point web browser to. It tells the browser which video stream to play and can instruct the browser to play/pause etc…
Has a MIDI interface that Cantabile can use to tell it which video to load, on which channel and then commands to play/ etc…
MIDI commands get forwarded to connected web clients to control what’s shown.
Support for 16 MIDI channels each capable of playing a different video stream.
It would have config file that maps MIDI program numbers to video/image files.
Content Server
A server that can stream video and/or static image content.
This is where the web browser will get the actual video/image media from
Can be the same machine/server as the control server.
Typical Use
Create a config file that maps MIDI program numbers to media files (either video files, static image files or other URL types). The media files could be on the same machine as the control server, or somewhere else (on a separate content server, you could even stream from web if you wanted)
Start the control server (probably a command line program)
Point a web browser (on any machine) to the control server and choose which MIDI channel it should control it (1-16). Put the browser in full screen mode. (repeat for as many displays/channels as you need)
Setup a MIDI connection between Cantabile machine and the control server (either physical midi connection, or virtual MIDI port, or rtpMIDI over network etc…)
Use bindings in Cantabile to send MIDI messages to load/play particular media file on a particular channel as and when required.
Any browsers configured to stream a particular channel will automatically stream/start/stop etc… according to MIDI commands sent from Cantabile (or any other controller).
Notes
Cantabile, the control server, the content server and web browsers can all be on the same or different machines. (though I’d recommend keeping the content server and web browsers separate from Cantabile).
You could have multiple control and content servers
Multiple browsers can all connect to the same channel - so you can essentially have unlimited views of 16 different channels.
Can be controlled by Cantabile or anything capable of sending MIDI events.
Not sure about video sync - it might be possible by having the web clients adjust playback speed on the fly to keep in sync, but that’s probably something for later. If you just want to play a video file while you perform and precise sync isn’t important then I think this solution should work fine.
Not sure about video camera support. It might work out of the box for cameras that can viewed directly from a web browser if such a thing exists (just map a MIDI program number to the URL of the camera). Aside from that it would require a content server that can connect a USB camera and make it available as a http/rtmp feed. I’ve done a little investigation into this - maybe using OBS: All Versions - How to set up your own private RTMP server using nginx | OBS Forums.
Hi,
I think you are con the right track. Most users might want video playback for visual cues or effects rather than audio streaming. The key features would likely be:
Video Display: Linking a video file to a song and displaying it on screen.
Playback Controls: Ability to start, stop and seek through the video.
Sync: Basic synchronization with Cantabile’s transport position, but without complex speed adjustments.
These features would cover common use cases like visual accompaniment for live performances or rehearsals.
