Interesting as that was what I was using SCS11 for, but it is a bit clunky
No promises, but I’m thinking about knocking together a simple content/control server for video playback that would be controllable via Cantabile.
I understand most of the above requirements except for live camera feed display. What are we talking about here - are these cameras attached directly to a PC, or IP/Wireless cameras. Any further detail in this area would be useful.
1 Like
I have the same user case #1 from Derek: in one of my groups we play music themes from old TV series, and videos are projected live behind the band , so a basic video start/stop/loop with MIDI CC would be more than enough (audio is not needed for me)
Luigi
1 Like
Hi @brad in media world, we tend to use MTC (MIDI Timecode) for video sync. It would be wonderful if Cantabile could support both sending and receiving MTC, in addition to MIDI clock.
Here are a couple of software for video sync, both paid and free, that might serve as inspiration:
https://non-lethal-applications.com/video-sync-6-pro
Cheers!
2 Likes
Hi @kurzy,
Thanks for the suggestion re MTC. I’ve looked at this before but can’t remember off hand why I never implemented it. I’ll take another look when I get a chance.
Brad
1 Like
Hey Guys,
I’m thinking of experimenting with a simple web based, MIDI controllable video solution. It’d be completely separate from, but controllable from Cantabile:
Control Server
-
A server that you point web browser to. It tells the browser which video stream to play and can instruct the browser to play/pause etc…
-
Has a MIDI interface that Cantabile can use to tell it which video to load, on which channel and then commands to play/ etc…
-
MIDI commands get forwarded to connected web clients to control what’s shown.
-
Support for 16 MIDI channels each capable of playing a different video stream.
-
It would have config file that maps MIDI program numbers to video/image files.
Content Server
-
A server that can stream video and/or static image content.
-
This is where the web browser will get the actual video/image media from
-
Can be the same machine/server as the control server.
Typical Use
-
Create a config file that maps MIDI program numbers to media files (either video files, static image files or other URL types). The media files could be on the same machine as the control server, or somewhere else (on a separate content server, you could even stream from web if you wanted)
-
Start the control server (probably a command line program)
-
Point a web browser (on any machine) to the control server and choose which MIDI channel it should control it (1-16). Put the browser in full screen mode. (repeat for as many displays/channels as you need)
-
Setup a MIDI connection between Cantabile machine and the control server (either physical midi connection, or virtual MIDI port, or rtpMIDI over network etc…)
-
Use bindings in Cantabile to send MIDI messages to load/play particular media file on a particular channel as and when required.
-
Any browsers configured to stream a particular channel will automatically stream/start/stop etc… according to MIDI commands sent from Cantabile (or any other controller).
Notes
-
Cantabile, the control server, the content server and web browsers can all be on the same or different machines. (though I’d recommend keeping the content server and web browsers separate from Cantabile).
-
You could have multiple control and content servers
-
Multiple browsers can all connect to the same channel - so you can essentially have unlimited views of 16 different channels.
-
Can be controlled by Cantabile or anything capable of sending MIDI events.
-
Not sure about video sync - it might be possible by having the web clients adjust playback speed on the fly to keep in sync, but that’s probably something for later. If you just want to play a video file while you perform and precise sync isn’t important then I think this solution should work fine.
-
Not sure about video camera support. It might work out of the box for cameras that can viewed directly from a web browser if such a thing exists (just map a MIDI program number to the URL of the camera). Aside from that it would require a content server that can connect a USB camera and make it available as a http/rtmp feed. I’ve done a little investigation into this - maybe using OBS: All Versions - How to set up your own private RTMP server using nginx | OBS Forums.
Anyway… is this worth pursuing?
Brad
1 Like
Hi,
I think you are con the right track. Most users might want video playback for visual cues or effects rather than audio streaming. The key features would likely be:
- Video Display: Linking a video file to a song and displaying it on screen.
- Playback Controls: Ability to start, stop and seek through the video.
- Sync: Basic synchronization with Cantabile’s transport position, but without complex speed adjustments.
These features would cover common use cases like visual accompaniment for live performances or rehearsals.
Thanks
You may be interested in the solution used by front of house engineer for Carl Palmer…
Please note, when used with Cantabile, VidPlayVST option ‘SYNC: Use PPQ position’ MUST BE SET
Again on the right track - but some additional suggestions -
- Since a video or image name can be encoded via sysex, I’d suggest using that method rather than a configuration file. E.g. expand “media player” to accept video files (and send the file names to the server). Then existing synchronization methods such as making the media player a slave, selecting which video to play, etc. could be used, with media player communicating appropriate info to the server. This wouldn’t though support 16 channels each having their own stream, but someone else would need to indicate they’d need this. We wouldn’t since we plan to only use one projector at gigs.
Re the config file, I currently am using another solution that requires using a config file and its easy to make mistakes. In “typical use 5” above, you would provide a method for a binding to send an encoded name to the server.
- Video camera support is important for my band. We want to be able to play video in background, then replace parts of the video (or overlay a subwindow) with live video of the band. This could be with either USB connected cameras or with IP connected cameras using DroidCamX or similar.
I looked at this option and didn’t find it to work well with Cantabile. The main issue is that I wanted “Detach Window” and “Mode: Full Screen 2” so I could display the output on my projector, but those have to be set manually - those options don’t seem to be exported in properties. Cantabile does save the song name, so it makes it easy to put a separate vidplayvst plugin into each song with the song name, but without a mechanized way to set the modes correctly this is useless for us. I did try putting it into a linked rack so that the mode settings wouldn’t change, but having the song set the video file name didn’t work. Did you get it to work?
I finally put together a little proof of concept for this.
Cantabile → MIDI → “The Server” → Web Browser
I think ths could work.
3 Likes
Hi, @brad
That would work for me if I can select a different video for each song and start and stop it on transport start/stop and some other trigger when you need to start manually.
If you could also display static images as well on a trigger that would be great as well - for band info pre-gig, post gig and during the interval
That would certainly float my boat for Floyd type video playback to be timed to backing/click start/stop
The latency looked good as well. Using SCS11 I never got a perfect start/stop, there was always some noticeable lag which was variable so working out a timed offset never sorted that.
Yes this will be possible by either A) a program list configured in the server to map MIDI program numbers to media files, or B) by accepting sys-ex events with media file names encoded. I haven’t decided on A and/or B yet.
Yes, this will be supported.
In the demo this was obviously with Cantabile/server/browser all running on same machine. In real world you might have different machines for each bit in which case the latency would depend on the MIDI link to the server and the WebSocket connection from server to browser.
Not sure yet whether I want to support full timing sync on this, or just ability to control playback. I suspect many use cases will be covered by simple playback control - but let me know if I’m wrong.
Brad
1 Like
Sysex would be great, but more ‘dangerous’ (perfect filename/string match)? I think A is good enough.
I hope that those who will be using these features will have PCs/Macs/whatever Ethernet networked. Maybe not a dual-core i3, though!
That would be a blessing for Floyd bands. Timing is really critical there.I don’t live in that world anymore, yet I know it well.
Thanks @Brad
I would go with simple to start with 
to see how it works. The icing on the cake can come later if needed.
I do not think tight timing sync ala MTC is needed for Floyd as I made the videos the same length as the Cantablile tracks. So long as the start can be consistent - even with a small amount of latency - that would be close enough. And only a videos had some facet to accentuate a facet of the music anyway.
The problem I had with SCS11 is it starting the videos after getting a trigger could be quite variable and noticeable. So how it waits for triggers and responds to them is probably sloppy. I am sure Brad could do much better
1 Like
A little more progress:
- WebUI now has a MIDI channel selector to choose which MIDI channel it displays. It will also pick this up from the url hash if set (eg:
http://localhost:3000/#ch=1) - WebUI has a full-screen button
- The server now accepts program change events and maps them to a media file via a simple config file that maps program numbers to video/image files.
- Web UI updated to fetch current channel state from server and listens for program changes to a load different media file
- For testing I’m using 3x controller bar buttons to send program change events to switch videos (normally this would be a per-song binding, or a background binding based on the set list program number).
Next up… support for static images.
Note: I’m developing this in NodeJS and will probably release it as a separate project. This will not really be an out of the box, point and click solution and something more for the tinkerers. But, it should still be simple enough that most people should be able to set it up.
4 Likes
Looking good. I have never heard of NodeJS before but have looked it up and it looks interesting.
More than happy to help test it once you are ready. It is depressing, but the UK summer is now close to being over (and the poorest we’ve had in years), but with the nights now noticeably drawing in, the benefit is more time in the toy room!
So happy to commit some of that time to testing it, including running it in a distributed computer setup using my video laptop connected to my GIGPC (and Cantabile) via RtPMIDI
Thank you. I’ll get the basics working, package it up, write some docs and let you know.
1 Like