Restarting Engine after Song Load

Hi Brad,

thanks alot for your response. I understand your point not to implement a hack and sure: if the “never ending story” could be stopped everthing should be fine. It’d set me into the position to decide for which song a reset of the engine is necessary.

Kind regards, humphrey

Hey Guys,

I’ve just put up a new experimental build 3246 that includes some changes for this:

  1. Restart Engine bindings has been renamed to Restart Engine (Full)
  2. A new binding “Restart Engine (Quick)” that restarts just the core audio engine and not the higher level objects like songs and racks etc… Basically this restarts the audio driver, MIDI devices and the core engine itself (multi-core execution engine, worker threads etc…)
  3. Restart Engine (Full) no longer fires song/rack load/unload trigger bindings so it can now be safely used in song load binding.

This is all fairly experimental so give it a go and let me know.

@humphrey - I’d be very interested to hear if the Restart Engine (Quick) binding works for you to reset performance issues.

Brad

2 Likes

Wow, great! This was fast! I’ll give it a try as soon as I can get my hands on the laptop.

Thanks alot, humphrey

A quick response: just implemented the “quick” binding version into every songs of an existing setlist (no binding delay necessary). It seems that load balancing is now working from start as I was not able to generate a decrease in cpu consumtion by restarting engine by hand any more.

I’ll send some more information when more tests are done (I’ll do some heavy load scenarios for this).

Meanwhile: thanks for making this happen,

Regards, humphrey

Some more results:

I set up a small setlist with 2 songs in it: one with 4 instances of DIVA (devine mode) the other with 4 instances of SERUM (pad sounds with lots oscillators inside).

Loading each of these songs from scratch gets me to massive cpu overloads easily (agressive mode active).

After implementing the quick engine reset in each song cpu is far beyond 100% right from the beginning (no need to do an initial reset by hand or the like).

I will go on with some tests but as it seems, this is the perfect fix. Best of all: as the quick-version seems to do the trick I get no noticable additional delay when switching from song to song.:sunglasses:

Kind regards, humphrey

1 Like

Things looking good here too. Quick start is a piece quicker and leaves the engine down for a fraction of a second. The full restart takes media players down, but the quick leaves them intact. I could get away with a quick restart mid-song if need be.

1 Like

Hi,

I premised to do some more testing:
I setup several setlists with (partially) heavy load and everything fine and stable. I always got a massive decrease in heavy load scenarios with the quick version of engine restart. Since I prepared all of my songs with respective bindins (Song/load -> Engine restart/quick). Additional delay (switching time) is not really noticable for me.

One additional observation: when I asked Brad for support, I handled Engine in aggressive mode due to the fact I had several identical plugs in use in parallel. Meanwhile I managed to only have one plugin of a kind in one song. So I gave compatible mode a try. I was surprised to see the same decrease in cpu use when implementing the mentioned binding - no need anymore for agressive mode here. As I don’t think this is a speciallity of my system I thought I should mention this - up to 30% decrease for free may be interesting for other users too…

Kind regards, humphrey

3 Likes

Hi All,

I am reviving this thread because of the behaviors I am experiencing on my performance rigs. I preload some hungry plugs and when the set-list finishes loading and loads Song one on the list the load value is on average about 30% higher than it would be if I did an engine reset before playing anything. On my rig while running Ivory, B-3X and MODO bass with a single reverb buss running Eventide ultra reverb that would be around 70 to 75% but after the engine reset it settles at about 40 to 45% so it is a great load reduction and gives the load headroom needed for B-3X Leslie speed ramping. I tried using Song on load bindings to do a restart after the first song was loaded but it executed before the song was fully loaded and crashed C♪3. I ended up having to use delayed engine restarts after the song was fully loaded. I was wondering what or if anyone else was still using engine restarts to help with load problems and how they do it. Also since I think I am not the only person having these results if there could be a way to address it that was part of the preload sequence and was carried out as the last instruction once the first song was loaded. I have noted that in my case doing the engine restart once corrects the load value and it holds till I shut down. Anyway, if you have something you would like to share please do so. And if you can @brad your observations would be great.

Thanks,

Dave

1 Like

Hi @dave_dore,

as I was the origin of the thread here
my observation on this: I remember the first step was an implementation by brad that enabled a forced restart of the engine. This really solved the problem even though introducing a noticeable delay when calling a new song.

Next step was a different implementation around load balancing in cantabile. After some intense testing I found that brad had done a splendid job (as usual :wink: ) so restart became obsolste (at least for me) and this lasts till now. So no problems here so far (but who knows?).

I will happily do some tests on heavy load scenarios and report back here (maybe tomorrow).

Kind regars, Volker

1 Like

Hi Dave,

I’ve never actually seen this on any of my test machines, so I’m only going on what I’ve heard from others. My suspicion here is that for some reason on first start of the audio engine Windows ties the audio threads to a sub-optimal set of processor cores and on the second restart selects a better set.

The first thing to try would be Options -> Audio Engine -> Thread Affinity Mode -> Forced.

If that doesn’t help, it’d be worth grabbing one of those CPU monitoring tools and look for things like CPU clock throttling or CPU parking. In particular look for differences between before and after restarting the engine.

Brad

1 Like

Hey Dave,

as I din‘t have the same plugs available I just loaded a large setlist with 62 plugs in it, quite some cpu hogs like Omni, Diva, Serum, Viper but also NI Session Horns Pro and Session Strings Pro 2 were involved.

I tried to figure out if cpu meter shows different behaviour after a hard stop and restart of the audio engine. Even though it is hatd to tell sometimes (you know there‘s always some deviation going on) I‘m pretty sure that effects stay beyond 5% if there is really an effect (for me I‘d say there is non).

Maybe one aspect to take note: I always useaudio engine in agressive mode - maybe this makes a different.

Sorry to be not that helpful. Should there be additional tests where I can help out just tell me.

Kind regards and good luck, Volker :smiley:

1 Like

Hi Volker and Brad,

I have been trying out your suggestions and found that “forced” and “aggressive” cpu switches didn’t change things as far as the initial load value after a pre-load of a set list. I ran a CPU monitor and the CPU parking is and was disabled so that box is checked. The load balance seemed to lean to more use (idling at 10% load) on the 1st and 4th cores and less (About 5% load) on the 2 and 3 cores. What I am doing that does correct the load balance is a 2 binding group that does an Engine reset after the song is fully loaded and after a short delay.

My rig didn’t liked a delayed reset with a single binding like this and crashed. It’s likely a no-no with my setups drivers or something, I’m not sure but by making sure the song loads fully (I figured the transport wouldn’t start till it was) and then doing the reset it works fine.

Thanks again to both of you for chiming in, I will stick with my current solution for now, I put it in the a first startup song in the set list and only run it once at the beginning of the show. It holds it’s load values after that till I’m done.

Cheers,

Dave

Yes. Noticed this behavior recently.

I came across this some time last year when we had another computer config discussion:

Interesting enough, I ran Cantabile on 4 machines; the problem only occurred on two of them.

@brad promised to run some tests on this at the time, but apparently nothing conclusive came from that - I imagine it’s pretty difficult when you can’t reproduce the issue…

I just tested the scenario again on my studio PC - now the time load stays stubbornly at around 35%, now matter how often I restart the audio engine.

And TBH, I think that this 35% is a fair representation of the CPU load - maybe the reduction in time load was a reporting / measuring issue? It doesn’t really make sense that my (weaker) old live laptop should run the performance test song at far lower time load than my newer one after engine restart.

3 Likes

While I haven’t taken the time to do scientific tests, I have noticed similar behavior - pre-loading a set list full of hefty plugins, and having the Time Load run what seems to be 20-30% higher than expected. Stopping and starting the audio engine seems to help.

1 Like

@Torsten, I don’t have any conclusions, but it “feels like” a power problem. When I’ve had this happen (before restarting), one of my heavier-layered songs got auditable crunchiness (distortion) and usage went over 100%. I did the on/off trick and usage went way down and the crunchiness went away. It seems like the measurement in that case was correct. This was on my live DeskMini with Core i5 cpu.

1 Like

I had the same experience as @RackedBrain two days ago. After switching thru a few different songs, I heard some crackling sound and saw that the cpu meter was over 100%. I turned the engine off and on again and cpu usage dropped and sound returned to normal.

1 Like

Hi @dave_dore,

I’ve been thinking about this more overnight and I’d really like to try to get to the bottom of this, but since I can’t reproduce it here if you’ve got time to run a couple of tests for me, that’d be great.

Firstly, could you capture a profiler log of this:

  1. Start Cantabile in a state where it’s in high-load state.
  2. Let it run for about 5 seconds or so
  3. Restart the audio engine and confirm it’s switched to low-load state
  4. Let it run for another 5 seconds or so.
  5. From View menu, open the Profiler
  6. Click the menu in top right and choose save.
  7. Send me a copy of the saved profile log.

Also, I’m wondering if this might be related to Cantabile’s multi-core thread pool and whether disabling that changes the behaviour:

  1. Start Cantabile and go to Tools -> Options -> Audio Engine -> Number of Audio Threads -> 1 (disable multi-core support).
  2. Shutdown and restart Cantabile
  3. Take note of the load percentage and then restart the audio engine.

Let me know if the same drop in load happens after restarting the engine when in single core mode.

Brad

1 Like

Hey @brad,

I made 2 profiles using your instructions, one using automatic 4 core and the other using single core. The 4 core profile showed a reduction in load on a restart whereas the single core didn’t look like it changed to me. No notes were played or any load added by any Leslie switching so it was at idle load for the tests. Thanks for looking at it and I hope maybe this will help

ddore october 2021 load profiles.zip (149.4 KB)

Best,

Dave

Hi Dave,

OK, thanks for that. That seems to point towards this being related to the audio thread pool, which kinda makes sense. Leave it with me, but since I don’t want to make any risky changes in the current release series, I might revisit that area for this TechUpdate work I’m doing and see if I can improve things.

Brad

5 Likes