Topic: Idea for Future Pedal Modeling?

Pianoteq's pedal sound is already quite good, perhaps the best out there in software pianos, but I think there's a lot more potential for the realism it could offer.

This is obviously extreme, but crank a piano's volume to the max and then also crank "pedal noise" to the max (you might even need to turn your main volume up a bit). Now depress your (continuous MIDI) pedal very slowly. You'll hear the pedal sound rapidly engage and reengage over and over again; same when you very slowly let off of the pedal. A real piano's pedal of course won't do this, and it will be louder or quieter depending on the speed the pedal is depressed.

Of course the pedal usually won't be nearly this loud, so it's not something most people will ever even notice, but when pedaling slowly in a very quiet passage, or when pedaling with a lot of force, the realism isn't quite there. And then of course there's the repedal effect that can be VERY dramatic, which isn't modeled yet.

I've also tried this with a good MIDI expression pedal (assigning it to the sustain pedal)--in case my Kawai pedal wasn't sending enough information to give a fair impression--but it doesn't actually make much difference, so doesn't seem to be an issue of MIDI resolution. I also realize this might be exceedingly difficult to improve, given the nature of MIDI...but perhaps there's some way to solve the puzzle?

---------------------

After thinking a bit, perhaps this could work:

On most real pianos, when depressing the sustain pedal there's a good bit of range at both the top and bottom where the dampers are not partially touching the strings. In other words, the "half-pedal" zone is a fairly small range; with the top and bottom ranges having no effect. Pianoteq seems to mimic this pretty nicely within the 0-127 MIDI range, with nothing from 0 to somewhere in the 20's (upper pedal range), and again nothing from around 90-127 (lower pedal range).

When half-pedaling, you're typically moving the pedal extremely carefully, because the effective range can be very small and tricky to hit, and so you're also usually moving the pedal slower than normal. I could be overlooking something important, but perhaps if Pianoteq saw a really quick pedal depression begin (in the 0-25 range perhaps), it could assume the dampers would fully lift, and use an appropriately loud damper lifting sound (depending on exactly how rapid the 0-25 range was). If on the other hand Pianoteq detected a fairly slow pedal depression in the 0-25 range...or especially if the pedal was kept above the 15-ish range and moving fairly slowly...then it could use a half pedal sound. That sound might be almost inaudible, as would a very slow/delicate full pedal, so the difference might be almost imperceptible there. So a sort of predictive pedal system. It could work the same way with the pedal release. The pedal release can be particularly loud, especially when students are still learning to control the pedal.

I think doing something along these lines could give a lot more control over the damper sound/dynamics, and go far further than any other digital piano (modeled or sampled). The next step would be to model a strong/rapid repedal, where the dampers are raised, dropped onto the strings and immediately raised again. This produces a strong effect where the dampers are made to "bounce" off the strings.

Last edited by NathanShirley (03-12-2022 08:01)

Re: Idea for Future Pedal Modeling?

NathanShirley wrote:

Pianoteq's pedal sound is already quite good, perhaps the best out there in software pianos, but I think there's a lot more potential for the realism it could offer.

This is obviously extreme, but crank a piano's volume to the max and then also crank "pedal noise" to the max (you might even need to turn your main volume up a bit). Now depress your (continuous MIDI) pedal very slowly. You'll hear the pedal sound rapidly engage and reengage over and over again; same when you very slowly let off of the pedal. A real piano's pedal of course won't do this, and it will be louder or quieter depending on the speed the pedal is depressed.

Of course the pedal usually won't be nearly this loud, so it's not something most people will ever even notice, but when pedaling slowly in a very quiet passage, or when pedaling with a lot of force, the realism isn't quite there. And then of course there's the repedal effect that can be VERY dramatic, which isn't modeled yet.

I've also tried this with a good MIDI expression pedal (assigning it to the sustain pedal)--in case my Kawai pedal wasn't sending enough information to give a fair impression--but it doesn't actually make much difference, so doesn't seem to be an issue of MIDI resolution. I also realize this might be exceedingly difficult to improve, given the nature of MIDI...but perhaps there's some way to solve the puzzle?

---------------------

After thinking a bit, perhaps this could work:

On most real pianos, when depressing the sustain pedal there's a good bit of range at both the top and bottom where the dampers are not partially touching the strings. In other words, the "half-pedal" zone is a fairly small range; with the top and bottom ranges having no effect. Pianoteq seems to mimic this pretty nicely within the 0-127 MIDI range, with nothing from 0 to somewhere in the 20's (upper pedal range), and again nothing from around 90-127 (lower pedal range).

When half-pedaling, you're typically moving the pedal extremely carefully, because the effective range can be very small and tricky to hit, and so you're also usually moving the pedal slower than normal. I could be overlooking something important, but perhaps if Pianoteq saw a really quick pedal depression begin (in the 0-25 range perhaps), it could assume the dampers would fully lift, and use an appropriately loud damper lifting sound (depending on exactly how rapid the 0-25 range was). If on the other hand Pianoteq detected a fairly slow pedal depression in the 0-25 range...or especially if the pedal was kept above the 15-ish range and moving fairly slowly...then it could use a half pedal sound. That sound might be almost inaudible, as would a very slow/delicate full pedal, so the difference might be almost imperceptible there. So a sort of predictive pedal system. It could work the same way with the pedal release. The pedal release can be particularly loud, especially when students are still learning to control the pedal.

I think doing something along these lines could give a lot more control over the damper sound/dynamics, and go far further than any other digital piano (modeled or sampled). The next step would be to model a strong/rapid repedal, where the dampers are raised, dropped onto the strings and immediately raised again. This produces a strong effect where the dampers are made to "bounce" off the strings.

You can adjust the pedal behaviour in pianoteq , just go the velocity curve , select the tab pedal and then you can adjust the curve to change  the pedal response the way you want it to work ( linear or any type of curve) obviously it depends upon how good your continuous pedal is . I personnally use a Roland DP10 which is working really well for half pedalling

Last edited by joannchr (03-12-2022 20:22)

Re: Idea for Future Pedal Modeling?

According Philippe the noises in modern grand pianos are already  modelled.

I presume what you find as problems (not perfection) are MIDI limitations or limitations of your MIDI pedal or just matter of adjustments.

Re: Idea for Future Pedal Modeling?

joannchr wrote:

I personnally use a Roland DP10 which is working really well for half pedalling

So I'm not actually talking about the function of pedaling or half pedaling within Pianoteq, but of the sound the dampers produce. The basic function works just fine.

Beto-Music wrote:

According Philippe the noises in modern grand pianos are already  modelled.

I presume what you find as problems (not perfection) are MIDI limitations or limitations of your MIDI pedal or just matter of adjustments.

Yes, the pedal sound has been modeled for some time now, I'm just thinking of ways the model could be made to sound better and respond more realistically to the speed of the pedal.

I figured this would be a bit difficult to explain clearly... If you follow the directions below, you should be able to hear what I'm getting at:

NathanShirley wrote:

This is obviously extreme, but crank a piano's volume to the max and then also crank "pedal noise" to the max (you might even need to turn your main volume up a bit). Now depress your (continuous MIDI) pedal VERY slowly. You'll hear the pedal sound rapidly engage and reengage over and over again; same when you very slowly let off of the pedal. A real piano's pedal of course won't do this, and it will be louder or quieter depending on the speed the pedal is depressed.

Let me see if I can summarize and clarify what I wrote about before:

With a real piano, if you depress the pedal very slowly you won't hear any sound as the dampers lift from the strings without disturbing them. If, on the other hand, you depress the pedal very rapidly you'll hear a fairly significant sound. That sound will be significant even if you don't bottom out the pedal.*

*NOTE: If you bottom out the pedal by really stomping on it, the resulting thud will resonate throughout the piano giving a really loud sound. I believe Philippe said he didn't think the bottoming out of the pedal should be included in Pianoteq since part of that sound is created from the pedal slamming down into the felt/wood underneath, and the MIDI pedal used when playing Pianoteq would also create a sound when stomping on it. I understand that perspective, but since the sound then resonates through the piano, I think it would make sense for Pianoteq to model it. (End side note.)

So with Pianoteq you don't really get any variation in the volume of the pedal sound. If you press the pedal very rapidly you hear more or less the same volume as if you press it very slowly.

Now, when you release the pedal on a real piano the range of volume is much greater still. If you release it very gently and slowly you won't hear much of anything other than the strings as their vibration is gently stifled. If, on the other hand, you release the pedal very rapidly, there will be a very strong sound as the dampers fall onto the strings, resonating through the instrument. This is true even if you don't let the pedal come fully up (but still let the dampers hit the strings), so in other words you don't let the pedal's mechanics slam into anything. If you do let the pedal come up fully and very rapidly (say if you depress the pedal and then slide your foot off the pedal so it snaps up quickly), then the sound will be even louder still. But that might get back to the question of how far the model should be taken which I brought up in the note above.

Again, in Pianoteq the sound is basically the same volume whether you release the pedal slowly or rapidly.

And finally, if you repedal extremely rapidly on a real piano ("repedal accent"), the resulting thud as the dampers hit the strings and then "bounce" off can be quite dramatic. You start with the pedal depressed, then very rapidly release the pedal, but the instant the dampers hit the strings you rapidly depress the pedal again (so the thud of the dampers resonates through the undampened strings). I suppose some would argue this might fall into the "extended technique" category, but since it doesn't require reaching into the piano or drumming on the case, and it's a natural effect of pedaling in a specific way, I would argue it would make sense to eventually model.

The sustain pedal is such a fundamental part of a piano, I think it deserves significant attention. That added realism would absolutely blow other piano software away.

Re: Idea for Future Pedal Modeling?

@NathanShirley I think your idea is awesome and should be implemented by pianoteq. Things like these are where modeling really shines compared to sampled: you can really create something that will make a very intimate recording extremely realistic, compared to sampling where even repedaling is either not supported or a lot of work to get it right.

Regarding the "thud" sound, in addition with agreeing with what you say about the resonances, I strongly disagree with Philippe for the thud itself! Heck, the action also make noises, still pianoteq includes the action noises, why that difference of treatment? It should give the option to turn it off if desired, but it needs to be there. If nothing else, to allow purely digital recording and not forcing one to put a physical microphone next to one's foot to capture the physical sounds of the thud!!

I really hope they see this message and act accordingly (you may want to ping them in the support in case they are too busy with other stuff and overlook this thread).

Where do I find a list of all posts I upvoted? :(

Re: Idea for Future Pedal Modeling?

NathanShirley, I can see what you mean on my Korg PU-2 Pedal. It is discrete and has three trigger points. Midi controller 64, three discrete values 38, 74, 127 and of course 0 when not engaged.

Depressing it unnaturally slow I hear three pedal noises with increasing loudness (loudness depending on Pianoteq's pedal velocity curve). I can see this three midi events in the midi event monitor. The same when releasing the pedal unnaturally slow.

Depressing the pedal fast or very fast to the bottom sometimes less controller values are scanned and triggered, for example
0, 38, 127
0, 74, 127
0, 127 (very fast)

So there are two pedal noises or just one.

Last edited by groovy (04-12-2022 16:13)

Re: Idea for Future Pedal Modeling?

personally i'm 100% behind this idea.  there are all kind of cool noises one can make with the damper pedal.  of course maybe most consumers aren't anxiously waiting for these effects, but i would be into it.

Re: Idea for Future Pedal Modeling?

groovy wrote:

NathanShirley, I can see what you mean on my Korg PU-2 Pedal. It is discrete and has three trigger points. Midi controller 64, three discrete values 38, 74, 127 and of course 0 when not engaged.

Depressing it unnaturally slow I hear three pedal noises with increasing loudness (loudness depending on Pianoteq's pedal velocity curve). I can see this three midi events in the midi event monitor. The same when releasing the pedal unnaturally slow.

Depressing the pedal fast or very fast to the bottom sometimes less controller values are scanned and triggered, for example
0, 38, 127
0, 74, 127
0, 127 (very fast)

So there are two pedal noises or just one.

I don't have a pedal with that small a number of trigger points, but yes, I can imagine what it would sound like. With a pedal with more "resolution" you get the same effect except the reengaging of the damper sound happens over and over, very rapidly (or less rapidly if you move the pedal very slowly). When I use a Yamaha expression pedal (rather than a standard MIDI damper pedal) with a very high resolution (pretty much 127 trigger points), the effect is actually not all that much different than with my Kawai pedal (with "decent resolution"). I should note the reengaging sound of the dampers is smoother on some pianos than on others.

I suppose if there were a sort of predictive element to the damper sound, it would still be less accurate with a "low resolution" pedal, but it might still work decently.

Re: Idea for Future Pedal Modeling?

In principle I think this is a good idea. Something that should be in the Pianoteq modelling.
I wonder though if the desired pedal behaviour you describe has not been modelled as yet for two possible reasons.

1) the number of users that are likely to benefit and have good enough pedals currently.
2) the CPU hit that the average user might experience may be deemed too high relative to how audible this mechanical factor is in the vast majority of usage cases?

Re: Idea for Future Pedal Modeling?

A better term than "pedal velocity curve" is "pedal position curve" in my opinion. With an analog pot and a 7-bit-controller we have 127 positions. In the worst case 127 different "whoosh" sounds are triggered when completely depressed and again 127 other sounds when released. Not very efficient for the CPU and the narrow MIDI-bus. But normally the value rate is limited by the scanning frequency ("sample frequency") of the A/D-converter or later by filtering the events. Nathan, how many sustain pedal events do you count in Ptq's midi monitor in one fullrange pedal movement with medium speed?

From the manual:
-------------------------------------------------------------------------------------------
1.4. Features in short

Natural instrument noises (sampled or modelled) including action key release noise, damper noise at key release (bass notes) and sustain pedal noise: pedal velocity dependent “whoosh” produced by the dampers rising together from the strings or falling down
-------------------------------------------------------------------------------------------

I don't know how the velocity is modelled using the positions, but the algorithm is probably more than determining the up/down direction (midi controller 64 increasing/decreasing) or measuring the time between value 127 and the next 0.

Re: Idea for Future Pedal Modeling?

Key Fumbler wrote:

In principle I think this is a good idea. Something that should be in the Pianoteq modelling.
I wonder though if the desired pedal behaviour you describe has not been modelled as yet for two possible reasons.

1) the number of users that are likely to benefit and have good enough pedals currently.
2) the CPU hit that the average user might experience may be deemed too high relative to how audible this mechanical factor is in the vast majority of usage cases?

That could very well be true. It's interesting, the piano sustain on most keyboards, even fairly "good" ones, is really pretty terrible. I started to suspect that part of the reason, aside from manufacturers wanting to use less memory with smaller samples, was to make it so people new to piano could basically just hold down the sustain pedal and not have to worry about the sound getting extremely muddy as happens on a real piano. The side effect is that many people used to keyboards are completely thrown off by the way the sound on a real piano blurs as they aren't refreshing the pedal enough, and/or with enough care.

Pianoteq has done a nice job extending the sustaining length and power, making the ability to blur sound with the damper pedal much more realistic. This might seem like a drawback to someone not used to playing piano, but it really just opens up so much more sonic possibility. It also makes the transition to a real piano easier.

I consider the damper sound proposals I've made here to be similar. The damper pedal might feel harder to control for a beginner with this added realism, but with practice it will open up more sonic possibilities and make playing real pianos easier.

As to the CPU hit, I suppose it could be significant. At the same time CPUs have come a long way...Perhaps it would make sense to have an "eco" mode, perhaps even on by default, and then a "performance" mode for that extra depth?

Groovy wrote:

But normally the value rate is limited by the scanning frequency ("sample frequency") of the A/D-converter or later by filtering the events. Nathan, how many sustain pedal events do you count in Ptq's midi monitor in one fullrange pedal movement with medium speed?.......I don't know how the velocity is modelled using the positions, but the algorithm is probably more than determining the up/down direction (midi controller 64 increasing/decreasing) or measuring the time between value 127 and the next 0.

So at more "average" pedaling speeds, my Kawai pedal sends between about 6 and 12 messages when I depress the pedal, and a good bit more than that when I raise the pedal (I raise the pedal slower towards the top to avoid the dampers from hitting the strings with too much force [from playing physical pianos]). But of course when pedaling slowly--particularly when using a bit of half pedal--I'll get a lot more MIDI messages. With a really fast pedal, I might only see a couple messages, and if I stomp it I can get straight from 0 to 127 (sometimes).

Last edited by NathanShirley (06-12-2022 18:41)

Re: Idea for Future Pedal Modeling?

NathanShirley wrote:
joannchr wrote:

I personnally use a Roland DP10 which is working really well for half pedalling

So I'm not actually talking about the function of pedaling or half pedaling within Pianoteq, but of the sound the dampers produce. The basic function works just fine.

Beto-Music wrote:

According Philippe the noises in modern grand pianos are already  modelled.

I presume what you find as problems (not perfection) are MIDI limitations or limitations of your MIDI pedal or just matter of adjustments.

Yes, the pedal sound has been modeled for some time now, I'm just thinking of ways the model could be made to sound better and respond more realistically to the speed of the pedal.

I figured this would be a bit difficult to explain clearly... If you follow the directions below, you should be able to hear what I'm getting at:

NathanShirley wrote:

This is obviously extreme, but crank a piano's volume to the max and then also crank "pedal noise" to the max (you might even need to turn your main volume up a bit). Now depress your (continuous MIDI) pedal VERY slowly. You'll hear the pedal sound rapidly engage and reengage over and over again; same when you very slowly let off of the pedal. A real piano's pedal of course won't do this, and it will be louder or quieter depending on the speed the pedal is depressed.

Let me see if I can summarize and clarify what I wrote about before:

With a real piano, if you depress the pedal very slowly you won't hear any sound as the dampers lift from the strings without disturbing them. If, on the other hand, you depress the pedal very rapidly you'll hear a fairly significant sound. That sound will be significant even if you don't bottom out the pedal.*

*NOTE: If you bottom out the pedal by really stomping on it, the resulting thud will resonate throughout the piano giving a really loud sound. I believe Philippe said he didn't think the bottoming out of the pedal should be included in Pianoteq since part of that sound is created from the pedal slamming down into the felt/wood underneath, and the MIDI pedal used when playing Pianoteq would also create a sound when stomping on it. I understand that perspective, but since the sound then resonates through the piano, I think it would make sense for Pianoteq to model it. (End side note.)

So with Pianoteq you don't really get any variation in the volume of the pedal sound. If you press the pedal very rapidly you hear more or less the same volume as if you press it very slowly.

Now, when you release the pedal on a real piano the range of volume is much greater still. If you release it very gently and slowly you won't hear much of anything other than the strings as their vibration is gently stifled. If, on the other hand, you release the pedal very rapidly, there will be a very strong sound as the dampers fall onto the strings, resonating through the instrument. This is true even if you don't let the pedal come fully up (but still let the dampers hit the strings), so in other words you don't let the pedal's mechanics slam into anything. If you do let the pedal come up fully and very rapidly (say if you depress the pedal and then slide your foot off the pedal so it snaps up quickly), then the sound will be even louder still. But that might get back to the question of how far the model should be taken which I brought up in the note above.

Again, in Pianoteq the sound is basically the same volume whether you release the pedal slowly or rapidly.

And finally, if you repedal extremely rapidly on a real piano ("repedal accent"), the resulting thud as the dampers hit the strings and then "bounce" off can be quite dramatic. You start with the pedal depressed, then very rapidly release the pedal, but the instant the dampers hit the strings you rapidly depress the pedal again (so the thud of the dampers resonates through the undampened strings). I suppose some would argue this might fall into the "extended technique" category, but since it doesn't require reaching into the piano or drumming on the case, and it's a natural effect of pedaling in a specific way, I would argue it would make sense to eventually model.

The sustain pedal is such a fundamental part of a piano, I think it deserves significant attention. That added realism would absolutely blow other piano software away.

I see what you mean.
I think the solution is rather simple : we just need to calculate continuously the speed of the pedal by deriving its position (ie. CC 64 values) against time. The pedal sound should be proportional to the pedal current speed.
For on/off only sustain pedals, the speed is always equal to 0 or a maximum value when engaged/disengaged. For continuous pedal, speed can be inbetween.

Last edited by Paulo164 (06-12-2022 16:36)

Re: Idea for Future Pedal Modeling?

Hi Nathan, yes, this question is indeed a difficult question. We found for now no way to make it work as you would like without having at our disposal a continuous MIDI message, which is *very* far from what most pedals provide (we need something like 100 values per second to start having something nice and smooth). The reason is simple: we cannot read in the future, so we don't know if at the next time step the pedal will go up or down, and unfortunately there is no workaround, except ignoring the received steps and providing a uniform noise (possibly depending on a calculated velocity), or introducing a large delay which is of course not desired. Another way to go would be to try to guess from the music itself, but I'm not sure that would lead somewhere. Your ideas look interesting, we did have thoughts along that line, but then there are counterexamples where it won't work. Nevertheless, we will have to think about it to see how we could improve further the sustain pedal behavior.

Re: Idea for Future Pedal Modeling?

Perhaps there is a gap in the market for a separates high end USB high resolution piano pedals solution. HD MIDI with mechanical behaviour designed to simulate the action and feel of real piano foot pedals.

Re: Idea for Future Pedal Modeling?

Key Fumbler wrote:

There you go, that's who said that.

Oh sorry, I'll fix that.

Re: Idea for Future Pedal Modeling?

Philippe Guillaume wrote:

Hi Nathan, yes, this question is indeed a difficult question. We found for now no way to make it work as you would like without having at our disposal a continuous MIDI message, which is *very* far from what most pedals provide (we need something like 100 values per second to start having something nice and smooth). The reason is simple: we cannot read in the future, so we don't know if at the next time step the pedal will go up or down, and unfortunately there is no workaround, except ignoring the received steps and providing a uniform noise (possibly depending on a calculated velocity), or introducing a large delay which is of course not desired. Another way to go would be to try to guess from the music itself, but I'm not sure that would lead somewhere. Your ideas look interesting, we did have thoughts along that line, but then there are counterexamples where it won't work. Nevertheless, we will have to think about it to see how we could improve further the sustain pedal behavior.

I figured it might be a bit more complex than what I was imagining. And yes, I see what you mean about the "predictive" idea working in many situations, but not all (sudden pedal direction change). My Yamaha expression pedal (not a standard MIDI sustain pedal) seems to send somewhere in the 70 messages per second range (my Kawai pedal seems to be somewhere in the 20's).

Anyway I do appreciate your ongoing thinking about this. Perhaps one day Pianoteq will crack it.

Last edited by NathanShirley (06-12-2022 19:12)

Re: Idea for Future Pedal Modeling?

Indeed it is not an easy problem, more of the kind ...

"If you think you understand quantum mechanics, you don't understand quantum mechanics." (Richard P. Feynman?)

I like the idea of an "eco mode". Maybe the best eco mode is to let the user opt-out a CPU- and MIDI-intensive effect like pedal noise. In other words a switch pedal noise ON/OFF.

Don't get me wrong, I like elaborate features like pedal noise etc. very much, but I think it is always a good idea to stay modular and CPU friendly when possible. -

Or is it "already there"?
Just saw that the pedal noise fader shows "Off" in the left position. Does this completely avoid the cost of sound generation at each midi controller 64 event? Or just mutes the audio level?

Thank you very much for your insights, Philippe!