Binaural, head tracker, virtual reality... why not the PIANOTEQ future (Page 1 of 2)

Great to see you expressing this desire YvesTh

Also thank you for that interesting link dv.

I share the hope that perhaps Pianoteq's radiated model may be actually greatly suited to 'repositioning' existing (binaural earphone placement) in real time - and their own I.P. could be the best solution for this.?. anyway worth putting that thought up at top. The rest could turn out moot if that's the case. But fun - and useful stuff abounds..

The Waves Abbey Road Studio 3 NX plugin comes with also surround sound modes 5.1 and 7.1 - based upon the surround speakers positioned in their studio.. engaging that mode may improve the experience yet further beyond the stereo mode.

Some of the other studios in that NX range may just have 5.1 or no surround.

When I mentioned that plugin, I did also mention it is not 1:1 like a real piano in the room, certainly, but probably the easiest '2 speakers in a space' solution (selectable between small, near-field and the larger wall mounted babies). My apologies, at the time I overlooked surround mode though. Doh! Hope you give that a try, to see what it adds to the experience for you.

Whilst NX is primarily for listening/monitoring/mixing, there are many other valuable plugins on the recording side of the signal..

My fav is "dearVR Pro" by "Dear Reality". This may cost 17 times more than an NX plugin - and have more of a learning curve, which is the main reason I didn't suggest it first. But their less expensive "DearVR Mix" plugin shares much overlap with NX.

Like Pianoteq, earnest founders - Dear Reality is in Düsseldorf, Germany so nice and close-ish to Philippe and Julien maybe.. they are now part of Sennheiser group - so some layers of consults may require time, meetings - etc. - but if founders are still in some position to drive decisions, maybe they'd be open to some licensing or other IP licensing exchange? Sometimes, good things happen like this between 2 wonderful teams.

https://www.dear-reality.com/

DearVR Pro also has head-tracking (and also sibling plugins) and being designed to also record with (not just for the 'listen bus' path) it may be more complex (in settings) but quite notable inre some interesting advantages. There are others but I find them so far to be too specific to 'static environments' (like point-click add a listening source - and nice, but not as malleable) and others too specific to older stereo-ization techniques (HAAS, Mid/Side - also nice but I will apply that branch of things myself normally).

If anyone's interested in DAW ideas/templates for Pianoteq, my usage is usually to have two sends from a Pianoteq track, to 'DearVR Pro' on 2 buses, one panned L84, other R84 (note your DAW may use different 'panning Law' so different numbers may be better for you - my DAW of choice here is Studio One by Presonus - I find their pan OK - but they do provide a binaural pan too which you could place on both/either/any chanel/bus FX/return - endless).

Both of these buses mentioned, have the DearVR Pro plugin set to stereo spatialization (instead of its binaural one - also a fine choice which might be better for your useage - I A/B both and over a long timeline usually find myself with the non-binaural mode set) - this just means the plugin is not generating the whole space in 1 instance, but I can choose 'depth' etc. by altering my pan (and other on-deck settings, tones, other plugins etc. - because.. in reality, left space IS often rather different to the right side and vice versa - aha! Esp. when a raised piano lid is taken into account, or being on a stage with wall closer at left, audience open space to the right) instead of fussing within the plugin itself (once its own settings are established)..

I genuinely feel it offers a formidable kind of 'parallel' process which gives us incredible control).. using this plugin has given me one of the nicest ways I've found to position Pianoteq as a piano in a space - infinite variations, no exact rules - but I find few things as simple, yet effective as this plugin for a full suite solution beyond listening, and I am extremely satisfied when applied just to reverb only - at least when going for a solo piano 'album' sound. I may still use Pianoteq's own reverb (the spatial effect may make it so reverb can be vastly reduced in many use cases), or other reverbs may be applied on further FX sends to subtly build a total environment - every piece calls for something at least subtly different, at least to my ways of working. I've blabbed in the past about my use of 'close player bubble' + 'near cabinet bubble' and 'third order distant environment bubble'.. going beyond spatial plugins, head tracking etc.. everyone can develop an enormous range of 'their own spaces' by leveraging a little from here, a little from there - but often that missing element is movement at least for "player perspective" presets.)

Perhaps Dear Reality &/or via Sennheiser AMBEO might present a viable tech partnership or licensing end use for dev. opportunities with Pianoteq?

There may be strong merit in Pianoteq having the piano relatively stationary (as signal source), and 'the space' (radiated model) generating most of the spatialized effect of movement... all interesting stuff to consider. Perhaps that's doable with the least hassle?.. if possible of course.

All we can do is post these observations as they become relevant and, like you and some others here, enjoy the possibilities, knowing however that there are a lot of avenues for advancing the body of work for the relatively small but excellent team to toss around

Just should add, very few virtual instruments offer anything like a VR kind of thing 'baked in' - I worry more with time, that more new readers pass through thinking 'practically any discussion' we have here means we're talking about deficiencies, rather than opportunities. I see beyond skies, few limits within this audio VR field - although having to wear a tracking device, or enabling a facial recog cam can be a show stopper for a substantial number of pianists (that has come up before) - but even so, for those seeking 'wow' beyond norms, I'm all for particularly blue-tooth device tracking on a headset. It may be so 'normalized' in future, that I'd think for any VSTI maker to not consider it at all, could materialize at least some mild disadvantage further down the track.

Thank you for such an interesting ideas

I tried to assign a fader to the head angle and i move it during playing. My computer is old and if my movement is not too speed Pianoteq follows the movement without any glitch in the sound.

Let's tell them loud and clear : PIANOTEQ IS WONDERFUL .... TRY IT, BUY IT and PLAY IT...

For the head tracking it exist a little software ( Aitrack+opentrack) witch control the head movements (with webcam) and send coordonates to simulator or gaming. It is probably possible to transform that signal in a midi signal and send it to pianoteq... IMO It's a piece of cake compared to what Modartt has already achieved...

And why not soon, the possibility of 3D VR headset using with the perfect 3D modelisation of instruments...
Let's have a dream... I'm playing on the grand C.Bechstein on stage in a concert hall... And I have in front of me, the opened piano and the hall... but not the audience I mean...

Thanks for the test but I was talking about mic placement (particularly in binaural mode) to simulate head movement, not fader. Have you been able to test that too, and does is work smoothly? If so, PTQ already offer this feature under the hood, all we need is the user interface to make it happen which anybody can do with the very elegant JSON interface! That would be fantastic!!!!!

Yes yes yes!

Even though nobody told, that was the premise of all this discussion. If the mic position can be moved while playing with no interruptions (as speculating in the previous paragraph), this is basically already done!

Yes, I was talking about mic placement, I just used the fader to assign a midi value of the head angle to turn the head gradually, during playing... It works, except for  too fast deplacement.

What actually happens then during fast or rapid head movement?

I think something like this would have to be pretty darned reliable in operation, otherwise it's going to seem very fake with say jumpy performance which could be very jarring.

Interesting idea though.

I wish to sign up for your newsletter sir My thanks also for encouragement.

The API is something I haven't yet explored - but wow I'm impressed by all these ingenious ideas here!

The last APIs I spent time with eventually ended up with control panels, or dashboards to make most common tasks simple for any admin/users. The full power of that API I'd love to see as an accessible front-end pane.

In my proposition it would not be a question of creating a spectacular artificial sound, surround multicanal or other, undoubtedly interesting, but just reproduce the simple binaural sound ( 2 mics an headphones) with the continuous tracking of the head, simply to get closer to a real listening of the pianist's perspective ....

See also https://gitlab.com/hberntsen/pianoteq-head-tracking

Of course it's good to get this interest under the developers' radar, but do not forget that YvesTh has ALREADY tested and REPORTED IN AN EARLIER MESSAGE, that it now works without interruption. Why don't you test it yourself if you've got time?

It supports OSC like many others that I mentioned before. OSC should be easy to convert to MIDI if this particular one doesn't do the conversion out of the box (just Google OSC-to-MIDI conversion). Moreover, at least for quick and dirty tests you can experiment WITH YOUR PHONE in at least two different ways (motion sensor and eye tracking) without need to purchase anything. Again, why don't you test it yourself if you've got time?

I can use pianoteq in studio one, I can use the "NX waves abbey road studio 3" plugin, the NX head tracker works very well. But I don't have the skills to transfer the data from nx to pianoteq to drive the binaural headphones. I would need an application like NX-to-midi, if someone is able to do it...

Same result as budo here. I tried editing a bit the parameters to see if anything would work, but I only managed to crash Pianoteq. So I decided to try getParameters to see what they would look like.

curl -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","method":"getParameters()"}' http://localhost:8081/jsonrpc
curl -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","method":"getParameters"}' http://localhost:8081/jsonrpc

That crashed the app too, very weird.

That was with Pianoteq 7.5.4

I reported it to support, pointing them to this thread.

Thanks! This is exactly what I get, too. Can you try to run the command from my reply to @dv, above?

{"id":1,"jsonrpc":"2.0","result":[{"id":"Condition","index":0,"name":"Condition","normalized_value":0.0,"text":"0","unit":""},{"id":"Dynamics","index":1,"name":"Dynamics","normalized_value":0.2929292917251587,"text":"30","unit":"dB"},{"id":"Pitch Bend","index":2,"name":"Pitch Bend","normalized_value":0.5,"text":"0","unit":"cents"},{"id":"Unison","index":3,"name":"Unison Width","normalized_value":0.5,"text":"1.00","unit":""},{"id":"Stereo Width","index":4,"name":"Stereo Width","normalized_value":0.4541340470314026,"text":"0.80","unit":""},{"id":"Hammer Noise","index":5,"name":"Hammer Noise","normalized_value":0.3103448152542114,"text":"1.00","unit":""},{"id":"Sustain Pedal","index":6,"name":"Sustain Pedal","normalized_value":0.0,"text":"0","unit":""},{"id":"Soft Pedal","index":7,"name":"Soft Pedal","normalized_value":0.0,"text":"0","unit":""},{"id":"Sostenuto Pedal","index":8,"name":"Sostenuto Pedal","normalized_value":0.0,"text":"0","unit":""},{"id":"Harmonic Pedal","index":9,"name":"Harmonic Pedal","normalized_value":0.0,"text":"0","unit":""},{"id":"Rattle Pedal","index":10,"name":"Rattle Pedal","normalized_value":0.0,"text":"0","unit":""},{"id":"Lute Stop Pedal","index":11,"name":"Buff Stop Pedal","normalized_value":0.0,"text":"0","unit":""},{"id":"Celeste Pedal","index":12,"name":"Celeste Pedal","normalized_value":0.0,"text":"0","unit":""},{"id":"Mozart Rail","index":13,"name":"Mozart Rail","normalized_value":0.0,"text":"0","unit":""},{"id":"Super Sostenuto","index":14,"name":"Super Sostenuto","normalized_value":0.0,"text":"0","unit":""},{"id":"Pinch Harmonic Pedal","index":15,"name":"Pinch Harmonic Pedal","normalized_value":0.0,"text":"0","unit":""},{"id":"Glissando Pedal","index":16,"name":"Glissando Pedal","normalized_value":0.0,"text":"0","unit":""},{"id":"Harpsichord Register[1]","index":17,"name":"Harpsichord Register 1","normalized_value":1.0,"text":"1","unit":""},{"id":"Harpsichord Register[2]","index":18,"name":"Harpsichord Register 2","normalized_value":0.0,"text":"0","unit":""},{"id":"Harpsichord Register[3]","index":19,"name":"Harpsichord Register 3","normalized_value":0.0,"text":"0","unit":""},{"id":"Reversed Sustain","index":20,"name":"Reversed Sustain","normalized_value":0.0,"text":"0","unit":""},{"id":"Volume","index":21,"name":"Volume","normalized_value":0.7272727489471436,"text":"0","unit":"dB"},{"id":"Post Effect Gain","index":22,"name":"Post Effect Gain","normalized_value":0.5354166626930237,"text":"+0.9","unit":"dB"},{"id":"Diapason","index":23,"name":"Diapason","normalized_value":0.3333333432674408,"text":"440.0","unit":"Hz"},{"id":"Octave Stretching","index":24,"name":"Octave Stretching","normalized_value":0.23108820617198944,"text":"1.00","unit":""},{"id":"Unison Balance","index":25,"name":"Unison Balance","normalized_value":0.5,"text":"0","unit":""},{"id":"Direct Sound","index":26,"name":"Direct Sound Duration","normalized_value":0.5,"text":"1.00","unit":""},{"id":"Hammer Hardness Piano","index":27,"name":"Hammer Hard. Piano","normalized_value":0.15000000596046448,"text":"0.30","unit":""},{"id":"Hammer Hardness Mezzo","index":28,"name":"Hammer Hard. Mezzo","normalized_value":0.4000000059604645,"text":"0.80","unit":""},{"id":"Hammer Hardness Forte","index":29,"name":"Hammer Hard. Forte","normalized_value":0.800000011920929,"text":"1.60","unit":""},{"id":"Profil[1]","index":30,"name":"Spectrum Profile 1","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Profil[2]","index":31,"name":"Spectrum Profile 2","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Profil[3]","index":32,"name":"Spectrum Profile 3","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Profil[4]","index":33,"name":"Spectrum Profile 4","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Profil[5]","index":34,"name":"Spectrum Profile 5","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Profil[6]","index":35,"name":"Spectrum Profile 6","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Profil[7]","index":36,"name":"Spectrum Profile 7","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Profil[8]","index":37,"name":"Spectrum Profile 8","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Strike Point","index":38,"name":"Strike Point","normalized_value":0.5137072801589966,"text":"1/10.8","unit":""},{"id":"Pinch Harmonic Point","index":39,"name":"Pinch Harmonic Point","normalized_value":1.0,"text":"1/2.0","unit":""},{"id":"Pickup Symmetry","index":40,"name":"Pickup Symmetry","normalized_value":0.0,"text":"0","unit":""},{"id":"Pickup Distance","index":41,"name":"Pickup Distance","normalized_value":0.444444477558136,"text":"1.00","unit":""},{"id":"Soft Level","index":42,"name":"Soft Level","normalized_value":0.5,"text":"0.50","unit":""},{"id":"Clavinet Low Mic","index":43,"name":"Clavinet Low Mic","normalized_value":1.0,"text":"+1.00","unit":""},{"id":"Clavinet High Mic","index":44,"name":"Clavinet High Mic","normalized_value":1.0,"text":"+1.00","unit":""},{"id":"Impedance","index":45,"name":"Impedance","normalized_value":0.4999999701976776,"text":"1.00","unit":""},{"id":"Cutoff Freq","index":46,"name":"Cutoff","normalized_value":0.4999999701976776,"text":"1.00","unit":""},{"id":"Q Factor","index":47,"name":"Q Factor","normalized_value":0.4999999701976776,"text":"1.00","unit":""},{"id":"String Length","index":48,"name":"String Length","normalized_value":0.43717217445373535,"text":"2.20","unit":"m"},{"id":"Sympathetic Resonance","index":49,"name":"Sympathetic Resonance","normalized_value":0.5,"text":"1.00","unit":""},{"id":"Duplex Scale Resonance","index":50,"name":"Duplex Scale Resonance","normalized_value":0.4999999701976776,"text":"1.00","unit":""},{"id":"Quadratic Effect","index":51,"name":"Quadratic Effect","normalized_value":0.56982421875,"text":"1.00","unit":""},{"id":"Hammer Tine Noise","index":52,"name":"Hammer Tine Noise","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Blooming Energy","index":53,"name":"Blooming Energy","normalized_value":0.0,"text":"0","unit":""},{"id":"Blooming Inertia","index":54,"name":"Blooming Inertia","normalized_value":0.5,"text":"1.00","unit":"s"},{"id":"Equalizer Switch","index":55,"name":"Equalizer Switch","normalized_value":1.0,"text":"On","unit":""},{"id":"Aftertouch","index":56,"name":"Aftertouch","normalized_value":0.0,"text":"0","unit":""},{"id":"Morphing","index":57,"name":"Morphing","normalized_value":0.0,"text":"0","unit":"%"},{"id":"Bounce Switch","index":58,"name":"Bounce Switch","normalized_value":0.0,"text":"Off","unit":""},{"id":"Bounce Delay","index":59,"name":"Bounce Delay","normalized_value":0.4583333432674408,"text":"120.00","unit":"ms"},{"id":"Bounce Sync","index":60,"name":"Bounce Sync","normalized_value":0.0,"text":"Off","unit":""},{"id":"Bounce Sync Speed","index":61,"name":"Bounce Sync Speed","normalized_value":0.5652173757553101,"text":"1","unit":""},{"id":"Bounce Velocity Sensitivity","index":62,"name":"Bounce Velocity Sensitivity","normalized_value":0.5,"text":"50","unit":"%"},{"id":"Bounce Delay Loss","index":63,"name":"Bounce Delay Loss","normalized_value":0.5,"text":"10","unit":"%"},{"id":"Bounce Velocity Loss","index":64,"name":"Bounce Velocity Loss","normalized_value":0.5,"text":"20","unit":"%"},{"id":"Bounce Humanization","index":65,"name":"Bounce Humanization","normalized_value":0.5,"text":"50","unit":"%"},{"id":"Effect[1].Switch","index":66,"name":"Eff 1 Switch","normalized_value":1.0,"text":"On","unit":""},{"id":"Effect[1].Param[1]","index":67,"name":"Eff 1 DelMix","normalized_value":0.15000000596046448,"text":"15","unit":"%"},{"id":"Effect[1].Param[2]","index":68,"name":"Eff 1 DelTime","normalized_value":0.2283361405134201,"text":"60","unit":"ms"},{"id":"Effect[1].Param[3]","index":69,"name":"Eff 1 DelFb","normalized_value":0.0,"text":"0","unit":"%"},{"id":"Effect[1].Param[4]","index":70,"name":"Eff 1 DelTone","normalized_value":0.3499999940395355,"text":"-0.30","unit":""},{"id":"Effect[1].Param[5]","index":71,"name":"Eff 1 DelPol","normalized_value":1.0,"text":"On","unit":""},{"id":"Effect[1].Param[6]","index":72,"name":"Eff 1 Param 6","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[1].Param[7]","index":73,"name":"Eff 1 Param 7","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[1].Param[8]","index":74,"name":"Eff 1 Param 8","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[2].Switch","index":75,"name":"Eff 2 Switch","normalized_value":0.0,"text":"Off","unit":""},{"id":"Effect[2].Param[1]","index":76,"name":"Eff 2 Param 1","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[2].Param[2]","index":77,"name":"Eff 2 Param 2","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[2].Param[3]","index":78,"name":"Eff 2 Param 3","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[2].Param[4]","index":79,"name":"Eff 2 Param 4","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[2].Param[5]","index":80,"name":"Eff 2 Param 5","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[2].Param[6]","index":81,"name":"Eff 2 Param 6","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[2].Param[7]","index":82,"name":"Eff 2 Param 7","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[2].Param[8]","index":83,"name":"Eff 2 Param 8","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[3].Switch","index":84,"name":"Eff 3 Switch","normalized_value":0.0,"text":"Off","unit":""},{"id":"Effect[3].Param[1]","index":85,"name":"Eff 3 Param 1","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[3].Param[2]","index":86,"name":"Eff 3 Param 2","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[3].Param[3]","index":87,"name":"Eff 3 Param 3","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[3].Param[4]","index":88,"name":"Eff 3 Param 4","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[3].Param[5]","index":89,"name":"Eff 3 Param 5","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[3].Param[6]","index":90,"name":"Eff 3 Param 6","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[3].Param[7]","index":91,"name":"Eff 3 Param 7","normalized_value":0.0,"text":"0","unit":""},{"id":"Effect[3].Param[8]","index":92,"name":"Eff 3 Param 8","normalized_value":0.0,"text":"0","unit":""},{"id":"Reverb Switch","index":93,"name":"Reverb Switch","normalized_value":1.0,"text":"On","unit":""},{"id":"Reverb Duration","index":94,"name":"Reverb Duration","normalized_value":0.24997630715370178,"text":"0.80","unit":"s"},{"id":"Reverb Mix","index":95,"name":"Reverb Mix","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Room Dimensions","index":96,"name":"Room Dimensions","normalized_value":0.3333333432674408,"text":"20.00","unit":"m"},{"id":"Reverb Pre-delay","index":97,"name":"Reverb Pre-delay","normalized_value":0.38505247235298157,"text":"0.030","unit":"s"},{"id":"Reverb Early Reflections","index":98,"name":"Reverb Early Reflections","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Reverb Tone","index":99,"name":"Reverb Tone","normalized_value":0.5,"text":"0","unit":""},{"id":"Reverb Envelope","index":100,"name":"Reverb Envelope","normalized_value":1.0,"text":"0","unit":"dB/s"},{"id":"Reverb Resampling","index":101,"name":"Reverb Resampling","normalized_value":0.5,"text":"100","unit":"%"},{"id":"Limiter Switch","index":102,"name":"Limiter Switch","normalized_value":1.0,"text":"On","unit":""},{"id":"Limiter Sharpness","index":103,"name":"Limiter Sharpness","normalized_value":0.5,"text":"1.00","unit":""},{"id":"Limiter Threshold","index":104,"name":"Limiter Threshold","normalized_value":0.5199999809265137,"text":"-12","unit":"dB"},{"id":"Limiter Gain","index":105,"name":"Limiter Gain","normalized_value":0.8333333134651184,"text":"0","unit":"dB"},{"id":"Mute","index":106,"name":"Mute","normalized_value":0.0,"text":"0","unit":""},{"id":"Damper Noise","index":107,"name":"Damper Noise","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Damper Position","index":108,"name":"Damper Position","normalized_value":0.54688960313797,"text":"1/9.6","unit":""},{"id":"Last Damper","index":109,"name":"Last Damper","normalized_value":0.625,"text":"80","unit":""},{"id":"Pedal Noise","index":110,"name":"Pedal Noise","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Key Release Noise","index":111,"name":"Key Release Noise","normalized_value":0.5,"text":"0","unit":"dB"},{"id":"Damping Duration","index":112,"name":"Damping Duration","normalized_value":0.5,"text":"1.00","unit":""},{"id":"Keyboard Range Switch","index":113,"name":"Keyboard Range Switch","normalized_value":0.0,"text":"Off","unit":""},{"id":"Lid Position","index":114,"name":"Lid Position","normalized_value":1.0,"text":"1.00","unit":""},{"id":"Output Mode","index":115,"name":"Output Mode","normalized_value":1.0,"text":"Binaural","unit":""},{"id":"Mic Level Compensation","index":116,"name":"Mic Level Compensation","normalized_value":1.0,"text":"On","unit":""},{"id":"Mic Delay Compensation","index":117,"name":"Mic Delay Compensation","normalized_value":1.0,"text":"On","unit":""},{"id":"Head.X","index":118,"name":"Head X position","normalized_value":0.5379999876022339,"text":"+0.760","unit":"m"},{"id":"Head.Y","index":119,"name":"Head Y position","normalized_value":0.5083333253860474,"text":"+0.100","unit":"m"},{"id":"Head.Z","index":120,"name":"Head Z position","normalized_value":0.4057142436504364,"text":"1.420","unit":"m"},{"id":"Head Diameter","index":121,"name":"Head Diameter","normalized_value":0.20000000298023224,"text":"18.000","unit":"cm"},{"id":"Head Angle","index":122,"name":"Head Angle","normalized_value":0.5,"text":"0","unit":"°"},{"id":"Mic[1].Mic Switch","index":123,"name":"Mic 1 Mic Switch","normalized_value":1.0,"text":"On","unit":""},{"id":"Mic[1].X","index":124,"name":"Mic 1 X position","normalized_value":0.5900000333786011,"text":"+1.800","unit":"m"},{"id":"Mic[1].Y","index":125,"name":"Mic 1 Y position","normalized_value":0.6416666507720947,"text":"+1.700","unit":"m"},{"id":"Mic[1].Z","index":126,"name":"Mic 1 Z position","normalized_value":0.4285714328289032,"text":"1.500","unit":"m"},{"id":"Mic[1].Azimuth","index":127,"name":"Mic 1 Azimuth","normalized_value":0.75,"text":"+90.0","unit":"°"},{"id":"Mic[1].Elevation","index":128,"name":"Mic 1 Elevation","normalized_value":0.5,"text":"0","unit":"°"},{"id":"Mic[1].Level[1]","index":129,"name":"Mic 1 Level 1","normalized_value":0.8374999761581421,"text":"0","unit":"dB"},{"id":"Mic[1].Delay[1]","index":130,"name":"Mic 1 Delay 1","normalized_value":0.5006250143051147,"text":"+0.1","unit":"ms"},{"id":"Mic[1].Level[2]","index":131,"name":"Mic 1 Level 2","normalized_value":0.59375,"text":"-9","unit":"dB"},{"id":"Mic[1].Delay[2]","index":132,"name":"Mic 1 Delay 2","normalized_value":0.6875,"text":"+30.0","unit":"ms"},{"id":"Mic[1].Level[3]","index":133,"name":"Mic 1 Level 3","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[1].Delay[3]","index":134,"name":"Mic 1 Delay 3","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[1].Level[4]","index":135,"name":"Mic 1 Level 4","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[1].Delay[4]","index":136,"name":"Mic 1 Delay 4","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[1].Level[5]","index":137,"name":"Mic 1 Level 5","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[1].Delay[5]","index":138,"name":"Mic 1 Delay 5","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[2].Mic Switch","index":139,"name":"Mic 2 Mic Switch","normalized_value":1.0,"text":"On","unit":""},{"id":"Mic[2].X","index":140,"name":"Mic 2 X position","normalized_value":0.6000000238418579,"text":"+2.000","unit":"m"},{"id":"Mic[2].Y","index":141,"name":"Mic 2 Y position","normalized_value":0.5666666626930237,"text":"+0.800","unit":"m"},{"id":"Mic[2].Z","index":142,"name":"Mic 2 Z position","normalized_value":0.4285714328289032,"text":"1.500","unit":"m"},{"id":"Mic[2].Azimuth","index":143,"name":"Mic 2 Azimuth","normalized_value":0.75,"text":"+90.0","unit":"°"},{"id":"Mic[2].Elevation","index":144,"name":"Mic 2 Elevation","normalized_value":0.5,"text":"0","unit":"°"},{"id":"Mic[2].Level[1]","index":145,"name":"Mic 2 Level 1","normalized_value":0.59375,"text":"-9","unit":"dB"},{"id":"Mic[2].Delay[1]","index":146,"name":"Mic 2 Delay 1","normalized_value":0.6875,"text":"+30.0","unit":"ms"},{"id":"Mic[2].Level[2]","index":147,"name":"Mic 2 Level 2","normalized_value":0.8374999761581421,"text":"0","unit":"dB"},{"id":"Mic[2].Delay[2]","index":148,"name":"Mic 2 Delay 2","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[2].Level[3]","index":149,"name":"Mic 2 Level 3","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[2].Delay[3]","index":150,"name":"Mic 2 Delay 3","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[2].Level[4]","index":151,"name":"Mic 2 Level 4","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[2].Delay[4]","index":152,"name":"Mic 2 Delay 4","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[2].Level[5]","index":153,"name":"Mic 2 Level 5","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[2].Delay[5]","index":154,"name":"Mic 2 Delay 5","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[3].Mic Switch","index":155,"name":"Mic 3 Mic Switch","normalized_value":0.0,"text":"Off","unit":""},{"id":"Mic[3].X","index":156,"name":"Mic 3 X position","normalized_value":0.5960000157356262,"text":"+1.920","unit":"m"},{"id":"Mic[3].Y","index":157,"name":"Mic 3 Y position","normalized_value":0.429999977350235,"text":"-0.840","unit":"m"},{"id":"Mic[3].Z","index":158,"name":"Mic 3 Z position","normalized_value":0.2628571391105652,"text":"0.920","unit":"m"},{"id":"Mic[3].Azimuth","index":159,"name":"Mic 3 Azimuth","normalized_value":0.75,"text":"+90.0","unit":"°"},{"id":"Mic[3].Elevation","index":160,"name":"Mic 3 Elevation","normalized_value":0.5,"text":"0","unit":"°"},{"id":"Mic[3].Level[1]","index":161,"name":"Mic 3 Level 1","normalized_value":0.512499988079071,"text":"-12","unit":"dB"},{"id":"Mic[3].Delay[1]","index":162,"name":"Mic 3 Delay 1","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[3].Level[2]","index":163,"name":"Mic 3 Level 2","normalized_value":0.512499988079071,"text":"-12","unit":"dB"},{"id":"Mic[3].Delay[2]","index":164,"name":"Mic 3 Delay 2","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[3].Level[3]","index":165,"name":"Mic 3 Level 3","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[3].Delay[3]","index":166,"name":"Mic 3 Delay 3","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[3].Level[4]","index":167,"name":"Mic 3 Level 4","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[3].Delay[4]","index":168,"name":"Mic 3 Delay 4","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[3].Level[5]","index":169,"name":"Mic 3 Level 5","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[3].Delay[5]","index":170,"name":"Mic 3 Delay 5","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[4].Mic Switch","index":171,"name":"Mic 4 Mic Switch","normalized_value":0.0,"text":"Off","unit":""},{"id":"Mic[4].X","index":172,"name":"Mic 4 X position","normalized_value":0.4650000035762787,"text":"-0.700","unit":"m"},{"id":"Mic[4].Y","index":173,"name":"Mic 4 Y position","normalized_value":0.5316666960716248,"text":"+0.380","unit":"m"},{"id":"Mic[4].Z","index":174,"name":"Mic 4 Z position","normalized_value":0.2514285743236542,"text":"0.880","unit":"m"},{"id":"Mic[4].Azimuth","index":175,"name":"Mic 4 Azimuth","normalized_value":0.75,"text":"+90.0","unit":"°"},{"id":"Mic[4].Elevation","index":176,"name":"Mic 4 Elevation","normalized_value":0.5,"text":"0","unit":"°"},{"id":"Mic[4].Level[1]","index":177,"name":"Mic 4 Level 1","normalized_value":0.512499988079071,"text":"-12","unit":"dB"},{"id":"Mic[4].Delay[1]","index":178,"name":"Mic 4 Delay 1","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[4].Level[2]","index":179,"name":"Mic 4 Level 2","normalized_value":0.512499988079071,"text":"-12","unit":"dB"},{"id":"Mic[4].Delay[2]","index":180,"name":"Mic 4 Delay 2","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[4].Level[3]","index":181,"name":"Mic 4 Level 3","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[4].Delay[3]","index":182,"name":"Mic 4 Delay 3","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[4].Level[4]","index":183,"name":"Mic 4 Level 4","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[4].Delay[4]","index":184,"name":"Mic 4 Delay 4","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[4].Level[5]","index":185,"name":"Mic 4 Level 5","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[4].Delay[5]","index":186,"name":"Mic 4 Delay 5","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[5].Mic Switch","index":187,"name":"Mic 5 Mic Switch","normalized_value":0.0,"text":"Off","unit":""},{"id":"Mic[5].X","index":188,"name":"Mic 5 X position","normalized_value":0.5770000219345093,"text":"+1.540","unit":"m"},{"id":"Mic[5].Y","index":189,"name":"Mic 5 Y position","normalized_value":0.5699999928474426,"text":"+0.840","unit":"m"},{"id":"Mic[5].Z","index":190,"name":"Mic 5 Z position","normalized_value":0.5028571486473083,"text":"1.760","unit":"m"},{"id":"Mic[5].Azimuth","index":191,"name":"Mic 5 Azimuth","normalized_value":0.75,"text":"+90.0","unit":"°"},{"id":"Mic[5].Elevation","index":192,"name":"Mic 5 Elevation","normalized_value":0.5,"text":"0","unit":"°"},{"id":"Mic[5].Level[1]","index":193,"name":"Mic 5 Level 1","normalized_value":0.512499988079071,"text":"-12","unit":"dB"},{"id":"Mic[5].Delay[1]","index":194,"name":"Mic 5 Delay 1","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[5].Level[2]","index":195,"name":"Mic 5 Level 2","normalized_value":0.512499988079071,"text":"-12","unit":"dB"},{"id":"Mic[5].Delay[2]","index":196,"name":"Mic 5 Delay 2","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[5].Level[3]","index":197,"name":"Mic 5 Level 3","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[5].Delay[3]","index":198,"name":"Mic 5 Delay 3","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[5].Level[4]","index":199,"name":"Mic 5 Level 4","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[5].Delay[4]","index":200,"name":"Mic 5 Delay 4","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Mic[5].Level[5]","index":201,"name":"Mic 5 Level 5","normalized_value":0.0,"text":"Off","unit":"dB"},{"id":"Mic[5].Delay[5]","index":202,"name":"Mic 5 Delay 5","normalized_value":0.5,"text":"0","unit":"ms"},{"id":"Sound Speed","index":203,"name":"Sound Speed","normalized_value":0.5000000596046448,"text":"340","unit":"m/s"},{"id":"Wall distance","index":204,"name":"Wall distance","normalized_value":0.0,"text":"0","unit":"m"},{"id":"Lid","index":205,"name":"Lid","normalized_value":1.0,"text":"On","unit":""}]}

This whole idea never came to my mind so i tried it the quick and dirty way to get an impression.
I managed to track my headmovement to control PTQs head angle parameter with free software only.

I used opentrack to capture my head turning left and right.
Opentrack can output the head movement as joystick movements.

In order to convert the joystick movements into mididata i used Mjoy.
Mjoys midi output is directed to PTQ via loopbe1, a virtual midicable.

Now when i open PTQs microphone window and switch to binaural i can see the little virtual head turning left and right corresponding to my big real life head.

Now i got VR-PTQ for cheapskates.

But the experience is underwhelming nevertheless.

It seems PTQs head angle parameter isnt meant to be adjusted while playing.
The little hourglass next to the preset name is constantly flashing with every head movement so there's some processing going on.
This results in a rather glitchy and wobbly sound especially with fast head movements.

The VR concept is a nice idea and a fun experiment anyways.

Here i did a poor schematic:

Very good Zaskar, I will try it..
I would like use my NX tracker (witch has a good response ) but this solution seems interesting.
I think it is normal that the little hourglass was flashing during movement... it flash everytime we change any parameter. But it is important not hear that...

It sounds perfect for me, no sound interruption nor flashes in the GUI. Both when using the mouse and when using the JSON interface.

This Pianoteq Pro v 7.5.4 for Linux under Ubuntu 22.04 running on a powerful and reliable, but quite old Intel(R) Pentium(R) CPU G3220 @ 3.00GHz with a performance index of 80.

@dv @YvesTh

Ok, then i'm plain wrong.
Maybe i fiddled so long that my brain tricks me in hearing what i expect to hear.

Yes, i limited the head angle as well (-70 to +70).
Yes, the octopus is hard to tame. This is probably the hardest part. Took me half the afternoon.

The thing is, if you manage the octopus, you can even make use of PTQs X,Y and Z coordinates.
This would allow the player to lean sideways, move back and forth and to stand up!

Oh, i forgot to mention that there's a tiny flaw with my makeshift solution.
As i mentioned the tracked data is converted to midi (0-127).
This is a relatively low resolution for delicate head movements. So the head doesn't turn smoothly.
In my case it turns in increments of approx. 1.1 degrees. (140 degrees "field of view" / 128 ≈ 1.1)

Maybe the JSON guys get finer inputs.

OK I made a PoC using face tracking via webcam: https://sysd.org/pianoteq-pov/
Works in Chrome & Firefox but not Safari (which refuses to call the JSON-RPC interface on localhost due to security concerns).
FWIW, the source code is here.

Hi Zaskar,
Thank you very much for your help. I am currently not at home
and I won't be able to test your method for a couple of weeks.
I will try as soon as I can.

You tried with the PoC created by creaktive ?
I won't because I am in holidays... I'm impatient to do it...

https://www.soundonsound.com/reviews/genelec-aural-id

Not sure if this relates?

Warmest regards,

Chris

Hi créative,
I just came back from holidays and tried your PoC. By clicking on your link I get the head tracking by webcam, very well. But how to make the link with pianoteq? Do I have to open it first? After?  And how? I know absolutely nothing about json, do I have to install something? Thanks for your help...

Thank very much dv,
It works... very funny sensation...
I have a latency of the movement when I move my head a little too fast, but my PC is not very powerfull...
And thanks to creaktive for his excellent work...
Do you think it is possible to use a "nx headtracker bluetooth" to send the head movement signal? It seems to be more efficient than by webcam.