Interesting, but of course the model completely ignores (or corrects for) the effect of the sustain pedal so the piano renderings are a bit dry, let's say. Still, this can be useful as a way of getting an automatic score from the generated MIDI file. The Stevie Wonder is amusing because it would be the sort of thing obtained if a correct MIDI file was rendered while not following the general MIDI conventions for percussions...
yes, the model has no idea how the piece is being played. it just analyzes the waveform and tries to extract the events by their attacks and decays. definitely using it to produce a viable performance with pianoteq is suboptimal. but i just use it to get an instant "masterclass." it really seems like the player has come to life in front of you.
having access to the midi file makes it much easier to transcribe the piece too. unfortunately generating a score from a midi file still seems beyond the capabilities of machine learning, although there have been some better efforts lately. the midi file itself, together with the original performance, can be really useful.
here's another robot performance with pianoteq, but this time of a much larger ensemble
]]>https://magenta.tensorflow.org/onsets-frames
one can take a solo piano performance and produce a midi file. feed the midi file into Pianoteq, and it's an instant masterclass! it's as if you're standing behind the master and watching him/her play. here's an example, done in honor of the passing of Chick Corea, one of the all time greats. it's a solo performance of his famous tune Spain, from his album Solo Piano: Originals:
https://www.youtube.com/watch?v=o4N9SYhydMI
feeding other tracks into the neural network sometimes produces amusing results. the robots aren't always sure what to do with drums/bass/horns/guitars, since they interpret every waveform as having been generated by a piano. for instance here's Stevie Wonder's Superstition
https://hearthis.at/budosaurus/robotsst/
and here's Chasin the Trane from John Coltrane's Village Vanguard sessions
]]>