Are you using "windows exclusive audio" or "asio4all"? If not, that could be the reason.
If you only use the PC only for Pianoteq, you could try Ubuntu Studio or another audio oriented Linux Distro like AV Linux, many times they are a bit more efficient than ubuntu but not as easy to install.
(For Ubuntu you just need to click "next" several times).

I've figured it out by now. Lowering the buffer in the ASIO control panel did the trick. For some reason it was set extremely high (way above the blue area) for all buffer sizes, and I didn't notice.

Thanks for the post though, reading "asio4all" made me enter the control panel to double check if everything was right.

I will try it with a Topping DX1 instead of the dongle and see if it solves the issue. Help with making it work with the Apple dongle would still be appreciated because I would like to sometimes bring that instead of the Topping.

Pianoteq 8 works perfectly fine when I use headphones and my laptop's Realtek HD Audio on-board sound through the headphone jack, and a Kawaii stage piano connected by MIDI-to-USB as the input. There is no delay our unwanted noise.

However, I would like to use an Apple dongle since the Realtek sound isn't exceptionally good. The laptop doesn't have a USB-C port, so I have to plug the Apple dongle into a USB-A-to-C adapter, and then plug the headphones into the dongle. There are absolutely no issues with delay or sound quality when doing anything but using Pianoteq. When I use Pianoteq, there is noticeable delay between the sound and the key press on the keyboard, and a fairly loud crackling sound. Neither of this happens when I click the keys in the Pianoteq software.
Changing buffer size has some effect, but the values don't reflect the delay. When I use Realtek as my audio device and it shows 4.4 ms delay, it is not perceivable while playing. When I use the same settings with the USB DAC as my output it is noticeably delayed.

To sum it up, the issue is neither the connection between keyboard and laptop, nor the USB port or the Apple dongle. All of these things work fine independently. No delay, no crackling. ONLY when I use Pianoteq 8 and set my audio output to USB I run into these issues.
I'm running the latest version of Windows 10 Home.

Does anyone have an idea why this happens, and how to solve it?

Thanks a lot!