If you are having choppy, glitchy or stuttering playback using hardware transcoding on the AS5304T/AS5202T (Nimbustor 4/Nimbustor 2) models here is a fix. It may work for other models, I don't know if they are affected by this bug or not. A Plex update around Dec. 2019 caused incompatibility issues with the Intel HD driver.
SOLUTION:
SSH into your console, login as admin and then type:
Code: Select all
cd /volume1/.@plugins/AppCentral/plexmediaserver/lib/dri
Code: Select all
mv iHD_drv_video.so iHD_drv_video.bak
This will rename the Intel HD driver that is causing the problem. Plex can still use the hardware transcoding capability using VAAPI after you do this. Then you can enable the "Use hardware-accelerated video encoding" setting and it will work the way it is supposed to.
Note: You will have to repeat this process every time you upgrade the Plex version.
UPDATE (11/17/20): For me this appears to have been fixed as of version 1.20.5.3600-47c0d9038