Yup. I'd fly directly from HCMC to Siem Reap and then fly directly from Siem Reap on to Bangkok. If all you want to see is Angkor Wat, then there isn't any reason to fly to or through Phnom Penh. Cheers.
completely agree with the above.
Small note: a lot of flights also make a stop at Phnom Penh (before or after landing in Siem Reap) so there could be an option to see something of Phnom Penh (since the place got mentioned).