Completed in 1860, the palace is named Phra Nakhon Khiri, but is commonly referred to as Khao Wang.
Although very occasionally still used by members of the Thai royal family, today it's mostly open to the public as a museum and home for thousands of vicious monkeys that totally overrun the entire site.
There are three areas of interest on the hilltops. On the southern hill is Wat Maha Samanaram and Wat Phra Kaeo, which bears a resemblance to the Temple of the Emerald Buddha in Bangkok. On the centre hill, there is a chedi, Phra That Cham Phet, from where there is a great view of Phetchaburi, and on the northern hill sits the Royal Residence. The Royal Residence forms the centrepiece of the museum. Of particular interest is the observation tower, which was built to allow the king to study the stars. A number of different paths connect the sites and the best way to experience the site really is to just wander from hilltop to hilltop. Drinks are available.
A word on the monkeys -- the entire hilltop is owned by these beasts and they can be extremely aggressive. Keep all food hidden and do not offer them anything to eat unless you like having these beasts jump and climb all over you. Going by the sheer number of signs, people must get bitten and or attacked regularly -- we were attacked twice on our most recent visit.
How to get there
To get here from town, walk down Bandai It Road and take the right past Wat Chang. The walk up the hill is moderately strenuous, so a cable car has been installed to whisk you up. If you want to get the cable car, you may want to get a samlor to take you to the base, as the cable car runs up the far side of the hill.