If you look carefully from the beach road you'll spy a small Buddhist sala and Thai flag at the summit -- yes you can climb to the top! You'll need to have a fair degree of fitness and be good with heights as the climb up is quite strenuous while on the climb down you'll almost be abseiling down a couple of four to six metre near-vertical rock faces.
There are ropes when needed, but a couple really need to be replaced, so take care. Also wear decent shoes -- we wouldn't recommend doing this in flipflops.
The climb takes about an hour each way and begins from behind the historical park within the airforce base, where there is a family of gorgeous dusty langurs (a striking contrast to the feral beasts at Khao Chong Krachok). It starts with stairs and the trail is clear the whole way up. It is difficult in parts, so take care.
At the summit, ring the temple bell, take in the amazing views and watch the planes coming and going at the base. Legend has it the Japanese hid gold in caves within this peak, so keep your eye out for glittery stuff.