As the old name suggests, Wat Ao Noi is a cave temple -- well, a temple with a cave nearby -- and is worth a visit by those with a lot of time on their hands, both for the cave and also a wat building made of sea shells.
The monks have constructed themselves a new and rather gaudy wat down by the road, but the shell building, while rundown, is novel -- even the nagas lining the stairs are covered in shells. About 500 metres to the left (when facing the shell temple) along a partly collapsing staircase lies Khan Kadai cave.
Unfortunately when we visited the lights box had fused so there wasn't very much lighting to be had -- we made do with our camera flash and phone light. Two reclining Buddhas are within the caves, but note you can continue after the second (stick to the right) and there's at least two more chambers and a TONNE of bats -- we gave up at that point, but those with proper torches (or when the monks call an electrician) may be able to explore more.
The Travelfish newsletter is sent out every Monday and is jammed full of free advice for travel in Southeast Asia. You can see past issues here.