The Fishing Village on Perhentian Kecil is not exactly the 'cultural experience' that one might expect, but for those looking for an insight into how the non-tourist part of the island works, it may be worth a visit.
Over the years the village has grown considerably in size and at the time of our visit in April 2014, a new village to accommodate the growing population was being carved out of the forest on the north side of the new mosque on Perhentian Kecil. The existing village (to the south of the mosque) is a typical Malaysian fishing village (if far more affluent than your average fishing village) with a large pier, school, mosque and warungs. The food in the few restaurants here is marginally cheaper than what you'll get on the main beaches, but not to the point that it's worth getting a boat here to save money!
You're welcome to walk around whatever parts of the village you want to, but please remember you're peering into people's personal lives and homes. It goes without saying that you should always ask before taking a photo and conservative clothing would be appreciated -- don't go strolling through in your speedos. There is a small beach at the southern end of the village, but it isn't a scratch on the other beaches of the island.
From the village a trail runs to the south around to Petani beach (and eventually Coral beach) and to the north to Long beach. See our trekking on the Perhentians section for more details.
There is no village of any size on Perhentian Besar. Those who work on Besar commute from the village on Kecil (unless they're living in staff housing, of course).
The Travelfish newsletter is sent out every Monday and is jammed full of free advice for travel in Southeast Asia. You can see past issues here.