Lots of hotels don't allow them though so you will need to eat them outside. I prefer mangosteens, purple dragonfruit and rambutans.
#3 LeonardCohen1 has been a member since 24/7/2012. Posts: 2,148
I can't answer your question about Pha Ngan / Samui, but if you're really serious about durian (and judging by your screen name, you are), you should go to Rayong / Chanthaburi provinces in eastern Thailand between May and August. Some of the best durian in the world comes from these provinces, and there are tons of farms (though don't ask me to specify any). There's also an annual durian festival in Chanthaburi in May. Very good durian also comes from Ang Thong province. In Bangkok, the best place for durian is Or Tor Kor market. Farms from Chanthaburi and Rayong have their own stands there. Here's some more info: http://www.travelfish.org/blogs/thailand/2012/06/21/durian-king-of-thai-fruits/
I do like Chanthaburi durians but I wouldn't say there is much special about them except you can get more (and better) varieties than in some other places.
Check out my pictures from 2013 Chanthaburi Durian Festival: http://www.yearofthedurian.com/2013/06/chanthaburi-durian-festival-in-rain.html
I used to work for the Thai Dept. of Agricultural Extension in Prachinburi, and our office worked extensively with organic durian growers in that province. The durian they produced were fantastic! But they weren't as good as some of the non-commercially grown varieties that came from Nonthaburi. There's a farm there that has all species of durian that grow in the country. It's a last bastion for many of the species that have disappeared due to floods, land-development, etc. I can't remember the name of the farm but can probably dig it up if you're really interested.