If you want to do it direct, without stopping, quickest route is along the coast -- Phnom Penh to Ko Kong (by bus or share taxi) then cross the border (motorbike) then minibus from Had Lek to Trat, then songtheaw to Laem Ngop then ferry to the island.
You'll probably be able to find travel agents in Phnom Penh that can sort you out a through ticket, or if that fails, I think you can get a minibus/boat combo from Hat Lek to Ko Chang.
Either way, assuming you get good connections and leave Phnom Penh early, you can do it in a day.
#2 somtam2000 has been a member since 21/1/2004. Location: Indonesia. Posts: 7,738
Send somtam2000 a private message Where has somtam2000 been? Website Twitter Facebook Flickr Google+ Instagram Pinterest