At the time of writing, the only day trips of the reservoir available departed from Hanoi although Hoa Binh Tourism claims "all tours on demand are available", so see how you go.
Wherever you book, it's likely to be a private tour and include visits to some villages. The scenery is beautiful so it's well worth a trip if you can arrange it.
The Travelfish newsletter is sent out every Monday and is jammed full of free advice for travel in Southeast Asia. You can see past issues here.