Firstly: walking! The Old Quarter itself is reasonably small and it makes more sense to walk as then it’s easier to pop into shops, stop for a drink, take photos and generally observe. From the Old Quarter it’s not too far to the French Quarter and although the the Ho Chi Minh Memorial Complex is a bit of walk, it’s still feasible — although you might not fancy it in the height of summer.
The main disadvantage of walking — apart from the heat — is negotiating the roads and even the pavements, which are usually blocked by vendors and motorbikes. But once you’ve got the hang of that, in my opinion, it’s the best way to see the sights.
Unfortunately, although Hanoi’s bus system is reasonably reliable, very cheap — 3,000 for most trips around town — and extensive, it doesn’t really help if you just want to get to the main central sights. A few buses run from Old Quarter to the Memorial Complex and Botanical Gardens, although you’ll still have a short walk, which may negate the wait for the bus. The No 2 runs from the top of Hoan Kiem Lake to the flag tower on Dien Bien Phu, catch the No 9 from Trang Thi, at the southwest end of the lake, and get off on Le Hong Phong or take the No 18 from Phung Huong and get off at the Botanical Gardens.
If you’re heading further afield then buses are a very economical way to travel:
See online map for more bus route details.
Do be careful of your belongings: I never had any problems when I was travelling by bus — I don’t use them now I have a bike — but whenever I told Vietnamese people that I used the bus they thought I was crazy and warned me about pickpockets. Over cautious, maybe, but best to be safe.