Try the one in District Five called Lam Son, at Tran Binh Trong — it's open 08:00-20:00 and costs 5,000 VND an hour.
A more centrally located pool is at 258 Tran Hung Dao in District One, where entry will cost 80,000 VND and allow you to swim all day.
For a cheaper downtown pool try the Workers Club at 55B Nugyen Thi Minh Khai St, District Three (T: (08) 3930 1819) for 10,000 VND per hour.
Another option is to make use of the swimming pools at the hotels around town. The Rex Hotel has a nice rooftop pool — although it's rather small, it's perfect for a lazy afternoon. It's open daily 06:30-21:30 and costs 175,000 VND. Other hotels worth trying include the Omni, Sofitel and Metropole.