Baw Nam Horn Noi is small and not worth a visit.
The other is Baw Nam Horn Nyai, further away passing (approximately 50km from Phonsavan) along a road through attractive scenery, small villages growing chillis and mountains and hills covered with rice fields. Around the spring has sprung up a small town of sorts comprised of a restaurant, guesthouse, a few houses and the spring itself.
The actual hot spring is off limits to bathers. It is government-owned property and the water is piped out for use by the guesthouse. You can go and have a look by following the small path that starts across from the resort's front entrance. The path ends at the blue coloured hot spring which is surrounded by hedge to keep people from entering. A path leads to a smaller hot spring from the main lake. It's a 265m walk through the jungle along an overgrown path which is simply a bunch of concrete blocks which are inconveniently spaced. Once at the smaller spring, you'll realise you just wasted 10 minutes of your life as it is merely a small swamp in the jungle with absolute no redeeming qualities. There's not even a possibility of a nice photo. Nothing.
None of the bathing options here involve immersing yourself in nature. The water in the guesthouse is piped directly from the spring and bathrooms are outfitted with baths. If you don't plan to stay the night, the resort also has bathing rooms. A simple bathroom has two baths and an attached toilet and costs 5,000 kip per person. It's a popular spot for the people of Phonsavan to come during holidays and during the winter months. Entry to the hot spring area costs 2,000 kip.
The hot springs are an entirely local experience and if you have no inclination to experience it like the locals do, it's simply not worth the effort to come here. But if you're interested in how domestic tourists get their kicks, come and try it out for yourself.
Getting to the hot springs by public transport is impractical due to the distance between the main road where songtaews pass and the hot springs. If you want to give it a go, songthaews depart from the centre of Muang Kham when full, just up the road to Sam Neua from the main intersection. The cost will be 10,000 kip to the hot spring turn off. The return journey is more difficult due to a lack of songthaews, but the odd one will pass by up until about 4pm.
The other option is to charter a songthaew to make the journey and wait an hour or so for you. A fair price is 100,000 kip, although if you want to combine it with a trip to Tham Piew, you might be able to get the lot for 120,000 kip. There will be some variation to this price dependent on prospective passenger numbers. If you depart early in the morning when most of the locals are travelling, there is a good chance you will get a discount.