Can you rely on HRMs?

Most people seem to think that MFP either massively over or underestimates what they are burning.

The touted solution is always that you MUST have an HRM

But I've also seen posts that say an HRM is only designed to be accurate when the heart rate is consistently raised. Which in many exercise classes it won't always be (breaks between tracks, conditioning tracks etc). I've even seen people say they use HRMs for yoga and walking which won't raise your heart rate at all.

I can't remember what the post that worried me was about exactly but it was something to do with a lady who was wearing hers all day to calculate her daily burn and was told that no, it wouldn't work because her heart rate wasn't up all day. It just made me think 'eek, in that case, unless you are running or on a machine or lifting heavy weights, then you could be hugely overestimating a burn using an HRM too.'

I'm asking because I was considering buying one but that really put me off and I was relying on MFP and other websites and estimating on the low side.

I kind of want to experiment with eating my exercise calories but I just feel like there is no way of knowing and I'm not going to risk overeating. It seems safer to just know you've had a work out and burned 'some' calories without relying on a machine to let you know how much extra food you can eat.

Does anyone know a surefire way to know exactly what they've burned and eat their exercise calories accurately?