Should I “eat” the calories I exercise off?

If you are targeting a high weight loss rate (like 2 pounds a week, or possibly even less depending on your current weight), and regularly doing a large amount of exercise (many hundreds of calories daily), then definitely eat at least 50% of them back. To do otherwise could cause an unhealthfully rapid weight loss, which is a health risk - not worth it.

If your target weight loss rate is much slower than the healthy maximum for your current weight, and your exercise is not very high calorie burn (100-200, say) and not daily, then it should be safe to add the exercise to your calorie deficit to lose a little faster (because your calorie deficit is not extreme to start with).

In between those extremes, the best rule of thumb is to set a sensible weight loss rate target (no more than 1% of body weight per week, and less than that if below 50 pounds total to lose; slower is always fine). Then track your exercise, and eat back at least 50% of those calories to start, in addition to eating (nearly) all of your base calories. (If you're very confident that your exercise calorie estimate is reasonably accurate, you can eat back more than 50%).

Stick with this for 4-6 weeks (one full menstrual cycle plus a little), then look at your average loss per week. If it's in line with your target rate, and you feel good, stick with it. If you're losing faster than target, eat more. If you're losing slower than target, you can eat a little less if you wish. If you feel fatigued or weak (or have other negative symptoms) in that 4-6 weeks, eat a little more. Otherwise, stick with it. You may see the scale move up and down a bit; that's normal water weight fluctuation. After 4-6 weeks, you should have enough data to reasonably estimate your loss rate.

We have a tendency to want to hit it hard at first, to lose weight fast. But for most of us, this is a longer-term proposition, so sustainability is important. It should seem very doable, and we should feel energetic and strong while it's going on. That's what helps us keep going, and making progress!

Best wishes!