Exercise Deficit?

is there a deficit i should have? for instance, if i burn 500 calories doing an exercise, i should not eat 500 extra calories, right? so, at the end of the day, if i don't eat all of my calories, is there a limit as of what is too much leftover? i hope this makes sense!