Should I up my calorie limit?

I'm thinking about upping my calorie limit to 1500 instead of 1200. My new fitness routine is 3 days of cardio a week and 3 days of weights a week. What do you guys think? I'm worried about eating too few cals and messing up my metabolism - is that a legit concern, or just me being paranoid?