Question on Swapping Between Meat and Veggies.

Quieas

So I was debating!

What's if one week I eat solely veggies and fruits, with an egg or two for breakfast and lunch. I'd more than likely add yogurt to that. And then, the next week, go back to eating meat with my dinner/lunch but still eat veggies and fruit.

Could this potentially cause my body to freak out and not loose weight?

Thank you.