Sugar

So lately I've been trying to eat more fruit.
I usually have a banana in the morning, some carrots a little after that, two orange wedges and some apple slices after that, and then at some point later in the day I'll usually have one of those little cutie oranges or a banana.
It's good for me cause it helps me uhhh...well -shrug- poop (for lack of a better word) which I usually have trouble doing, and it really helps me keep my sodium down.
Obviously I eat more than just those fruits and some of the other things I eat contain sugar, I'm just wondering if I should be as worried about going over on sugar if more than half of it is the sugar from fruits.