It could be intolerance to me with the stomach symptoms. Like you have a problem digesting the dairy. But the breakouts could be a hormonal response to the histamines created if your body is having an allergic reaction. Either way the only way to know is to have a blood test. I had to do the same thing a few years ago.

I was ok with dairy as a kid but in college I started to notice congestion and tightness in my throat when I had really rich dairy foods. I always had acne so i didn't put the two together. My throat symptoms continued to escalate but I only noticed in retrospect. I did not figure it out until not until one day when I was given a late with whole milk and cream and I lost my voice for hours due to the inflammation in my throat. I realized it had to be the milk and went to get a blood test. The test said I had a low algery to Milk, well Casein, the protein in milk. The doctor said it was nothing to be worried about, that on a scale of 1 to 10 I was only a 1-2. He sent me away with no solutions. Just to monitor my reactions to dairy and come back if they seem to be getting worse in about 6 months.

I went home and did tons of research and discovered that I had a lot more symptoms that I was just dismissing as nothing. I was determined to try cutting out dairy completely. I did so for a few months and like magic. My skin cleared, my congestion when eating disappeared, my digestion was running like clockwork. I stayed away for a couple years in full. Then I started cheating here and there and noticed that my old symptoms were much less than I remembered them and thought that maybe I was growing out of my allergy. I started to relax more and more on the dairy and with out noticing it I was right back to where I was before. except Now I was in my late 20s with the acne of a 15 year old.

For me its obvious that my body could deal the dairy when it had too in small quantities but as soon as I was a semi regular part of my diet again. The effects started to build up until all my body could not fight it off. Whether that is an intolerance or an allergy, I don't care anymore. all I know is that I feel better when I do not eat foods with dairy (Casien). I should have listened to my body. I did not need a doctor to tell me that the milk products were making me look and feel terrible.

I hope my experience can help you decide what direction you want to go. I'm available if you ever want to talk about it more. Good Luck on your journey.