Have you got a long sleeved base layer? I usually wear that, either a short sleeved top or vest, coat, leggings, beanie and some gloves. Make sure you get sweat wicking clothing as you wont feel so soaked. This way I can remove layers if I get too cold. I am always cold when i first start but by the end of the run I usually want to take my coat off as I am too hot.

Just got back from light snow in the air and on the ground and just below zero with the wind. Agree with what all have said so far. Two other tips: I use a balm from a ski shop on my cheeks, nose, etc. and this helps. And I also do layers on my hands. At today's temps it is wind mitts over my gloves

Yep

Technical baselayer, breathable outer layers, full on toque, or if it's cold enough a baclava, and gloves.

I find I can wear cotton gloves til it gets pretty darn cold easy to take off if it's not too cold and you start to heat up.

that all depends on the degree of cold layers are the all important factor my habitual upgrades are usually:

0C to -10C: Running leggings, long sleeve wicking shirt, running jacket (but not necessarily my winter one) thin running gloves and headband.

-11C to -20C: Thermal running leggings, long sleeve wicking shirt, winter running jacket, running beanie, running gloves (with optional extra pair of mittens to start the run that I then tuck into my water belt when hands get too hot)

-21C to -35C: merino wool long johns, thermal running leggings, long sleeve wicking shirt, tank top for extra core warmth, wind proof winter running jacket (thin), fleecy running jacket, running beanie, face mask, running gloves, extra mittens, possibly also carrying chemical hand warmers.

below -35C: treadmill at the gym.

Been running in as low as "feels like" 2 degrees (with wind chill.)

Here is what I wear from toe to head (not counting shoes!):
compression socks
cw-x insulator running tights
mittens
sports bra
underarmour heatgear shirt
smart wool 3/4 zip
north face flash dry blaze jacket (currently on sale!)
neck gaiter (pulls up over my mouth and nose)
Aerial7 sound disc beanie

Last Monday I ran 22mi in freezing rain, which was an experience! I did not wear the neck gaiter and substituted the blaze jacket for the north face sonora jacket. I bought it on super duper sale last year and I can't say enough great things about it in the rain. It is a goretex pro shell and is like running in a breathable tent. Pit zips extend from just above the elbow to about 8 inches below the armpit. The hood has a visor and fit great over my wicking hat. It was a little tight over my hydration vest (Salomon Advanced Skin S-Lab 5 Set Hydration Pack- say that 5 times fast!) but not restrictive, and the shell has a slight stretch to it, so once I zipped it up, I didn't notice it again. If you plan to run in rain/freezing rain and can get your hands on a Gore Tex Pro Shell, there's nothing better. (I think this year's version is called the "zero" jacket.

Ok, so these options are not inexpensive, but I would try them on in stores and then look on eBay or 6pm.com (the zappos discount site.) Bear in mind that if you buy something you can wear for years, the cost per wear drops considerably....

Edit: North face products carry a lifetime warranty, so though they are expensive at first, they will truly last a lifetime!