Strength training - where do I even start?

I don't know about workout videos but if I wanted to add some strength training without use of a gym and limited equipment, you'll likely need to adapt a workout or come up with your own. Here are my thoughts...

If you want to focus on upper body, then you've chest, back and shoulders to target.

I'd guess your hand weights (dumbells?) are pretty light, so won't be enough to stimulate the chest or upper back in the main pressing or pulling movements.

I'd go for pushups to work the chest, reverse pushups (face up and pull yourself up onto a bar or table edge, not like a chin up as your body should be as horizontal as possible) for upper back. You can then supplement chest and back with fly exercises using your weights. The lower back can be targeted with good mornings or hyperextensions.

For shoulders, you could do shoulder press and lateral raises. For a complete workout, don't ignore the legs or core either.

If you unsure about exercises, look them up, should be lots of example videos on youtube.

For frequency, every second or third day would be enough, to give your body time to rest inbetween.

Check out the book New Rules of Lifting for Women if you want to get serious about lifting weights. As far as videos go, the ones out there are mostly cardio-ish, like Jillian Michaels. You're not going to gain the benefits of lifting weights through videos, you need to use dumbbells and barbells and actual weights.

I'm doing NROL4W myself and love it, but if you don't have access to a gym it won't do you much good. A great place to start is with body weight workouts. This one is awesome:

http://www.nerdfitness.com/blog/2009/12/09/beginner-body-weight-workout-burn-fat-build-muscle/

And just to inspire you to get lifting:

http://www.nerdfitness.com/blog/2011/07/21/meet-staci-your-new-powerlifting-super-hero/

Look up Zuzana Light on you tube - she incorporates a lot of calisthenics into her workouts: I am switching to working on using my own body weight more in my work outs after many years of weights on the advice of my Muay Thai trainer...
Loads of you tube but push ups, squat thrusts, burpees, mountain climbers are all some basic brilliant strength training exercises.

X

I think Jillian Michaels 30 Day Shred is a good start! Free on youtube and all you need are hand weights and a mat or carpet. Search for the results here.