Should I rest or push it?

First off, are you having actual pain or is it muscle soreness? It's entirely possible that you can't answer that but there is a difference. If you are having any sharp pains, see a doctor. If it's more like muscle soreness, take an extra day before you lift again but do a light activity like walking.

What program are you following with your lifting? I started off completely overweight and out of shape (super sedentary) and I was so completely miserably sore after my first bodyweight workout. I learned to really scale back what my program called for. You may need to do that and trust me, it is okay. It is far better to start out slow and increase intensity with baby steps than it is to go all out, hurt yourself, and risk falling off the exercise wagon all together.

Good job getting started!

Definitely stretch it out. Drink plenty of water. As far as whether or not you should take a rest day, that is up to you. For me, I usually still go (my gym schedule is 4 days a week). If it is just your legs that are sore, you could do an upper body day too. Rest days are important and beneficial to muscle repair.

It's normal to feel a certain amount of discomfort when embarking on a new fitness regimen especially when starting from scratch.

It may sound counter-intuitive but one of the best ways to counteract DOMS is to keep moving (something low impact and not too strenous like a good walk, swimming, an easy bike ride). A warm bath with Epsom salts can help (in extreme cases and ice bath may be in order too......I've had a few of those after really hard runs, they're not pleasant but they can help) you may also want to take a NSAID such as advil to relieve some of the inflammation.

The best way to avoid soreness in the future is to not overdo it. It sounds like you may have bitten off more than you can chew with your first workout, I'm going to guess that you may have just winged it?

Find a program that fits your needs, I'm a big fan of stronglifts 5 x 5 as a beginning strength program. It's structured, progressive and doesn't take a huge time commitment. It's intended for barbells (you start with an empty Olympic bar which is 45 lbs) but you can do all of the exercises with lighter dumbbells if that's too heavy to start.

Feel better!

I found when I started with weights I hurt so bad I could barely walk, but I found that gentle exercise on the day after a weights session helps to loosen me up. I would also ensure if I am doing leg exercises I dont do it the day before I am due a run, I plan a swim to help my legs recover. I will do a run the day after I do arms to allow them to recover etc

If you are new I would say aim for 3 times a week to do weights, and rest, so maybe Mon, Weds and Fri do weights at the gym and then on your other days do something like cycling, swimming or running and ensure you have one or two days a week when you have rest from exercise, rest days are equally as important as working out as its during rest that you build muscle

My advice would be full body workouts and rest until you are no longer sore. As a beginner this will mean 2-3 workouts per week max.

Pretty much just chiming in with everyone else. If you are sure it's just "sore" rather than an injury, maybe break for a day or two tops, and just go a bit lighter until you are more comfortable. But if it's an actual injury, stop working that area and let ot heal, to avoid permanent damage.
If you aren't 100% positive on which it is, take a day off to rest ice and heat, and see if it feels better, centralizes to a more specific spot (usually a good indication of injury) or just fades a bit.