Working out at home - shoes or no shoes?

My husband just went downstairs to lift - in his slippers! He usually wears Chuck T's.

Me: running shoes most of the time, but I am trying my five fingers for lifting days since I usually warm up on the elliptical (no running), and barefoot lifting (with a little bit of extra grip from the Vibrams) is a lot easier than trying to balance in my running shoes.