Always carry a torch. The +25 Warmth is enormous. You can still cast spells with your right hand. Drop the torch if you need a two-handed weapon or dual casting, but pick it back up when moving.
You don’t need a matching set. Wear a fur chest, leather boots, hide bracers, and a Skaal helmet if you have it. Mix for max warmth. skyrim survival mode armor warmth ratings