Choosing layers for big temperature swings between afternoon and dawn

On our last trip it was warm enough for a T-shirt while setting up camp, then cold enough before sunrise that I struggled to get out of the sleeping bag. I brought one heavy jacket and spent the evening either overheating or shivering. Next time I'm thinking of a base layer, fleece, light wind shell and dry sleep clothes instead. For that kind of big day-night swing, which layer do you consider non-negotiable, and what do you leave in the bag until camp?