Deciding how much unstructured time to leave in a first visit

We're taking a first trip to a city packed with sights. My instinct is to schedule morning, afternoon and evening slots so we don't miss anything. My partner prefers discovering places as we walk. I'm trying a compromise: one timed booking per day, one optional neighborhood and the rest unscheduled. How much free time would you leave on a short first visit before it feels like you've underplanned?