Here's how long you'll need a jacket as temperatures cool down