You should be able to do 'a lot better' than 4s/day just by setting the proper chip load-capacitors. See my discussion:
Re: RTC accuracy with MQX 4.1 on a K70
But as I say there, your 'worst case parts' will still only get you about 2s/day at room temperature with 'inexpensive' crystals -- a nice oscillator can hold 1s/day. The APNOTE mentioned above helps you improve a crystal over temperature, BUT that requires the CPU to be 'awake', at least periodically, to be able to MEASURE said temperature and 'do something with it' -- the SiTime part 'does that for you'.