How long does it take for that annoying little voice that says “give it a miss” or “do it later” to go away?

In my experience, it doesn't ever really go away. You just have to learn to not listen to it. Something that helps me is to tell yourself that if you complete this task, you'll give yourself a reward immediately after. For example the task could be exercising, reading, doing work, and the reward might be a snack or watching an episode of your favorite show. If it's a long task, divide it into 30min to 1hr sections and take 5-10 min breaks.