Visual short-term memory – the ability to hold information in mind over a short period – declines with age. We show that connections within the visual and attention networks increase as people hold more information in mind, but age negatively affects the ability to increase connectivity as the task gets harder.