Visual working memory (VWM) is a central cognitive system used to compare views of the world and detect changes in the local environment. This system undergoes dramatic development in the first two years; however, we know relatively little about the functional organization of VWM at the level of the brain. Here, we used image-based functional near-infrared spectroscopy (fNIRS) to test four hypotheses about the spatial organization of the VWM network in early development. Four-month-olds, 1-year-olds, and 2-year-olds completed a VWM task while we recorded neural activity from 19 cortical regions-of-interest identified from a meta-analysis of the adult fMRI literature on VWM. Results showed significant task-specific functional activation near 6 of 19 ROIs, revealing spatial consistency in the brain regions activated in our study and brain regions identified to be part of the VWM network in adult fMRI studies. Working memory related activation was centered on bilateral anterior intraparietal sulcus (aIPS), left temporoparietal junction (TPJ), and left ventral occipital complex (VOC), while visual exploratory measures were associated with activation in right dorsolateral prefrontal cortex, left TPJ, and bilateral IPS. Results show that a distributed brain network underlies functional changes in VWM in infancy, revealing new insights into the neural mechanisms that support infants’ improved ability to remember visual information and to detect changes in an on-going visual stream.