Increased ambient heat exposure poses a health risk to pregnant women, which may be amplified by environmental and social determinants, but these interactions have been insufficiently characterized. We examined critical windows for the associations between heat exposure during pregnancy and fetal growth and investigated the role of air pollution, vegetation, and social stressors in these associations. Weekly exposure to ambient temperature and air pollutants (PM(2.5), NO(2), O(3)) from highly resolved spatiotemporal models, vegetation, and contextual deprivation were estimated for 20,904 French women (2002-2017). Distributed lag nonlinear models evaluated associations between heat and term birth weight (tBW), tBW Z-score, and small-for-gestational-age. We further adjusted our models for air pollutants and stratified on vegetation and social determinants. Heat exposure during the first two trimesters was associated with reduced fetal growth. A mean temperature of 21.6 °C (95th percentile vs median 13.6 °C) during weeks 2-15 was associated with a reduced tBW (-199 g [95% CI: -268; -131]). These associations differed after adjusting for O(3) exposure. Trends for stronger associations were observed in women with low vegetation exposure, low social position, and high contextual deprivation. This study highlights how heat stress during early pregnancy could reduce birth weight.