Objectives To establish the effect of different degrees and kinds of physical activity on bone marrow fat (BMAT) content at different anatomical locations in a population-based cohort study undergoing whole-body MR imaging. Methods Subjects of the KORA FF4 study without known cardiovascular disease underwent BMAT fat fraction (FF) quantification in L1 and L2 vertebrae and femoral heads/necks (hip) via a 2-point T1-weighted VIBE Dixon sequence. BMAT-FF was calculated as mean value (fat image) divided by mean value (fat + water image). Physical activity was determined by self-assessment questionnaire regarding time spent exercising, non-exercise walking, non-exercise cycling, and job-related physical activity. Results A total of 385 subjects (96% of 400 available; 56 +/- 9.1 years; 58% male) were included in the analysis. Exercise was distributed quite evenly (29% > 2 h/week; 31% 1 h/week (regularly); 15% 1 h/week (irregularly); 26% no physical activity). BMAT-FF was 52.6 +/- 10.2% in L1, 56.2 +/- 10.3% in L2, 87.4 +/- 5.9% in the right hip, and 87.2 +/- 5.9% in the left hip (all p < 0.001). Correlation of BMAT-FF between spine and hip was only moderate (r 0.42 to 0.46). Spinal BMAT-FF, but not hip BMAT-FF, was inversely associated with exercise > 2 h/week (p <= 0.02 vs. p >= 0.35, respectively). These associations remained significant after adjusting for age, gender, waist circumference, and glucose tolerance. No coherent association was found between BMAT-FF and physical activity in the less active groups. Conclusions In our study, exercise was inversely correlated with vertebral BMAT-FF, but not hip BMAT-FF, when exercising for more than 2 h per week. Physical activity seems to affect the spine at least preferentially compared to the hip.