The present study investigated the association of carbohydrate intake and isocaloric substitution with different types of fat with visceral adipose tissue (VAT), subcutaneous adipose tissue (SAT) and hepatic fat content as determined by magnetic resonance imaging (MRI). Data from 283 participants (mean age 56.1 +/- 9.0 years) from the MRI sub study of the KORA FF4 study were included. VAT, SAT and total body fat were quantified by a volume-interpolated VIBE-T1w-Dixon MR sequence. Hepatic fat content was determined as the proton density fat-fraction (PDFF) derived from multiecho-T1w MR sequence. Dietary intake was estimated using information provided by two different instruments, that is, repeated 24-h food lists and a food frequency questionnaire. Replacing total carbohydrates with an isoenergetic amount of total fat was significantly positively associated with VAT and hepatic fat, while there was no significant association with SAT. The multivariable adjusted -coefficient for replacing 5% of total energy (5E%) carbohydrates with total fat was 0.42 L (95% CI: 0.04, 0.79) for VAT. A substitution in total fat intake by 5E% was associated with a significant increase in liver fat content by 23% (p-value 0.004). If reproduced in prospective studies, such findings would strongly argue for limiting dietary fat intake.