Background The aim of our study was to determine the relation of alcohol consumption and cigarette smoking on continuous-measured hepatic fat fraction (HFF) in a population free of cardiovascular disease. We suggested a direct correlation of alcohol consumption with HFF and increased HFF in former smokers compared to current smokers. Methods Data from 384 subjects (mean age: 56 years, 58% men) of a population-based cohort study (KORA) were included in a cross-sectional design. Liver fat was assessed by 3 Tesla magnetic resonance imaging (MRI) using a multi-echo Dixon sequence and T2-corrected single voxel multi-echo spectroscopy (H-1-MRS). Smoking status was classified as never, former or current smoker and alcohol consumption as non-, moderate (0.1-39.9 g/day for men and 0.1-19.9 g/day for women), or heavy drinker (>= 40 g/day for men and >= 20 g/day for women). Fatty liver disease was defined as HFF >= 5.56%. Results Average HFF was 8.8% by H-1-MRS and 8.5% by MRI. Former smokers showed a higher HFF (MRI: beta = 2.64; p = 0.006) and a higher FLD prevalence (MRI: OR = 1.91; p = 0.006) compared to never smokers. Current smokers showed decreased odds for FLD measured by H-1-MRS after multivariable adjustment (OR = 0.37; p = 0.007) with never smoker as reference. Heavy drinking was positively associated with HFF (H-1-MRS: beta = 2.99; p = 0.003) and showed highest odds for FLD (H-1-MRS: OR = 3.05; p = 0.008) with non-drinker as reference. Moderate drinking showed a positive association with HFF (H-1-MRS: beta = 1.54; p = 0.061 and MRI: beta = 1.75; p = 0.050). Conclusions Our data revealed lowest odds for FLD in current smokers, moderate drinkers showing higher HFF than non-drinkers and heavy drinkers showing highest HFF and odds for FLD. These findings partly conflict with former literature and underline the importance of further studies to investigate the complex effects on liver metabolism.