The role of consuming different types of fatty acids (FA) at the expense of carbohydrates (CHO), on the blood lipid profile of adolescents is largely unknown, as is the modulating effect of different levels of physical activity (PA). Children from the GINIplus and LISA birth cohorts, with complete data on dietary FA (assessed by food-frequency questionnaires), objectively-measured PA (assessed by accelerometers) and blood lipids (lipoprotein cholesterol and triglycerides) at age 15 years, were included (N = 837). Sex-stratified associations between dietary FA and blood lipids were assessed by linear regression in substitution models which represented isocaloric replacements of CHO with saturated FA (SFA), monounsaturated FA (MUFA), n-3 polyunsaturated FA (PUFA) or n-6 PUFA. To assess the interactions with PA, analyses were then performed stratified by tertiles of different PA levels (sedentary, lifestyle, moderate-to-vigorous (MVPA)). Both sexes presented a significant inverse association between MUFA and triglycerides, and females a direct association between n-3 PUFA and high-density lipoprotein. Stratifying by PA tertiles, associations were mainly restricted to participants with the lowest levels of lifestyle PA, or the highest time spent sedentary. The effects of dietary FA on the lipid profile vary in an activity-specific manner, emphasizing possible synergistic roles of diet and PA.