Aims: The aim of the current study was to investigate the association of type 2 diabetes (T2D) and insulin treatment with changes in muscle mass, muscle strength, and physical performance in older adults. Methods: In 731 participants of the population-based KORA-Age study aged 74.6 ± 6.2 years (T2D: n = 118; insulin treatment: n = 20), skeletal muscle index (SMI [kg/m2]), hand grip strength (GS [kg]), and a timed up and go test (TUG [s]) were performed at baseline and after a follow-up time of 3 years. The association of T2D and insulin therapy with changes in muscle parameters was analyzed using linear regression models. Results: After adjustment for sex, age, BMI, physical activity, smoking, and multimorbidity, T2D was associated with the change in SMI during follow-up (β − 0.1 (95% CI − 0.3 to − 0.02) kg/m2; p = 0.02), but not with a change in GS (β − 0.9 (95% CI − 1.9 to 0.04) kg) or TUG (β − 0.1 (95% CI − 0.7 to 0.5) s). Insulin therapy was positively associated with change in SMI (β 0.6 (95% CI 0.3–0.9) kg/m2; p = 0.001), but not in GS (β − 1.6 (95% CI − 4.1 to 0.8) kg) or TUG (β 1.6 (95% CI − 0.2–3.4) s) in comparison with treatment with oral anti-diabetic medication alone. Conclusions: Participants with T2D showed an accelerated decline in muscle mass compared to non-diabetic participants. Insulin therapy was associated with preserved muscle mass, but not muscle function parameters, indicating a discrepancy between muscle mass and function in this high-risk population.