Minimizing the Mumford-Shah functional is frequently used for smoothing signals or time series with discontinuities. A significant limitation of the standard Mumford-Shah model is that linear trends-and in general polynomial trends-in the data are not well preserved. This can be improved by building on splines of higher order which leads to higher order Mumford-Shah models. In this work, we study these models in the univariate situation: we discuss important differences to the first order Mumford-Shah model, and we obtain uniqueness results for their solutions. As a main contribution, we derive fast minimization algorithms for Mumford-Shah models of arbitrary orders. We show that the worst case complexity of all proposed schemes is quadratic in the length of the signal. Remarkably, they thus achieve the worst case complexity of the fastest solver for the piecewise constant Mumford-Shah model (which is the simplest model of the class). Further, we obtain stability results for the proposed algorithms. We complement these results with a numerical study. Our reference implementation processes signals with more than 10,000 elements in less than 1 s.