Real-time forecasting of infectious diseases is essential for public health decision-making. Traditional forecasting methods for epidemics exhibiting repeated waves, such as influenza and RSV, rely on strongly regular patterns in the incidence data, with stable duration (i.e., one year), peak timing, magnitude and initial incidence. In 2022-2023, COVID-19 epidemic waves became more regular compared to early in the pandemic, but they were nonetheless characterised by an absence of seasonality that shaped their overall trajectories, rendering existing methods unfit for use. Furthermore, the total number of waves with which to train the models was highly limited (as few as two). Leveraging an observed regularity in the dynamics of the growth rate, as opposed to the incidence, we developed a Bayesian framework for epidemic forecasting in situations where traditional forecasting methods would struggle. Our method learns from past epidemic waves to construct priors on the shape of the growth rate trajectory and updates the forecast as new data become available. We report a 27%-61% improvement in the weighted interval score for 14-day ahead forecasts compared to baseline models, as well as the ability to predict medium-term statistics such as peak timing and magnitude. We also introduce Gaussian processes for real-time smoothing and growth rate estimation, leading to a 41% reduction in root mean squared error on a simulated dataset over a popular, traditional technique. Our work highlights a promising approach for forecasting infectious diseases that do not follow strict seasonal patterns and reveals opportunities for further research into nonmechanistic time series models.