Monitoring and predicting above ground biomass yield of grasslands are of key importance for grassland management. Established manual methods such as clipping or rising plate meter measurements provide accurate estimates of forage yield, but are time consuming and labor intensive, and do not provide spatially continuous data as required for precision agriculture applications. Therefore, the main objective of this study is to investigate the potential of sward height metrics derived from low-cost unmanned aerial vehicle-based image data to predict forage yield. The study was conducted over a period of 3 consecutive years (2014–2016) at the Rengen Grassland Experiment (RGE) in Germany. The RGE was established in 1941 and is since then under the same management regime of five treatments in a random block design and two harvest cuts per year. For UAV-based image acquisition, a DJI Phantom 2 with a mounted Canon Powershot S110 was used as a low-cost aerial imaging system. The data were investigated at different levels (e.g., harvest date-specific, year-specific, and plant community-specific). A pooled data model resulted in an R² of 0.65 with a RMSE of 956.57 kg ha⁻¹, although cut-specific or date-specific models yielded better results. In general, the UAV-based metrics outperformed the traditional rising plate meter measurements, but was affected by the timing of the harvest cut and plant community.