Agriculture, the world’s most dominant land use type, burdens freshwater biodiversity with a multitude of stressors such as diffuse pollution and hydromorphological alteration. However, it is difficult to directly link agricultural land use with biota response as agricultural stressors can also originate from other causes. Also, there is evidence for positive and negative effects of agriculture on organisms, agricultural impact differs strongly with the biological metric and study region considered and agricultural impact differs among practice and type, which in turn affects different organism groups with varying severity. Against this background, our study aimed at assessing, if agricultural land use has a consistent effect on river biota. We conducted a systematic review of the literature, which yielded 43 studies and 76 relationships between agriculture and aquatic organism groups. The relationships were subjected to a meta-analysis using Hedge’s g to calculate the standardized mean difference of effects. Overall, we detected a medium to strong effect g = − 0.74 of agricultural land use on freshwater biota, only marginally influenced by study design, river type and region. Strong differences in biota response could be observed depending on the biological metric assessed, with ecological quality indices of agricultural impairment performing best. Sensitive taxa declined with agricultural impact, while tolerant taxa tended to benefit. In addition, the biota response differed among agricultural types and practices and organism group, with macroinvertebrates showing the strongest effect. Our results quantify the effects of agriculture on riverine biota and suggest biological metric types for assessing agricultural impact. Further research is needed to discriminate between agricultural types and account for intensity.