Low-input agricultural systems aim at reducing the use of synthetic fertilizers and pesticides in order to improve sustainable production and ecosystem health. Despite the integral role of the soil microbiome in agricultural production, we still have a limited understanding of the complex response of microbial diversity to organic and conventional farming. Here we report on the structural response of the soil microbiome to more than two decades of different agricultural management in a long-term field experiment using a high-throughput pyrosequencing approach of bacterial and fungal ribosomal markers. Organic farming increased richness, decreased evenness, reduced dispersion and shifted the structure of the soil microbiota when compared with conventionally managed soils under exclusively mineral fertilization. This effect was largely attributed to the use and quality of organic fertilizers, as differences became smaller when conventionally managed soils under an integrated fertilization scheme were examined. The impact of the plant protection regime, characterized by moderate and targeted application of pesticides, was of subordinate importance. Systems not receiving manure harboured a dispersed and functionally versatile community characterized by presumably oligotrophic organisms adapted to nutrient-limited environments. Systems receiving organic fertilizer were characterized by specific microbial guilds known to be involved in degradation of complex organic compounds such as manure and compost. The throughput and resolution of the sequencing approach permitted to detect specific structural shifts at the level of individual microbial taxa that harbours a novel potential for managing the soil environment by means of promoting beneficial and suppressing detrimental organisms.