Increasing efficiency in livestock production and reducing the share of animal products in human consumption are two strategies to curb the adverse environmental impacts of the livestock sector. Here, we explore the room for sustainable livestock production by modelling the impacts and constraints of a third strategy in which livestock feed components that compete with direct human food crop production are reduced. Thus, in the outmost scenario, animals are fed only from grassland and by-products from food production. We show that this strategy could provide sufficient food (equal amounts of human-digestible energy and a similar protein/calorie ratio as in the reference scenario for 2050) and reduce environmental impacts compared with the reference scenario (in the most extreme case of zero human-edible concentrate feed: greenhouse gas emissions -18%; arable land occupation -26%, N-surplus -46%; P-surplus -40%; non-renewable energy use -36%, pesticide use intensity -22%, freshwater use -21%, soil erosion potential -12%). These results occur despite the fact that environmental efficiency of livestock production is reduced compared with the reference scenario, which is the consequence of the grassland-based feed for ruminants and the less optimal feeding rations based on by-products for non-ruminants. This apparent contradiction results from considerable reductions of animal products in human diets (protein intake per capita from livestock products reduced by 71%). We show that such a strategy focusing on feed components which do not compete with direct human food consumption offers a viable complement to strategies focusing on increased efficiency in production or reduced shares of animal products in consumption.