The location entropy method, super efficiency SBM-UN model, and panel Tobit model were adopted to explore the impact mechanism of county-level industrial agglomeration on land use efficiency. The research results indicate that the agglomeration degree of the secondary industry in Inner Mongolia counties first increased and then decreased, and the agglomeration degree of the tertiary industry first decreased and then increased. The overall trend of land use efficiency in Inner Mongolia’s counties shows a decrease followed by an increase, with high-value areas gradually concentrating towards the center. The agglomeration of county-level industries in Inner Mongolia has a positive driving effect on land use efficiency, and its impact mechanism is significantly heterogeneous. The level of economic development and population density have a significant promoting effect on the improvement of land use efficiency, while land use structure, consumption level, and enterprise size have a negative hindering effect on the improvement of land use efficiency. Due to differences in resource endowments and economic development levels, various counties (districts) have significant differences in industrial agglomeration levels and land use efficiency. In the future, it is necessary to continue to strengthen the guidance and planning of industrial agglomeration, and optimize the allocation of land resources.