Surface waves in seismic exploration data provide crucial information about the near surface overburden structure. However, the surface wave energy is distributed in each mode, it is not accurate to discriminate the surface wave mode based on dispersive energy, so the conventional multi-mode surface wave inversion has the problem of mode misidentification, which affects the accuracy of near-surface structure inversion. In this paper, the modes of dispersion curves involved in inversion are determined on the based Kuhn-Munkres pattern searches algorithm, the algorithm is also applied to process and inverse actual seismic surface waves, and the underground velocity structure is established by the inversion based on accurately determining the order of higher-mode dispersion curves. Firstly, the traditional algorithm is utilized to invert the single fundamental mode dispersion curve to obtain a preliminary model. Then, the preliminary model and a weighted bipartite graph of phase velocities with no-explicitly-specified-mode at each frequency is established. The phase velocities of different modes are optimally matched by the Kuhn-Munkres algorithm, and then the mode discrimination and joint inversion with higher-order surface wave dispersion curves are implemented. The effectiveness of this algorithm is verified by three typical theoretical stratigraphic model examples, finally, two sets of actual data are processed to obtain the near-surface one-dimensional shear-wave velocity structure and two-dimensional profile of the relevant study area. The results demonstrate that the algorithm effectively solves the problem of mode misidentification in surface wave multi-mode dispersion curve inversion.