Aiming at addressing the difficulty in selecting parameters for the support vector machine (SVM) model in predicting slope safety factors, a Newton-Raphson Backtracking Optimization (NRBO) algorithm was optimized to assist the SVM model in rapidly selecting appropriate hyperparameters. The NRBO algorithm was improved by introducing a dynamic opposition-based learning strategy, horizontal and vertical crossover strategies, and a modified adaptive coefficient calculation formula, so as to construct an INRBO-SVM model for predicting slope safety factors. Six factors, including bulk density, cohesion, internal friction angle, slope angle, slope height and pore water pressure ratio, were selected as model inputs, with the safety factor as the output. The trained INRBO-SVM model, NRBO-SVM model, SVM model and RBF model were used to predict the safety factors of nine test samples. The results show that the INRBO-SVM model exhibits the best performance in safety factor prediction, with a correlation coefficient of 0.999 9, higher than those of the other models. Its root-mean-square error and mean absolute error are significantly lower than those of the other models. Engineering application results indicate that the prediction errors of the INRBO-SVM model for safety factors are all less than 10%, with most below 5%, confirming the accuracy and practical application value of the model in predicting safety factors.