To safeguard employees' psychological health and improve the accuracy and interpretability of psychological stress evaluation methods, taking multimodal physiological time-series data as the research object, a LSTM with AM(LSTMA) method was proposed to accurately evaluate employees' psychological stress states in the paper. Firstly, using the multimodal physiological time-series data (Blood Volume Pulse (BVP), Electrocardiogram (ECG), Electrodermal Activity (EDA), Electromyogram (EMG), Respiration (RESP), Body Temperature (TEMP), and three-axis Acceleration (ACC)) from the WESAD dataset were adopted as the research carrier, the gating memory mechanism of the modal-specific LSTM modules was utilized to accurately capture cross-time-step temporal dependency features, effectively retain key physiological features strongly associated with psychological states, and filter out short-term random noise, thereby ensuring that the physiological feature data could truly characterize the dynamic evolution of employees' psychological states. Secondly, after feature fusion, the attention mechanism was introduced to adaptively assign attention weight coefficients based on the feature importance of physiological data across different modalities and time steps, enhancing key features and micro-response features sensitive to psychological stress states while suppressing the interference of redundant information. Finally, the accurate evaluation of psychological stress states was accomplished through a fully connected neural network. Experimental results show that the LSTMA method achieves an evaluation accuracy of 94.56% for the four-classification task (neutral, stress, pleasure, and meditation) of psychological stress states. After Leave-One-Out Cross-Validation (LOOCV), the accuracy is improved to 98.08%. Ablation experiments verify the synergistic enhancement effect of the modal-specific LSTM and the attention mechanism, and model interpretability analysis further confirms the scientificity and rationality of LSTMA.