Cage aquaculture is one of the most important types of marine aquaculture. Different types of cages have varying shapes in remote sensing images, and the background is complex. Previous methods for cage extraction have not been able to fully simulate human visual behavior and efficiently utilize spectral information. To address these issues, we propose a Spectral Loopy Attention U-Net (SLA-UNet) network model for cage aquaculture information extraction. The model utilizes the Random Forest (RF) algorithm based on the Estimation of Scale Parameter (ESP) to remove redundant spectral information after band operations. It also incorporates a human-like attention mechanism to enhance the important feature channels that affect cage information extraction. Additionally, edge completion is performed to supplement the loss information, achieving high-precision extraction of cage aquaculture information. We selected Zhanjiang City, Guangdong Province and Lingao County, as the study areas. Comparisons were made with the extraction results of the Canny algorithm, Otsu algorithm, PCA_Kmeans algorithm, RF algorithm based on ESP, and the U-Net model. The extraction accuracy of the SLA-UNet model for nearshore cages is 98.3%, and the average extraction accuracy for deep-sea cages is 98.9%, validating the effectiveness of the SLA-UNet model in cage aquaculture recognition.