Portable microwave stroke detection devices often suffer from sparse spatial sampling due to a limited number of antennas. This limitation leads to severe ill-posed and ill-conditioned problems in microwave inverse scattering imaging. To address these technical challenges, this paper proposes a cascaded imaging architecture based on a physics-constrained Swin Transformer, termed deep learning-enhanced truncated singular value decomposition (DL-TSVD) algorithm. By leveraging the global self-attention mechanism of the deep learning network, this architecture achieves high-fidelity mapping from a low-dimensional 8×8 sparse scattering matrix to a 16×16 dense matrix. Subsequently, the dense matrix is fed into the truncated singular value decomposition (TSVD) algorithm to accomplish the high-resolution inversion of the brain's permittivity distribution. In terms of S-parameter fitting accuracy, the complex-domain root mean square error of the predicted matrix is significantly reduced by 67.53% compared with traditional interpolation methods. Additionally, phantom experiments verify the noise-resistant generalization ability of the DL-TSVD architecture under real hardware noise and antenna mutual coupling environments. The structural similarity index measure (SSIM) of the reconstructed images reaches 0.541. The results demonstrate that the proposed method effectively mitigates the ill-conditioned nature of the electromagnetic inverse scattering problem under sparse arrays. It breaks through the resolution bottleneck of spatial sampling without increasing the number of antennas, providing a valuable technical reference for portable microwave stroke monitoring systems.