For some polymers below or near their glass transition temperature, a particular type of non-Fickian solvent diffusion, known as Case Ⅱ diffusion, is typically observed. To describe the coupling effect of Case Ⅱ diffusion and swelling deformation in polymers, theoretical models are established based on continuum mechanics. Here, governing equations for solvent penetration into polymer are derived and specialized in the reference configuration, including the mechanical-chemical equilibrium state equation, the concentration-dependent diffusion equation, and the molecular number conservation equation. Additionally, a visco-hyperelastic constitutive equation taking into account the time-dependent deformation characteristics of the material is integrated to reflect the competition mechanism between relaxation rate of the polymeric network and migration of solvent in Case Ⅱ diffusion. This modeling approach is used to analyze the transient free swelling process for two material systems, so as to investigate the behavior of unidirectional Case Ⅱ diffusion in columnar and tabular polymer specimens without constraint. By applying appropriate boundary and initial conditions, the concentration, stress, and deformation field variables during the unidirectional diffusion are directly obtained. The distribution and evolution of these calculation results are compared with experimental observations, moderately validating the effectiveness and adaptability of the proposed coupling analysis method regarding polymer swelling. This developed theory may provide important guidance for practical applications such as membrane designing or drug delivery systems, where Case Ⅱ diffusion commonly occurs. It also aids in enhancing understanding of the combination of different polymer-solvent diffusion scenarios, from Fickian to non-Fickian circumstances.