Spherical alumina was introduced in drawing and filtering process of graphene nanosheets to build a binary porous structure of “pea-pod-like” alumina-graphene, and an alumina-graphene binary structure reinforced epoxy resin composite was prepared. Its thermal conductivity was tested, and the mechanism of “pea-pod-like” alumina-graphene binary structure enhancing the thermal conductivity of epoxy resin was analyzed. The results show that the horizontal arranged graphene generates partial orientation transformation under the action of spherical alumina, showing a “pea-pod-like” structure, in which graphene provides an efficient heat transfer channel for epoxy composites in plane and out of plane directions and greatly enhances the thermal conductivity of epoxy composites. The thermal conductivity of “pea-pod-like” binary alumina-graphene reinforced epoxy composite is up to 13.3 W/(m·K) and 33.4 W/(m·K) in plane and out of plane directions, respectively with 12.1% of graphene and 42.4% of alumina loading. The “pea-pod-like” alumina-graphene binary structure has a significant effect in improving the thermal conductivity of epoxy resin, which has potential application prospects in the field of electrical packaging.