To address the issues of high computational cost and complex network models in point cloud classification and segmentation methods, a point cloud classification and segmentation algorithm based on lightweight networks and weighted Random Forest (RF) was proposed. The algorithm achieved efficient classification and segmentation in a hierarchical manner. Firstly, to address the issues of multiple layers and complex computation in traditional neural networks, a lightweight neural network was constructed to extract point cloud features such as global shape, inter-regional relationships, curvature, normal vector, and color, thereby achieving rapid rough classification and segmentation of point clouds. Then, to address data imbalance, an adaptive classification and segmentation strategy was designed. By introducing a weighted RF and combining inconsistency-measurement screening with dynamic-weighting optimization mechanisms, fine classification and segmentation of point clouds were achieved. The algorithm conducted classification experiments on the ModelNet40 dataset and segmentation experiments on the Semantic3D dataset and outdoor-scene point-cloud data. The results showed that compared with Local Geo-Transformer, PointNeXt, and FastPointNet++, classification and segmentation accuracy increased by approximately 1.9%, 1.6%, and 1.7%, respectively, while classification and segmentation time was reduced by approximately 40%, 30%, and 20%, respectively. Thus, the proposed point-cloud classification and segmentation algorithm based on lightweight networks and weighted RF can effectively reduce the training time of the model and improve the efficiency of classification and segmentation while maintaining high accuracy, making it an effective point cloud classification and segmentation algorithm.