The traditional fixed single sensor for rail defect detection is limited by the monitoring range and accuracy, hardly meeting the advanced rail non-destructive testing (NDT) requirements. The dynamic detection manner with mobile inspection vehicles is a promising way to solve these problems. Therefore, combining the sensitivity advantage of acoustic emission (AE), an improved correlation-based adaptive joint sparse representation (JSR) method is proposed for multi-channel dynamic detection of rail damage, implementing the precise wide-range NDT in a noisy background. In this method, the multi-layer wavelet packet transform is utilized to pre-process the weighted wavelet coefficients based on the energy gradient of each node, eliminating the random noise. Subsequently, the correlation constraint is integrated into the JSR process to promote the fusion effect of multi-channel signals. Meanwhile, a novel sparsity-adaptive matching pursuit algorithm with a self-adjusting step size is proposed to address the overlap between wheel–rail rolling noise and damage signals. It continuously modifies the iterative step size according to the energy variations, which improves the efficiency and accuracy of the joint sparse coefficient (JSC) estimation, while effectively enhancing and isolating the critical damage information. Then, an adaptive threshold is developed to detect the damage without signal reconstruction, thus reducing the detection time. The validity and reliability of the proposed method are verified by designed wheel–rail rolling experiments. Based on the result, the onboard sensor arrangement strategy with better noise resistance is analyzed, which provides a new approach for the AE dynamic detection technology application of rails.