The oil palm industry, vital to tropical economies, faces significant challenges from leaf damage. Despite advancements in leaf damage detection, accuracy and efficiency still need improvement. This study implements the Xception model for classifying oil palm leaf damage. We evaluated the model’s performance using a dataset of healthy and infested leaf images, optimising parameters such as epochs, batch size, learning rate and optimiser. The best results were achieved with 15 epochs, a batch size of 64, the RMSProp optimiser and a learning rate of 0.001. The Xception model demonstrated exceptional performance, achieving an accuracy, precision, recall and F1-score of 99.73%, with a computational time of 272 s. Despite the promising results, challenges due to environmental variations and limited data remain. Future study should focus on further optimising hyperparameters, expanding the dataset and applying the model in field settings with a decision support system. This study provides a solid foundation for developing practical solutions to support the palm oil industry in addressing leaf damage.