Infertility is a global health issue (1) that affects one in six couples in Australia (2), nearly half of which involve an element of male-factor infertility (3). A semen analysis including the morphological assessment is a routine and essential step in the infertility assessment pipeline. In most clinics, such analysis is still carried out manually leading to subjective results. The fundamental difficulty of sperm morphology classification is ideally suited to machine learning algorithms to automatically analyze and classify sperm images (4). Machine learning-based algorithms have been used to classify images of stained/dead sperm cells based on their head morphology status (5,6). Here, we demonstrate a Meta-classifier algorithm for morphology classification of stain-free live human sperm cells considering head, midpiece, and tail abnormalities. Combining VGG16, VGG19, modified ResNet-34, and DenseNet-161 architectures, an ensemble deep learning model has been developed to generate a comprehensive set of visual features for the morphology classification of sperm cells. The model includes fully automated detection and cropping algorithms to extract individual sperm from 100× magnification images and pass them through the classification algorithm. To prepare our training dataset, a user-friendly software interface is being developed and used by two expert clinicians to independently label the cells based on their morphological characteristics. We achieved a classification rate of 74.5% with 78.5% precision for overall normal/abnormal unstained human sperm images. Additionally, our model is capable of classifying cells based on their head, midpiece, and tail abnormalities with classification rates of 76% and 70.5%, respectively. We demonstrate a comprehensive machine learning algorithm capable of classifying sperm based on their head, midpiece, and tail abnormalities, providing a promising opportunity to improve sperm morphology analysis in fertility clinics.