Oral cancer is the 6th most common cancer associated with high diseases related mortality. The prime reason is more than 2/3rd of patients are diagnosed at later stages of cancer. Majority of oral cancers are preceded by noticeable changes in the oral mucosa known as oral potentially malignant disorders (OPMDs). Early diagnosis of OPMDs can elude cancer development in 88% of cases. Artificial intelligence (AI) has gained popularity in the field of medicine including oncology and has shown efficacy in the diagnosis and prediction of cancer prognosis. In the present study, pre-trained Convolutional Neural Networks (CNNs) are used for identifying oral pre-cancerous and cancerous lesions and to differentiate them from normal mucosa using a dataset of clinically annotated photographic images. This study is conducted by collecting clinical oral photographs of biopsy-proven Oral squamous cell carcinoma cases, and normal mucosa. Transfer learning using various pre-trained CNN architectures was employed for image classification. An accuracy of 76% for VGG19, 72% for VGG16, 72% with MobileNet and 68% for InceptionV3 and 36% with ResNet50 was obtained. The results of the present study showed that VGG19 exhibited better performance in comparison of other models for identification and classification is comparable to a biopsy report.