Epilepsy is a neurological disorder characterized by seizures that can disrupt a patient’s quality of life. EEG has been used to detect underlying neural activity for diagnosis and treatment. However, standard methods of seizure detection are time-consuming and require manual detection by a trained clinician, with poor inter-clinician agreement. Automated analysis of EEG data offers the potential to improve diagnostic accuracy and reduce manual error. Here, we introduce a transformer-based model pretrained using annotated EEG scalp data that can detect the presence of seizures in a behind-the-ear wearable device setup for the 2023 ICASSP Signal Processing Grand Challenge. Our model has high sensitivity (100%) and a low False Alarm rate (1.78 FA/hr) for seizure detection on a hold-out test set. We also demonstrate how boosting gamma power in preprocessing can improve the performance of ChronoNet, an established EEG abnormality detection model.