It’s hard to give a complete answer without more information, but since it seems like the data are paired you can use McNemar’s test for dependent proportions (e.g., sensitivity). The information on the WIkipedia page (in the Definition section and the exact P-value calculation under Variations) is essentially the same as what I read in Statistical Methods in Diagnostic Medicine, 2nd Edition (2011) by Zhou, Obuchowski, and McClish; specifically Chapter 5.1.1 in the text.
If the data are clustered, it is slightly more complicated. To borrow an example of clustered data from their text, if you were studying polyps such that the diagnostic unit of study was a single polyps, then the subject/patient constitutes a cluster.
Either way I don’t think a mixed effects model is necessary. I don’t think it would be since it doesn’t look like you’re adjusting for any covariates.