this seems to be the problem

If there were no God, there would be no Atheists. - Chesterton

This seems to be the essential argument of theists, that atheism is more about the disbelief, or the negative act of belief, than about either a) skepticism, or b) the lack of interest. I'm pretty sure that atheists generally attest less to "negative belief" and more to lack of belief (and interest in disbelief). Most of "us" do not care to reference deities in discourse on subjects other than religion or mythology (and to lesser extents, history or anthropology).

I think the core problem is that theists tend to assume that we might believe otherwise if we were to be convinced by some argument constructed of faith, or that our secular notions of life are in and of themselves a construction of faith. They are not, nor are they wholly incompatible with people who live within their own personal constructions of faith. This is the fallacious sentiment proposed with the idea that Darwinian evolution necessitates atheism, with the idea being that secularism is its own distinct philosophy fundamentally requiring the absence of god to function. Generally secularists do not care whether god exists or not, the issue for them is still centrally how human beings should conduct themselves regardless of faith or lack thereof and/or learn about their surroundings in order to live in accordance with it. The sort of extremist proposition being waged as a cultural war is annoying and diverting from attention areas on which human beings can amass real and fundamental understandings of things, as opposed to these purely metaphysical constructs of faith.