We investigate the reaction $e p \to e PS X$ where PS denotes a pseudoscalar
meson $\pi^0, \eta, \eta'$, or $\eta_c$ and X either a proton or resonance or
continuum states into which the proton can go by diffractive excitation. At
high energies photon and odderon exchange contribute to the reaction. The
photon exchange contribution is evaluated exactly using data for the total
virtual photon-proton absorption cross section. The odderon exchange
contribution is calculated in nonperturbative QCD, using functional integral
techniques and the model of the stochastic vacuum. For the proton we assume a
quark-diquark structure as suggested by the small odderon amplitude in $pp$ and
$p \bar{p}$ forward scattering. We show that odderon exchange leads to a much
larger inelastic than elastic PS production cross section. Observation of our
reaction at HERA would establish the soft odderon as an exchange object on an
equal footing with the soft pomeron and would give us valuable insight into
both the nucleon structure and the mechanism of high energy diffractive
scattering.