I think we're supposed to assume that the Doctor didn't lose anything - in particular, it appears that the god physically consumes that which contains the memories, whether person or leaf, and that didn't happen.
It was a little unclear, as written it sounded like it consumed the Doctor's memories and then wanted more - but look at it from a different angle and it could be interpreted that it actually saw all the Doctor's dark secrets and instead of consuming them, it physically recoiled from them.
The ongoing theme is the Doctor's secrets, and the things he keeps secret, supposedly to protect others.