It could be both. I think some people might be attracted to children because they are easy to manipulate because of their lack of experience much as some adults seek out vulnerable adult partners. It might be to do with them not being able to relate to equals and needed an advantage ie powerful position.
This is no doubt true in some cases. Thinking back to my childhood, I encountered what would have been called 'inadequates' (sad word), but also good looking young men of 18 or so and dirty old men generally.
This was between the ages of 8 to ten and during the 50s at a time when I was on my own a lot so more vulnerable.
ETA Just to be clear, the young men - one came and sat by me in the cinema and tried to feel me up but I moved and another followed me through a wood, hiding behind a tree watching me at one point and then chased me but I escaped. Very frightening. There are other incidents I could recount but have to go out just now.