I haven't entirely thought this through, but I'd better get it down before the thinking escapes or dissolves.
The publication of Darwin's 'On the Origin of Species' in 1859 marked the opening up of what's been called The Great Schism - a huge gap between science and religion which seems to still be growing wider. This has been to the detriment of both sides, leaving religion to merely 'fill the gaps' that science has not yet answered but, supposedly, eventually will; and science has adopted a position strongly favouring Methodological Reductionism and Materialism, long since abandoning Cartesian dualism - the philosophy of mind, in which mind and body are understood as separate but interactive entities.
In Methodological Reductionism, everything is explainable if you reduce it to its fundamental parts and then describe and predict their behaviour. This approach has allowed us to solve most problems in nature, with the tree of science now bearing much delicious fruit: technology - that which makes our lives long, rich and comfortable. Who needs that messy old religion stuff? If we can't answer something with science, ignore it - it's just superstition anyway.
What got me thinking about this is the book 'Parapsychology and the Nature of Life - A Scientific Appraisal' by John L. Randall (Souvenir Press, UK, 1975). The book is deeply flawed and also out of date in many of the views expressed. Randall does not accept Evolution as an explanation for the variety and complexity of life on Earth, and believes that life requires a Great Mind in order to originate from basic building blocks. He makes a compelling but flawed argument for this (one I don't want to go into here) which predates the pseudoscience of Intelligent Design but in many respects is a forerunner of it.
On the positive side, Randall is critical but finally accepting of the evidence amassed by parapsychologists in favour of Psi powers; he holds that these powers have been proved to exist. His argument is long and complex and may even be largely correct - I don't know. But at least he has persuaded me to re-evaluate parapsychological findings in a more open-minded light. (This will involve a lot of re-reading.)
The first two parts of Randall's book are irritating in many ways and places, due to his skepticism of Evolution (I think mainly due to a lack of understanding of it, perhaps because of the time the book was written) and contrasting credulous acceptance of parapsychological experimental findings; but 'Part III - Towards a New Synthesis' is really mind-bending. It's turned my head.
Quantum Mechanics, one of the strongest theories in science, is quite happy with the dualistic concept of understanding light as both a wave and a particle, and choosing which behaviour suits the experiment that's being performed. The famous Double-slit experiment is the clearest example of this, but trying to get your head around the concept is pretty tricky, as it's highly unintuitive, even totally alien to most people's understanding of how things in nature behave.
So, if Randall is right about the findings of the parapsychologists, and Psi (telepathy, psychokinesis, clairvoyance, etc - if it's even valid to break it down in this way - Randall suggests it isn't) does exist - it is in fact a thing - then it lies outside science's Materialist, Reductionist way of understanding nature. It does not and will not fit. Thus, it is proscribed from Science and remains beneath its consideration. That is principally why Parapsychology is kept on the outer. (At this point, Randall made me realise that my hope that science will eventually offer answers to all of the riddles of the paranormal, even if it's not looking for them, is totally bound up in Reductionist thinking and thus invalid, or at least extremely unlikely to eventuate.)
What is needed is a dualistic philosophical approach to understanding nature, one that retains Materialism on one hand and includes some updated version of the the Philosophy of Mind on the other, and uses whichever approach (or maybe a blend of both) is best for an experimental approach to understanding a phenomenon.
We don't yet understand consciousness. Maybe it's time to scientifically re-evaluate the concept of the mind as separate from the body (not merely the product of brain activity, which is the materialist position) but interactive with it, and also perhaps that it may reside outside the body and even in some way connect with other minds in the space it resides.
Think I'll leave this here, now. It's making my head spin, and dinner's nearly ready.
Scientists closing in on a Theory of Consciousness - on LiveScience
Many Interacting Worlds and other mind-bending Quantum stuff
For another view on parapsychological experiomental results, read Dr Susan Blackmore's autobiography 'The Adventures of a Parapsychologist'
I'm not putting this forward as any kind of theory, just recording some thoughts.