And that is for a very simple reason: THE BUTTERFLY EFFECT.
Suppose you do travel back in time, to have a chat with GrandPa and GrandMa when they were the middle-age Parents of your Mom (and to find out the impossible: Mom was a kid once and even misbehaved a few times, too; unthinkable for any of us, of course, but Logic has it, that – against our very fundamental World-view, common sense and instincts – this was a possibility actually PERMITTED by the Laws of Physics).
NOW!! Suppose you were to take a brief stroll through town, as it had been back then.
A car stops to let you cross a street.
at the ORIGINAL time-line, it did not need to stop just then…
As a result of THAT newly-inserted stop, the driver comes to the next traffic-light after it has changed into red and as a result, comes to his appointed meeting half-a-minute later, than he ORIGINALLY would have.
There, at the office-building, he just happens to meet a charming young lady, who exited the elevator/office when he just stepped nearby – and whom ORIGINALLY, he would not have met.
They are falling instantly inLove, getting married and about a year later, their eldest son is born – WHO WAS NOT BORN AT THE ORIGINAL TIME-LINE – and that son turns out to be a World leading genius in biology, who invents a cure for leukemia already before he turns 19.
Among others, that cure just happens to heal a young patient, who – instead of dying (as in our World) gets better and, two years later, gives birth to another remarkable child – an ASTOUNDINGLY charismatic girl, who is ALSO an ultra-extreme nationalist and a raving militarist, in addition to her being a political genius.
And so, due to her being so awfully talented, she gets to become the minister of defense in her country (let's assume, just for the sake of this story, it is my own sad country, of israHELL) and – by razing down jerusalem's mosques – brings about a third world war, at which the amewicans supported israHELL, the Russians supported the Arabs and they both chose to push all buttons, for their sick and twisted fun and pleasure.
Hence, upon coming back to our time, you would find it is the realm of intense radiation, nuclear winter and giant insects – and all THIS, just because you wanted to take a stroll, through historic Tel-Aviv (or wherEVER).
This is merely ONE example, of what MIGHT happen, when you choose to mess around with Time Itself.
So – as much as I would have loved to go back in time (especially since I lost my Dad as a child, in an incident very easily avoided!!) – most likely, we should better ABANDON that notion, and forget we had ever even been interested.