That would work, but Link never really time travels into the future, he just goes to sleep.
No, his spirit is sealed (effectly goes to sleep, anyway) the FIRST time he removes the Master Sword.
However, the rest of his travelling back and forth is basic time travel. The game never says otherwise. It just says that the Master Sword is a 'vessel' through which Link can move through time. It wouldn't make any real sense any other way, because then we end up with many, many Links sealed in the Sacred Realm. Overly complex, and Link only awakens in the Sacred Realm once. The other occasions he just moves from the past to the future in the Temple of Time. Thus, time travel.
It's called the Temple of Time for a reason, afterall.
Time travel implies paradoxes and thus having a single timeline becomes illogical and incomphensible.
Face it. Time travel destroys single timelines, especially so in the case of Majora's Mask which introduces more paradoxes and OoA, which introduces the worst time paradoxes of all the three games that involve time travel.
Actually, we're only looking at OoT. In OoT, there are no paradoxes. Everything fits. In fact, it appears physically impossible for a paradox that causes a split to happen in OoT. The Song of Storms sequence is in itself a paradox but that doesn't mean it causes a split. It just means it messes with your head. That's what time travel is like, however. Time is not linear.
In MM, the Goddess of Time is involved. That and the whole game implies a degree of time manipulation rather than simple time travel.
As for OoA... Well. Lets save that for another topic, but again it involves the Oracle of Age's magical powers, rather than just time travel on a basic scale. Her powers presumably give her a similar ability to manipulate time, which is why Veran wanted to possess her rather than just steal her harp thingie.