We report the first 3-D kinematical measurements of stars in the direction of several recently discovered substructures in the southern periphery of the Large Magellanic Cloud (LMC) using a combination of Gaia proper motions and radial velocities from the APOGEE-2 survey. More specifically, we explore stars that lie in assorted APOGEE-2 pointings in a region of the LMC periphery where various diffuse or stream-like overdensities of stars have previously been identified in maps of stars from Gaia and DECam; some of the APOGEE-2 pointings were deliberately selected to overlap these overdensities. By means of a comparison to a model of the LMC disk rotation, we find that a sizeable fraction of the APOGEE-2 stars have extreme space velocities that are distinct from, and clearly not a simple extension of, the LMC disk. Using N-body hydrodynamical simulations of the past dynamical evolution and interaction of the LMC and Small Magellanic Cloud (SMC), we explore whether the extreme velocity stars may be accounted for as LMC and/or SMC tidal debris created in the course of that interaction. We conclude that the combination of LMC and SMC debris produced from their mutual interaction is a promising explanation for the extreme velocity stars observed, although we cannot rule out other possible origins, and that these new data should be used to constrain future simulations of the LMC-SMC interaction. We also conclude that many of the stars in the southern periphery of the LMC lie out of the LMC plane by several kpc. Given that the chemistry of these stars suggest they are likely of Magellanic origin, our results suggest that a wider exploration of the past interaction history of the Magellanic Clouds is needed.