My Tokina 11-16mm did something similar to this on my old AF100. It was a problem with the adapter shifting a tiny bit when I applied rotational torque to the lens when activating the follow focus.
I noticed that image shift in a Tokina Zoom I owned briefly. Briefly as I sold it on ebay asap. I don't see that effect so much on any of my Nikkor Primes.