Sounds like your turbo has excessive shaft play. Mine did the exact same thing as yours. About 35,000 miles ago, I rebuilt my stock turbo with 1.0 exh housing, WW2, and ceramic coating. The boost response was far improved over stock. I have 160cc injectors and a TS Chip. I could easily hit 30 pounds of boost very quickly in the hot tune position and 20 pounds in the stock position.
The turbo started howling under load and had a ton of shaft play due to a worn thrust plate from all the exhaust back pressure. I think the 1.0 housing caused this along with a hot tune. I upgraded to bellowed up pipes and the KCTP38R a few weeks ago.
Since the turbo upgrade a few things I’ve noticed......even with the stock tune, I get a some black haze when I first roll into the throttle and the rpm’s are under 1500, stock turbo did not do this. With the stock turbo it would blow tons of black smoke all the way through the pull on the hot tune. The KC turbo cleans it up almost completely one it “lights” off. Big difference there. On the hot tune, I can hit 38 pounds very very quickly. I had to replace all my IC
boots because they kept blowing off! I personally feel it isn’t as loud, spool up and “whoosh”, when I lift under load. You can hear is spoil down up to 20 seconds after the key is turned off. Overall......unloaded and just driving, it feels less responsive when under 1500 rpm. With a load 3500 pounds or more, it feels much stronger when accelerating and my egt’s seem to stay about the same as the stock turbo was. I also noticed that I now drive in the stock tune position more now, so I guess that means it is more responsive than the stock, “worn”, turbo. With the KC turbo, when it “lights”, it feels like a 4 barrel opening up. I hope this helps.