If you do a cam swap, I'd imagine you'd need to also address removing the AFM lifters for some regular ones; you'd also have to make sure the engine is adjusted with regards to the VVT (Variable Valve Timing) -- everything in the computer is set up for the stock cam specs, and I would think running the engine without a re-tune could harm it.
As simple as it all sounds, any number of things can cause VVT problems:
Electrical problems with the supply voltage, ground connection or wiring to the oil flow control solenoid can prevent the cam phaser from doing its thing.
Low oil pressure due to a worn oil pump or worn cam bearings may prevent the cam phaser from developing sufficient internal pressure to rotate the cam.
Debris, sludge or varnish that restricts or blocks the oil flow control valve or inlet screens can also interfere with oil flow to the phaser, preventing it from working properly. The same thing can happen if the phaser itself is clogged with sludge, debris or oil varnish deposits.
Using the wrong oil viscosity in the engine may also make the phaser slow to respond, causing various fault codes to set.
Physical wear or damage inside the cam phaser housing may prevent it from rotating, cause it to stick, or make the unit noisy. A broken return spring may prevent it from returning to base timing.
Think of a lobed or vane style cam phaser as a backwards operating gerotor style oil pump. A gerotor oil pump produces oil pressure as the pump rotates. By comparison, a cam phaser rotates when oil pressure is applied to the rotor. Because of this, the clearances between the housing and rotor have to be fairly tight otherwise you end up with internal pressure losses that affect the operation of the unit.
With lobed rotor and vane style cam phasers, wear in the phaser housing, vanes or lobes reduce the unit's ability to hold pressure and function normally. The dowel pin that holds the rotor in its neutral position may also become worn or shear off, or the hole that the pin fits into may become elongated or worn preventing it from holding base timing. This can make the phaser noisy at idle as well as cause erratic valve timing.