Hackers have devised methods to manipulate the outputs of ChatGPT and Gemini, steering users toward fraudulent scam centers. By exploiting prompt injection or model biases, the AI systems inadvertently promote malicious links. This technique highlights a new vector for social engineering attacks via generative AI.