Nano Banana Pro is now generating fake Aadhaar and PAN cards: what we found

The recent debut of Google's Gemini Nano Banana Pro has sparked intense discussions online, primarily due to its remarkable advancements in character consistency and 4K image generation capabilities. Since its launch last week, users have been exploring a variety of practical applications, including crafting appealing portraits and transforming LinkedIn profiles into engaging AI-infused infographics. However, amidst the innovative use cases, a troubling trend has emerged. Users have begun to recognize that the model's realistic image generation prowess could enable the creation of counterfeit Indian identity documents, such as Aadhaar and PAN cards. This situation poses significant privacy and security risks. In our own testing, we attempted to generate realistic Aadhaar and PAN cards using the Nano Banana Pro, and to our surprise, the AI produced them seamlessly without raising any flags. The model incorporated my image along with the typical identifiers and fabricated details, showcasing its sophistication. While Google does place a Gemini watermark on the generated images and uses invisible SynthID watermarking to distinguish them from authentic documents, these measures may not be enough to prevent misuse. The potential for these lifelike IDs to be misrepresented as genuine, especially when printed or shown briefly, is alarming. The oversight raises questions about how such a fundamental risk could be overlooked by Google's safety teams, especially considering the company's previous scrutiny over its strict content moderation policies. Users have noted on social media instances where the Gemini model refused requests for images deemed inappropriate or sensitive. This isn't the first instance of an AI model generating fake ID proofs. Earlier, OpenAI's ChatGPT (GPT-4o) also faced similar issues, producing realistic PAN and Aadhaar card images. However, the situation has escalated with the Nano Banana Pro, which significantly surpasses ChatGPT in generating realistic visuals. The implications of this technology on privacy and security remain a pressing concern.