The rise of general-purpose artificial intelligence (GPAI) systems is transforming industries by generating human-like text, images, and other content. However, these advancements bring a significant challenge: AI hallucinations—instances where AI produces plausible but false or nonsensical information. Such hallucinations undermine the reliability of AI outputs and pose risks when disseminated as factual data, especially in critical fields like law, healthcare, and journalism.This article explores the complex interplay between AI hallucinations and data subject rights under the General Data Protection Regulation (GDPR). It examines high-profile cases where individuals were inaccurately portrayed by AI systems, leading to data protection complaints. In April 2024, the consumer organization Noyb notoriously filed a complaint with the Austrian Data Protection Authority (DPA), alleging that ChatGPT violated GDPR's accuracy principle by providing an incorrect date of birth for a public figure and failing to rectify the error when notified. Drawing on regulatory perspectives, the article focuses into the nuanced approaches proposed by DPAs such as the Hamburg DPA and the UK's Information Commissioner's Office. In July 2024, the Hamburg DPA published a Discussion Paper that ignited extensive debate. This paper's significance lies in the Hamburg DPA's focus on the critical distinction between GPAI systems and Large Language Models (LLMs), which constitute only one component of GPAI systems. According to the Hamburg DPA, LLMs themselves do not contain personal data and, as such, fall outside the scope of the GDPR—a stance that has drawn criticism examined in detail in the paper – as well as the Hamburg DPAs response.However, the true significance of the Discussion Paper, lies in its call to shift regulatory attention toward other components of GPAI systems—particularly their outputs, where the GDPR clearly applies—rather than the internal mechanics of LLMs. The Hamburg DPA’s paper underscores an important point: LLMs do not store personal data in discrete records or operate as traditional structured databases. Consequently, applying the GDPR's accuracy requirement in its conventional form may be neither feasible nor appropriate.Similarly, the ICO proposed a risk-based approach to the issue of AI hallucinations, tailoring accuracy requirements to the purpose and context of AI use and emphasizing information and transparency. The combination of these guidances could be very helpful to mitigate the risks of violating the principle of accuracy and data subject rights under the GDPR when GPAI systems generate incorrect personal information, without hindering the development of these technologies in Europe.This article also explores the multifaceted efforts by GPAI system creators to address these issues, explaining in detail the technical and legal measures implemented to reduce hallucinations and mitigate associated risks. While these measures represent significant progress, they are yet far from perfect, and ongoing refinement is necessary as the technology evolves.By weaving together regulatory insights and industry practices, the article argues for a balanced approach and for ongoing collaboration among stakeholders to refine strategies that effectively manage AI hallucinations within the GDPR framework.