Artificial Intelligence: The ethical dilemma of Hugging Face and deepfakes

The vulnerability of open source to malicious use

The rise of artificial intelligence has democratized access to unprecedented visual creation tools. However, this openness has a dark side. According to a recent investigation by the non-profit organization AI Forensics, the Hugging Face platform is being used to generate non-consensual deepfakes, including sexualized images of women and minors.

Unlike large tech companies that integrate safety filters, the open-source ecosystem presents unique challenges. While tools like Gemini or ChatGPT have strict barriers in place, the study points out that seven of the nine most popular image editing models in the repository lack the necessary safeguards to prevent abuse.

Is moderation failing in the AI ecosystem?

The gap between innovation and security

The premise of transparency in machine learning is fundamental to technical progress, but the lack of moderation on platforms that host open-access models allows malicious users to exploit the technology. The findings from AI Forensics are alarming: simple prompts are enough for these tools to perform actions that violate people's privacy and dignity.

"Seven out of nine image editing models analyzed complied without restriction to requests to undress people using simple prompts," the report highlights.

The impact on the industry

This scenario raises a necessary debate about the responsibility of model repositories regarding ethics. While the community values the freedom of code, the proliferation of harmful content forces a rethink of how models are supervised before public deployment. The industry is watching this problem with caution, while other tools seek to balance openness with security, as analyzed in our recent article on Artificial Intelligence: Goose challenges Claude Code with free code.

Conclusion

LLM and image generation technology is not inherently malicious, but its social impact depends on usage policies and active oversight. It is imperative that platforms like Hugging Face implement robust security mechanisms to curb the creation of abusive material, ensuring that technological innovation does not become a tool for violating fundamental rights.

---

Sources:

• The Verge - Hugging Face is being used to easily undress women and children