The AI Economy Is DEAD. 6 Billion Images Now POISONED.
채널: The Infographics Show | 링크: https://www.youtube.com/watch?v=zF-mbwc5Mmw | 길이: 16:13
요약
🎯 TL;DR — 예술가와 창작자들이 Glaze·Nightshade 같은 "데이터 오염" 도구로 AI 이미지·음성 생성 모델을 역으로 공격하기 시작했고, 이는 무단 스크레이핑에 기반한 AI 산업의 "데이터는 공짜·무한·깨끗하다"는 전제 자체를 무너뜨리고 있다.
📌 핵심 논지 — 영상은 AI 모델이 학습 데이터셋에 은밀히 섞인 소수의 조작된 이미지만으로도 특정 개념(예: 개) 전체를 망가뜨릴 수 있다는 "데이터 포이즈닝" 현상을 설명한다. 이는 해킹이 아니라, 창작자들이 자신의 작품과 목소리가 동의 없이 스크레이핑당한 것에 대한 반격이며, AI 랩과 창작자 사이의 끝나지 않는 군비 경쟁으로 이어지고 있다고 화자는 강조한다.
🗺️ 영상 구성
- 포이즈닝 현상 소개 (개 이미지 왜곡 사례)
- 스크레이핑의 기원 — LAION-5B와 저작권 소송
- 대응 도구의 등장 — Glaze → Nightshade
- 전선의 확장 — 이미지에서 음성으로 (Safe Speech)
- AI 랩의 반격과 현재의 교착 상태
━━━ 본문 ━━━
포이즈닝은 어떻게 작동하는가 화자는 먼저 "달리는 개" 이미지를 생성했을 때 다리가 이상하게 연결되고 얼굴이 uncanny valley로 빠지는 사례를 제시한다. 이는 버그가 아니라 학습 데이터 자체가 오염되었기 때문이라고 설명한다. 시카고 대학 연구팀이 Stable Diffusion에 조작된 개 이미지 50장을 주입하자 즉시 이미지가 왜곡되기 시작했고, 300장에 이르자 모델은 "개"를 요청해도 고양이 이미지를 만들어냈다. 화자는 모델이 특정 개념 하나당 실제로는 수천 장의 이미지에만 의존하기 때문에, 전체 데이터의 1% 미만만 오염시켜도 해당 개념이 붕괴한다고 강조한다.
분노의 기원 — 무단 스크레이핑 화자는 이 저항이 "도둑질"에서 시작됐다고 규정한다. AI 기업들은 동의나 대가 없이 수십억 개의 이미지·음성·텍스트를 웹에서 긁어모았고, 그 상당량이 LAION-5B(60억 개 이미지·텍스트 쌍)라는 공개 데이터셋에 담겼다. Stable Diffusion을 비롯한 대부분의 생성 모델이 이를 학습했다. 작가들은 자신의 화풍이 프롬프트 하나로 몇 초 만에 복제되는 것을 목격했고, 컨셉 아티스트 Karla Ortiz는 2023년 Stability AI를 고소했으며 Getty Images도 별도 소송을 제기했다. OpenAI는 영국 의회에 저작권 있는 자료 없이는 오늘날의 최상위 모델을 만드는 것이 불가능하다고 인정했다고 화자는 전한다.
반격의 시작 — Glaze와 Nightshade 시카고 대학은 2023년 Glaze를 내놓았다. 이는 AI가 인식하는 이미지의 수치 구조를 사람 눈에는 보이지 않게 미세하게 왜곡해, 기계가 화풍을 잘못 학습하게 만드는 도구다. Glaze는 600만 다운로드를 넘겼지만, 화풍만 숨길 뿐 이미지 자체의 도용은 막지 못하는 한계가 있었다. 이에 2024년 1월, 같은 팀이 더 공격적인 Nightshade를 출시했다. 사람과 기계 눈에는 정상으로 보이지만, 실제로 학습되면 모자가 케이크로, 핸드백이 토스터로, 자동차가 소로 그려지는 식으로 모델을 "감염"시킨다. 출시 5일 만에 다운로드 25만 건을 기록했고, 대학 서버가 트래픽을 감당하지 못해 다운될 정도였다. 화자는 이를 해킹이 아니라 적대적 머신러닝(adversarial machine learning)이라 부르며, "모델이 스스로를 잘못 가르치도록 속이는 마술"이라 표현한다.
전선의 확장 — 목소리를 지키는 기술 포이즈닝은 이미지에 그치지 않는다. 화자는 ElevenLabs 같은 서비스가 몇 초의 음성 샘플만으로 목소리를 복제할 수 있으며, 이것이 이미 사기 범죄에 악용되고 있다고 경고한다. 2024년 복제된 음성·영상으로 화상통화 사기를 벌여 약 2,500만 달러를 탈취한 사건이 실제로 발생했다. 이에 대응해 개발된 Safe Speech는 음성의 고유한 "지문"을 라디오 잡음처럼 미세하게 흐트러뜨려, 사람에게는 똑같이 들리지만 복제 모델은 핵심 특징을 학습하지 못하게 만든다.
AI 랩의 반격과 교착 상태 AI 기업들은 오염된 데이터를 걸러내기 위해 이미지와 캡션의 일치 여부를 검사하는 방식을 도입했지만, 이는 오염 이미지의 40~60%만 걸러내며 동시에 멀쩡한 데이터까지 삭제하는 부작용이 있다고 화자는 지적한다. 이미지를 사전에 "세척"하는 정제 모델도 시도됐지만 느리고 비용이 크며, 포이즈닝 기술이 이 세척 과정을 견디도록 계속 진화해 얼룩처럼 다시 배어 나온다고 설명한다. 결국 이는 소수의 팀이 끊임없이 새 기법을 실험하는 쪽과, 몇몇 대형 AI 랩이 구멍을 메우는 쪽의 비대칭적 싸움이며, 현재로서는 포이즈닝을 거는 쪽이 우세하다고 화자는 결론짓는다.
🔢 핵심 수치
- LAION-5B 데이터셋 = 이미지 60억 개
- 오염 이미지 50장 → Stable Diffusion 즉시 왜곡
- 오염 이미지 300장 → "개" 요청 시 고양이 생성
- 개념 하나를 오염시키는 데 필요한 최소 비율 = 전체 데이터의 1% 미만
- 하나의 개념을 오염시키는 데 필요한 이미지 = 100장 미만
- Glaze 다운로드 = 600만 건 이상
- Nightshade 다운로드 = 출시 5일 만에 25만 건
- 캡션 대조 필터링의 포이즈닝 탐지율 = 40~60%
- 2024년 음성·영상 복제 사기 피해액 = 단일 사건 약 2,500만 달러
💬 결정적 발언
"You don't need millions of bad files. You need less than 1% of right ones aimed at the right concept."
"No firewall gets breached and the tool never touches a single company server."
"The poisoners are winning for now, and the reason is lopsided."
⚖️ 화자가 다룬 반론·한계
- Glaze는 화풍은 숨기지만 이미지 자체의 도용(원본 유출)은 막지 못한다는 한계가 스스로 언급된다.
- 캡션 대조 필터링은 오염 데이터를 절반 정도만 걸러내며, 동시에 정상 데이터까지 함께 삭제하는 부작용이 있다고 지적된다.
- 대형 상용 모델(Midjourney, DALL-E 등)이 실제로 얼마나 취약한지는 외부에서 검증할 수 없다는 한계를 화자 스스로 인정한다.
- 음성 보호 기술도 충분한 후처리(프로세싱)를 거치면 보호 효과가 약화될 수 있다는 점이 언급된다.
🧩 핵심 용어
- 데이터 포이즈닝(Data poisoning) = 학습 데이터셋에 조작된 예시를 섞어 모델이 잘못된 패턴을 학습하게 만드는 기법
- 적대적 머신러닝(Adversarial machine learning) = 시스템을 직접 공격하지 않고 모델이 스스로 오학습하도록 유도하는 연구 분야
- Glaze = 화풍(스타일)을 AI가 잘못 인식하도록 이미지의 수치 구조를 은밀히 왜곡하는 도구
- Nightshade = 학습 시 모델이 개념 자체를 오학습하게 만드는 "트로이 목마"형 이미지 오염 도구
- Safe Speech = 음성의 AI 인식용 "지문"을 미세하게 흐트러뜨려 음성 복제를 방해하는 도구
🔎 보완 & 확인
- 배경 보완: LAION-5B는 독일 비영리단체 LAION이 공개한 이미지-텍스트 쌍 데이터셋으로, Stable Diffusion 등 다수 오픈소스 생성모델의 핵심 학습 자원으로 쓰였다는 것이 일반적으로 알려져 있다. Nightshade·Glaze는 실제로 시카고 대학 SAND Lab(Ben Zhao 교수팀)이 개발한 것으로 알려져 있다 (확인필요).
- 사실 유의점: 영상에서 언급된 "오염 이미지 50장/300장" 실험 결과는 시카고 대학 연구진의 학술 논문(Nightshade 관련 연구)에 기반한 수치로 보이나, 정확한 실험 조건(모델 버전, 파인튜닝 여부 등)은 자막만으로는 확인이 어려워 추정으로 남겨둔다.
- 사실 유의점: "2024년 클론 음성·영상으로 2,500만 달러 탈취" 사건은 홍콩에서 발생한 딥페이크 화상회의 사기 사건을 가리키는 것으로 보이나, 영상 자체는 구체적 지역·기업명을 언급하지 않아 정확한 사건 특정은 확인 필요하다.
- 사실 유의점: "캡션 대조 필터링이 40~60% 탐지" 수치의 출처가 자막에 명시되지 않아, 특정 연구 결과인지 업계 일반 추정치인지는 확인 필요하다.
자막 원문 (참고)
Kind: captions Language: en An AI takeover is inevitable, or at least that's what you've been told. Once you upload a photo or a voice note, it's gone forever, scraped into AI systems feeding an insatiable hunger for data. But what if that is not the full story? Right now, a digital insurgency is taking place. Traps are being laid inside the data these models depend on. Researchers have figured out how to lobotomize AI models using just a handful of modified JPEGs, and it works. So what happens when the scrapers don't just learn from the internet, but start breaking because of it? Let's say someone types a simple request into an AI model. They ask for a photorealistic dog running through the park. It's the kind of thing a confident model nails every single time. But this time, the results are different. [music] The legs don't connect properly. Extra joints appear where they shouldn't exist. The face subtly drifts into something that's from the uncanny valley. The fur loses structure and definition. It looks like something that has seen a dog a thousand times, but it doesn't actually understand what it is. [music] And that's the work of a poisoned AI model. It's not a bug. The model is still doing what it was trained to do. The problem is what it was trained on. A poisoned AI model is what happens when someone messes with the data set it learns from. They slip in manipulated or misleading examples during training, so that the model starts picking up the wrong patterns. No one [music] has to hack anything or break into the system. Artists and creators simply post their work online, like they always have. And then the scrapers arrive. These are automated bots that crawl the internet scooping up massive amounts of images, text, data, and audio from websites. They don't understand what they're gathering. [music] They just vacuum it up to build training data sets for AI models. That's where the problem starts. Mixed in with all that normal content, poisoned examples [music] get collected, too. Nothing looks wrong at the time. The trap only reveals itself later during the next training run. The team at the University of Chicago put that theory to the test. They fed [music] Stable Diffusion about 50 altered pictures of dogs, and then they put the AI model to work. Almost immediately, the images were warped. Every single one had something wrong with it. The team kept going. Once they got to around [music] 300 poisoned images, the model started producing images of a cat. The model trains on billions of images, but for [music] any single concept, say a dog, it really only relies on a few thousand of them. Corrupt a small slice and it all starts to fall apart. You don't need millions of bad files. You need less than 1% of right ones aimed at the right concept. These models aren't isolated. [music] They're powering Midjourney, DALL-E, and the image tools built into your phone. So, if one system becomes corrupted, the effects don't stay contained. They spread. It's the opening shot of a new kind of [music] fight, and the people doing it aren't rival labs or bad actors. They're illustrators, photographers, creatives, and ordinary users with a free app, and they all have a reason to be furious. But, before we go any further, imagine this. You're online every single day. You check your email, open a few apps, look things up, stream videos, maybe even do all of that while traveling. And to you, that feels totally normal. But, behind the scenes, your internet provider, advertisers, network admins, and sometimes even governments can build a surprisingly detailed picture of what you're doing online. Now, to be clear, no VPN can protect you from everything. You still have to be smart online. Don't click suspicious links. Don't hand over personal information to sketchy emails. And definitely don't trust the so-called [music] Prince of Nigeria. But, a good VPN is an important layer of privacy because privacy shouldn't be something you only think about after something goes wrong. It should be the default. And that's exactly what ProtonVPN is built for. [music] ProtonVPN helps keep your browsing private wherever you are, whether you're at home, traveling, or just trying to stop your online activity from being tracked. Their no-logs policy has been verified by independent auditors, and Proton [music] is backed by a foundation dedicated to privacy, transparency, and user rights. ProtonVPN is also fully open source, meaning its code is publicly available for inspection. So, instead of [music] just asking you to trust them, Proton gives people a way to verify what they're doing. And privacy doesn't have to mean slow. Proton VPN offers high-speed connections with VPN accelerator plus Netshield, which blocks malicious ads and trackers before they ruin your browsing experience. It also gives you more freedom online. You can connect through servers around the world, access [music] content more securely while traveling, and use it for streaming, gaming, and P2P file sharing, all while keeping your online activity more private. So, whether you're trying to stop advertisers from following you, keep your browsing [music] private from your ISP, or just take back control of your online life, Proton VPN makes privacy simple. Click the link in the description or go to protonvpn.com/theinfographicshow to get up to a 70% discount when you sign up to Proton VPN's 2-year plan. Nobody poisons their own work for fun. They do it because they felt like their work was taken from them. That feeling doesn't come from nowhere. It all started with a theft. For years, the firms building these models treated the open web as a warehouse. They scraped billions of images, voices, and paragraphs from it. The original creators were never asked, and they were never paid. It was a heist, plain and simple. Much of that haul went into one open data set called LAION 5B, which held a massive data set made up of 6 billion images and accompanying text. Stable Diffusion learned from it, and so did most of the art that followed. It was a compressed version of the visible internet folded into something machines learn from. Then the artists started paying attention. [music] They started recognizing their own work in the output. Styles and techniques that took decades to develop were being reproduced in seconds. [music] By 2023, a generator could mimic the brushstrokes and the talent of an artist in a heartbeat. A career's worth of experience was reduced to a prompt [music] and undercut by a tool that cost nothing to use. Conceptual artist Karla Ortiz, along with other artists, sued Stability AI in 2023. Getty Images filed its own case around the same time. The lawsuits dragged [music] out, and all the while the scraping continued and the models improved. The companies behind the models freely admitted to their practices. Open AI told the British Parliament that building today's top models without copyrighted work would be impossible. Sure, there were opt-out forms, but [music] it put the onus on the creator instead of asking permission up front. Artists had to hunt [music] down the models, file requests one by one, and then hope they were honored. The balance of power remained with the developers. So, the artists adapted. Ordinary uploads turned into weapons one file at a time. In 2023, the University of Chicago released a program called Glaze, and it was aimed directly at the AI models. Type in an artist's name into an AI model, and it'll generate a painting based on that particular style and the data it's learned on. Glaze [music] changes that. It works by exploiting how a computer sees an image. To an AI, a painting isn't really a painting at all. It's a series of numbers that make up the style and structure. Glaze distorts the numbers slightly. It's subtle, something that isn't visible to the human eye, but the machine learns from the cloaked information. It works like an optical illusion. Two viewers can look at [music] the same image and see completely different things. In this case, the viewers are you and the machine. You see your painting exactly as was intended. The model sees something else. An oil painting might seem like it's in charcoal. A watercolor might be seen as completely different medium. The visual style that the scraper came to learn from has effectively been moved. Word spread [music] fast, and Glaze is now past 6 million downloads, but there are limits. Glaze [music] could hide a style, but it couldn't stop someone from taking the image itself. And every time people found a new way to alter the image, scrapers came back with better ways to recover it. It was like playing defense forever. The creators needed something that could end the fight. They needed a poison pill. It came in January 2024, and it was Nightshade. Another product from the University of Chicago, it had one goal: to infect. An image run through Nightshade will look completely normal to both human eyes and the machine. But, it's like a Trojan [music] horse. If a machine is trained on that one image, the model starts learning nonsense. The results are almost comical. Hats turn into cakes, handbags into toasters, and the cars learn to be drawn as a cow. For artists and creators, it felt like payback. For years, AI companies had scraped artwork to train their models. Now, the very thing being taken could be turned against the people taking it. The demand was instant. Nightshade hit 250,000 downloads in just 5 days. University servers buckled under the traffic. The team behind it had to scramble to post backup download links as thousands more rushed to get it. Most people assumed this was a form of hacking. It's not. No firewall gets breached and the tool never touches a single company server. Researchers call it adversarial machine learning. But, a simpler way to look at it is that Nightshade is a magic trick for the machines. The model isn't attacked from the [music] outside. It is fooled into teaching itself the wrong thing, and then it trusts that mistake as if it were true. It doesn't take much to start the process. Fewer than 100 carefully crafted images can be enough to poison a single concept inside a top AI model. And if you pair Nightshade with Glaze, that same image can hide an artist's style and poison the training data at the same time. But, those results came from models researchers could test directly. The bigger AI systems are harder to study, and nobody outside those companies knows exactly how vulnerable they are. But, that is not really the point. Nightshade took the idea from a theory on paper to something that actually worked. The damage doesn't stay contained, either. Poison the concept of a dog and the related ideas like huskies, puppies, and wolves can start drifting with it. In some tests, researchers fed hundreds of corrupted images in a single model until it could barely generate recognizable images at all. And that's where this stopped being purely defensive. Every poisoned image uploaded to the internet becomes a potential sleeper cell. It can remain unnoticed inside a data set for months or years, waiting for the next training run. The person who uploaded it might never even know if it was collected or which model eventually learned from it. Images were only the start. The same trick works on anything a machine learns from, from your writing to your face, and your voice is the next target. A service like 11 Labs needs only a few clean sounds of your voice to build a convincing copy. A podcast clip can be enough. So can a YouTube video, a few seconds of footage on social media, or even an old voicemail buried in someone's phone. It's become a favorite tool of scammers, and it isn't a hypothetical risk. In 2024, criminals used cloned voices and faces to steal about $25 million in a single fake video call. Banks and law enforcement agencies now warn people about calls from cloned relatives asking for money. A tool called Safe Speech was built by security researchers for this exact reason. Your voice has its own signature, a kind of audio fingerprint that AI systems use to recognize and copy you. Safe Speech subtly smudges that fingerprint. To another person, you sound exactly the same, but to a voice cloning model, you become much harder to copy. To the machine, it works a bit like radio interference. You hear the voice clearly, but the AI doesn't. The details it needs most get scrambled. Train a voice clone on bad recording, and the results come out wrong. They're close enough to sound human, but missing the subtle traits that make you sound like you. The timing is what makes it clever. This isn't a filter slapped onto a fake after it's generated. It's in the original recording itself. The AI learns from the protected version, which means the cloning process breaks before it ever succeeds. There is no clean copy for the model to learn from. That's what makes this different from protecting artwork. A stolen portfolio is one kind of loss. Your voice is another. It's the sound your family and friends recognize instantly, the thing a scammer wants the most when they're trying to impersonate you. It turns the tables. The target gets to set the trap. For For first time, artists, writers, and everyday people had a way to fight back against systems trained on their work and identities. It sounded like the next step in digital security. And then, the AI labs responded. They weren't about to lose access to the data that fueled their models. If artists [music] could poison the training set, the labs would try to remove the poison. And that kicked off something bigger than a security tool, an arms race. They couldn't [music] just accept corrupted data sets, and they couldn't afford to throw most of them away, either. So, they tried a straightforward fix. They [music] began checking each scraped image against its caption. And then, they dropped anything that didn't match. In theory, poisoned data should stand out. In reality, it only catches part of it. Tests on these systems show it flags maybe 40 to 60% of poisoned images. >> [music] >> The rest slips through. And then, there's a second problem. It also deletes plenty of clean data, leaving the model with either contaminated data or not enough to learn from. So, they tried something else. They started scrubbing every image through a cleaning model before training, trying to wash out anything suspicious. It sounds clever, but it's slow and expensive. And [music] it still has limitations, because the poisoning adapts. Artists design it to survive that cleaning step, [music] so it reappears on the other side like a stain bleeding back through. Not long after Glaze launched, one research group said it had already broken through the cloaking. So, the Chicago team pushed out a tougher version. Whenever the program was breached, they developed a new patch. It was a war of attrition. Even if a voice has protection built in, there are still ways to strip parts out of it. Push it through enough processing, and some of that protection starts to fade. In some cases, it gets noticeably weaker. Not gone, but damaged enough to matter. The poisoners are winning for now, and the reason is lopsided. You have small teams constantly trying new tricks in the open. On the other hand, a handful of AI labs are trying to patch holes as they appear. So, every time a lab rolls out a fix, it gets tested immediately, and almost immediately a new version shows up, adjusted just enough to get around it, usually within weeks. And this all comes with a price tag. It doesn't land on the labs first, it lands on the people training the models with the data. The whole AI business model was built on the promise that the data would be free, endless, and clean. The poison breaks the idea of clean. Once that goes, the other two stop feeling true as well, because now nothing can be trusted at face value. A poisoned file looks identical to a real one. There's no label or warning, so every data set turns into something that has to be checked manually or with tools that still miss a lot. Training a top model already costs hundreds of millions of dollars, and every extra cleanup step adds to that bill. A single poisoned batch can set a project back by weeks, and that's the point. The goal was never to wreck one model for fun. It was to make [music] stolen data more expensive than paid data. There's already a legal cleaner method being used. Adobe trained its Firefly tool on Midjourney images, Shutterstock on similar deals. If you want clean, quality data, you need to pay for it. That is a serious problem for the industry. Investors [music] poured tens of billions into AI on a single bet that the data would stay cheap forever. That bet isn't looking like a sure thing anymore. For a long time, the belief was that if you created something, you own it. The scrapers tore that deal up without asking anyone. The AI companies act as if human work was free, endless, and owned by no one. That was the original mistake. Paintings were scraped, photos were downloaded, voices, faces, videos were collected in huge amounts. If it was online, it was fair [music] game. Consent seemed optional. In every other industry, the opposite is the standard. A photographer licenses an image before a brand uses it. Musicians clear samples before a track goes live. Even film studios pay for every frame of stock footage they use. The work of millions of people got treated as if it belonged to no one. This battle wasn't created by the artists. It was the AI labs and their trainers. The moment work was taken without asking, the balance changed. Everything after that was just a reaction. You can't build an empire on the idea that people do not count [music] and then act shocked when those same people fight back and question how the system works. Because the problem isn't sitting in a contract or on a policy page, it's built into the way the models learn in the first place. Every safeguard before this tried to politely control behavior through copyright notices or opt-out [music] forms, but none of it really stopped the scraping. This is different. For the first time, a single person can influence what the system learns next and the system doesn't get to ignore it. With people turning the tables on AI scrapers, digital theft is getting harder, especially when it comes to scams. But this isn't the first time people have had to push back. Watch Time Scammers [music] Messed With the Wrong People to find out what happens when karma is dished out or click on this video.