Damaged Traffic Signs Are Fooling AI and Self-Driving Cars

Researchers in South Korea have developed an artificial intelligence framework that uses realistic wear and tear on traffic signs to test vulnerabilities in computer-vision systems, raising questions about whether everyday deterioration such as fading, cracks and corrosion could cause AI models to misclassify signs.

The framework, called Adversarial Wear and Tear (AdvWT), was developed by researchers from Kyung Hee University and Seoul National University of Science and Technology (SEOULTECH). The researchers say it can be used both to expose weaknesses in AI systems and to improve their ability to recognize objects that have deteriorated naturally.

The study does not show that a deployed autonomous vehicle has been fooled by a naturally damaged traffic sign. Rather, it demonstrates the vulnerability in traffic-sign recognition models under controlled digital and physical experiments, an important distinction as automakers and technology companies work to validate automated-driving systems in increasingly complex environments.

“We focused on traffic signs because they are exposed to weather and environmental damage throughout their lifetime, and their accurate recognition is essential for safety-critical applications,” says Dr. Lee. “Unlike temporary optical attacks, natural deterioration can persist until a physical object is repaired or replaced.”

The findings were published in May in IEEE Transactions on Dependable and Secure Computing.

The research comes as computer vision becomes an increasingly important component of automated-driving and advanced driver-assistance systems. The U.S. Department of Transportation has identified AI and computer vision as foundational technologies for automated driving, while research on automated-vehicle infrastructure has warned that the condition and placement of road signs and markings can affect the reliability of automated vehicles.

A 2022 SAE research report said road conditions, lane markings and signs provide essential guidance to both human and machine-controlled vehicles, but that their condition and placement vary with local conditions, practices and funding. It said irregular sign placement and road markings can confuse both human and machine drivers and create unsafe conditions.

That makes the condition of physical infrastructure an issue beyond cybersecurity. An autonomous vehicle must not only resist deliberately manipulated objects but also interpret the imperfect roads, signs, markings, lighting and weather conditions it encounters during normal operation.

The SEOULTECH and Kyung Hee University team, led by Associate Professor Seong Tae Kim and Assistant Professor Hong Joo Lee, trained a generative image-to-image translation model based on StarGAN-v2 to learn visual patterns associated with damaged and undamaged traffic signs.

The model learned a latent representation of “damage style” that could reproduce different forms of deterioration while preserving the sign’s identity and meaning. Researchers then adjusted the representation to generate signs that appeared naturally degraded but were more likely to be misclassified by AI systems.

In a study involving 32 people, images generated by AdvWT received high naturalness ratings and were perceived as similar to real damaged traffic signs.

The researchers evaluated the framework using two traffic-sign datasets and eight recognition architectures. AdvWT achieved near-perfect attack success rates against lightweight convolutional neural networks such as ResNet-18 and MobileNet and also remained effective against transformer-based models.

The researchers said the framework recorded the highest average transferability across most tested model combinations, indicating that the adversarial effects could carry over between different AI architectures.

The finding builds on a growing body of research showing that computer-vision systems can be vulnerable to physical changes that humans may regard as insignificant.

In a landmark 2018 study, researchers including Kevin Eykholt, Bo Li, Dawn Song and Tadayoshi Kohno demonstrated that specially designed stickers could cause a deep-learning system to misclassify a real stop sign. In laboratory tests, their attack caused targeted misclassification in 100% of captured images, while a field test involving a moving vehicle produced the target misclassification in 84.8% of video frames.

Other experiments have similarly shown that physical adversarial objects can affect computer-vision systems outside laboratory environments. In one USENIX study, researchers reported that a YOLO-based detector failed to recognize adversarial stop signs in more than 85% of laboratory video frames, while outdoor poster and sticker attacks fooled the detector in 72.5% and 63.5% of frames, respectively.

AdvWT differs from those approaches because it attempts to make the attack resemble something that could occur naturally. Instead of deliberately applying a conspicuous sticker or pattern, the framework modifies a learned representation of damage so the resulting sign looks like one that has deteriorated over time.

The researchers also tested whether the effects would survive outside a purely digital environment. They printed clean and adversarial speed-limit signs and photographed them at different distances and viewing angles under indoor and outdoor conditions.

The resulting images continued to mislead traffic-sign classifiers, suggesting that the effects survived the printing and photographing process and remained effective under varying physical conditions.

The significance of such experiments is partly about how automated-driving systems are tested. The National Transportation Safety Board has said investigations of developmental automated-driving systems have exposed functional limitations, particularly in detecting hazards and predicting the movement of road users. It has also said the safety implications depend on the redundancies and risk-mitigation strategies built into the systems.

The NTSB has separately warned that partial-automation systems have limitations in detecting hazards and maintaining an appropriate travel path, underscoring the need to understand the operating conditions under which automated systems can be relied upon.

Still, the existence of a vulnerability in an individual vision model does not necessarily mean an autonomous vehicle would make the same mistake on the road. Commercial automated-driving systems can combine multiple cameras and other sensors, maps, vehicle-to-infrastructure information and other layers of redundancy.

Waymo, for example, says its automated-driving system had accumulated more than 220 million fully autonomous miles through March 2026. The company’s latest analysis found 94% fewer crashes causing serious or fatal injuries, 82% fewer crashes involving airbag deployment and 82% fewer crashes involving any reported injury compared with human drivers in the same areas and period. Those figures are company-reported and are not evidence that individual perception vulnerabilities do not exist.

Waymo has also argued that autonomous driving cannot depend on a single technological approach. The company says it has learned from more than 200 million miles of driverless testing that there is no shortcut to safe autonomy and that multiple technologies and safety measures are needed.

The researchers said the framework could also have defensive applications. The same bidirectional model was used to restore naturally damaged traffic signs, while training recognition models with AdvWT-generated damaged signs improved their ability to generalize to real-world damaged signs.

That defensive application may ultimately be more important than the attack itself. Rather than treating adversarial examples solely as a security threat, researchers are increasingly using them as stress tests to expose weaknesses before systems are deployed.

The U.S. Department of Transportation has also identified computer vision as a potential tool for monitoring transportation infrastructure. A 2024 federal AI research report noted that computer-vision models could be used to conduct high-frequency assessments of transportation assets and detect missing or damaged safety-critical infrastructure, potentially allowing faster maintenance responses.

This creates a potential feedback loop: AI could be used to identify deteriorating road infrastructure, while similarly damaged infrastructure could be used to test whether AI systems remain reliable.

“Building reliable AI requires more than improving average performance. It requires continuously identifying where AI systems fail, understanding why those failures occur, and using those insights to make the systems more robust. Over the next five to ten years, research in this direction could play an important role in developing AI systems that can be deployed more reliably in real-world, high-stakes domains such as healthcare and finance,” says Dr. Kim.

For autonomous driving, the findings reinforce the need to evaluate perception systems under conditions that resemble the real world rather than relying exclusively on clean benchmark images.

A recent review of adversarial threats to traffic-sign recognition described traffic-sign recognition as a safety-critical component because the information it extracts can directly influence vehicle planning and control. The review also highlighted the growing range of adversarial attacks and the need for defenses that improve robustness and interpretability.

The challenge is particularly relevant because road infrastructure is rarely static or perfectly maintained. Signs can fade under sunlight, become obscured by vegetation or dirt, lose reflective properties, crack, corrode or be partially damaged in collisions. Weather can also degrade the visual information available to cameras.

Earlier U.S. government research on automated vehicles similarly identified weather and visibility as conditions that can push automated systems outside their verified performance envelope.

The result is a broader question for autonomous-driving developers: how much degradation should a vision system tolerate before it decides that its perception is unreliable and hands control back to a human or activates another safety mechanism?

AdvWT does not answer that question. But by turning natural deterioration into a systematic test, the researchers provide another way to measure where AI vision systems begin to fail.

The approach could potentially be extended beyond traffic signs to other safety-critical vision applications where physical deterioration affects how AI systems interpret objects, although further research would be needed to determine whether the method generalizes beyond the datasets, models and controlled physical tests used in the study.

Stay ahead of the Stories shaping our world. Subscribe to Impact Newswire and join our 
WhatsApp Channel for updates on global tech, business, and innovation—all in one place.

Dive deeper into the future with the Cause Effect 4.0 Podcast, where we explore the ideas, trends, and technologies driving the global AI conversation.

Got a story to share? Contact Us to reach a global audience with Impact Newswire.