Serpentine is a highly stretchable, self-powered sensing interface equipped with signal processing for accurate and energy-efficient gesture recognition. This practical solution can transform wearables into gestural sensing interfaces without additional instruments needed. Fabricated in a coil-like structure, the shape-conforming and durable form-factor of the interface enables a broad variety of expressive input modalities from an extensive range of inputs. The six specific interactions evaluated by researchers—pluck, twirl, stretch, pinch, wiggle, and twist—exhibit 95.7% accuracy for a user-dependent recognition model using a real-time system and 92.17% for user-independent online detection. This Georgia Tech innovation is also self-powered, leveraging triboelectric—or friction-based—nanogenerators instead of an external power source or external sensing instruments.
Serpentine’s flexibility and small size makes it ideal for integration into wearable, on-body human-computer interface applications to allow for natural and unobtrusive interactions. Beyond its potential for gesture recognition and assistive technologies, other applications could include human motion and health monitoring, structural health monitoring of buildings, and even electric musical instruments.
- Self-contained: Includes a signal processing pipeline, providing gesture recognition for an extensive range of input modalities without the need for external power or sensing equipment
- Assistive: Enables eyes-free, unobtrusive, and ergonomic interaction with computing devices embedded in wearables
- Versatile: Demonstrates far-reaching potential for applications beyond gesture recognition including in actuation and data acquisition
- Accurate: Exhibits 85% accuracy in gesture classification across different users with a wide range of expressive gestural inputs and almost 96% accuracy for the six specific interactions evaluated.
- Self-powered: Uses a triboelectric nanogenerator (TENG) and works on the principles of electrostatic induction and triboelectrification to generate electric signal; gestural inputs create electric signals, and increased signal amplitude results in increased harvested power
- Affordable: Assures scalability of system for a wide range of applications with a low- cost, simple, and safe fabrication process
- Sport wearables and other accessories (e.g., hoodie bands, shoelaces, necklaces, etc.)
- Human health monitoring
- Building structural health monitoring
- HVAC integration (e.g., tracking of air pressure differential)
- Musical instruments
Old and analog sensing interfaces, such as light switches, volume knobs, and video game joysticks are being replaced by touch- and voice-based sensing interfaces. Over the last few decades, there have been numerous efforts to develop novel tangible interfaces with a variety of form factors and sensing modalities to keep up with growing demand (e.g., analog light switches/dimmers are being replaced by advanced touch-based lighting controls). New interfaces have been developed with active textiles as deformable/stretchable reactive objects, but they have seen little commercial success. While new sensing modalities have attempted to use optical, resistive, capacitive, piezoresistive, piezoelectric, and other innovative form factors, they still rely on some form of external power. Most forms of external power require either bulky and cumbersome power sources or cables running to power sources, greatly reducing the use environments of the sensing interfaces.
Advancements in sensing interfaces must maintain the deformable/stretchable properties capable of sensing a range of expressive interactions, while also having self-powered capabilities. The elimination of an external power source will greatly expand the design potential of sensing interfaces and expand their applications in industries such as consumer electronics, smart homes, Internet-of-Things devices, smart textiles, home appliances, and video games.
Serpentine cord prototype demonstrating six user interactions including two levels of waveforms (first red, then green) illustrating intensity for each interaction: 1) double pinch, 2) twirl, 3) pluck, 4) stretch, 5) twist, and 6) wiggle. The y-axis represents open circuit voltage in volts, and the x-axis is time in seconds.
Serpentine sensor prototype
Sensor fabrication with approximate dimensions
Applications of Serpentine include: (a) wearable hoodie strings, (b) expressive switch, and (c) slingshot
Various interactions suggested by the users at the beginning of the user study. Green borders highlight interactions implemented by the study. The numbers shown by each interaction represents the frequency suggested by the 12 participants.