A research team at Seoul National University has developed "Ouroboros," a system that tracks moving subjects and backgrounds within video and reuses previous computation results. In video analysis using Vision Transformers (ViT), even when the same content persists across consecutive frames, movement of subjects or camera shake shifts the position of patches, often triggering recomputation. Ouroboros reads motion from the motion vectors generated by the video encoder and treats the content as occupying the same coordinates. In evaluations on NVIDIA Jetson Orin, the team reports reducing computation by up to 87.0%, speeding up inference by up to 2.61x, while keeping accuracy loss in object detection and instance segmentation under 1%.
The research was authored by Chanjeong Park and colleagues, presented as a paper at ACM MobiSys 2026, with Seoul National University announcing the results on July 14. The goal is not simply to redesign lighter models for small devices. Rather, the work pushes into how far information remaining across adjacent video frames can be reused in a form usable by ViT.
87.0% Reduction Measured on Jetson Orin
This evaluation ran a ViT-based video AI on NVIDIA Jetson Orin. According to Seoul National University, in addition to cutting computation by up to 87.0%, energy consumption also dropped by up to 64.5%. Simply reducing the number of computations doesn't necessarily improve speed or power on edge devices if memory access increases. The research team states that by structuring the system to compute only the changed parts while carrying over the rest from the previous frame, this cost was also kept in check.
An 87.0% reduction means computation dropped to 13.0% of the amount required when processing every frame in full. However, a reduction in computational operations doesn't directly translate into a proportional speedup in inference. This is because devices also spend time reading and writing tokens from memory. Ouroboros was designed with this pathway in mind as well, translating into a measured speedup of up to 2.61x.
The object detection and instance segmentation tasks targeted here involve, respectively, identifying the position and category of objects, and separating individual outlines for each instance. Both are close to applications dealing with continuous video, such as road footage, surveillance cameras, and robot vision. The result of keeping accuracy loss under 1% demonstrates the effect of selecting where to skip computation based on content after movement, rather than fixed positions on the screen.
Reusing Based on Motion Rather Than Position
Conventional lightweighting methods have often determined which regions to skip computing based on differences from the previous frame or on identical pixel positions. However, when the camera shakes or a car or person crosses the frame, the visual information itself remains, but it moves to a different location within the patch grid. Recomputing every time this happens fails to exploit the temporal redundancy inherent in video.
Ouroboros estimates an affine transformation from the motion vectors produced by the hardware video encoder, aligning consecutive frames to a common global coordinate system. It determines which information from the previous frame corresponds to a moved car or background, and uses past computations for parts that haven't changed. The motion vectors referred to here represent information about which direction and how far blocks moved in compressed video, not an input for adding a new image recognizer.
To avoid discarding content that crosses the edge of the frame, the team also introduced a torus-like treatment that connects the top, bottom, left, and right of the input space. Positional information that becomes disordered when crossing a boundary is rearranged and corrected. Since ViT references both patch content and positional encoding, the meaning of reused tokens would be thrown off unless content tracking and positional encoding adjustment are combined.
A Cache Design That Computes Only Changed Locations
Aligning coordinates alone does not make ViT computation lighter. Ouroboros adopts a partial computation approach that processes only the changed tokens while using the Keys, Values, and final-layer features obtained from previous frames via a cache. This increases the regions that don't need to be passed through the full frame every time, creating a mechanism where computation reduction translates into actual gains in processing speed and energy consumption.
Instead of continuously streaming entire video to a server, increasing the information that can be reused on the device also reduces the data sent over the network. Seoul National University states that even in server-processing scenarios, the system achieved higher accuracy with less network bandwidth than existing methods.
Complex Motion and Device Dependency Remain
However, the 2.61x speedup and 64.5% energy reduction are peak figures obtained on Jetson Orin. Whether the same ratios hold on smartphones, drones, or robots will be affected by changes in ViT configuration and video codec. Memory bandwidth and how motion unfolds in the video also matter. It also presupposes that compatible motion vectors can be extracted from the device's hardware.
The public review for ACM MobiSys pointed out that a single global affine transformation struggles to accommodate scenes where multiple objects move independently, subjects deform, or appearance changes due to depth differences. Because ViT's self-attention depends on all tokens, how accurately partial cache updates can be handled remains a verification challenge. What the research team should demonstrate next is where accuracy, latency, and energy consumption break down in the face of complex motion and device variation.