A patent application filed by Sony Interactive Entertainment (SIE) for technology that uses AI to predict the next frame of a game was published in the United States on October 15, 2026. The publication, titled "Generation Super Sampling", describes a mechanism that displays a predicted image, generated from past images and controller input, before the GPU has finished rendering the next frame. Unlike conventional interpolation, which generates an intermediate image from two actual frames, this approach can display an image without waiting for the next frame to be completed. That could reduce the delay before an image appears, but prediction accuracy and display control when switching to the actually rendered frame remain challenges.
AI generates an image before the next frame is complete
The published application is numbered US 2026/0320740 A1. The applicant is SIE, and the inventor is listed as Denis Dupeyron. The filing date is June 23, 2026, but the cover page states that it is a continuation of U.S. patent application 18/393,547, filed on December 21, 2023. The publication therefore does not mean that Sony first conceived of frame prediction technology in October 2026. It is also only the disclosure of a patent application, and it does not mean the technology has been confirmed for any product.
The core of the technology is a configuration combining an encoder and a decoder. The encoder extracts features from input images and converts them into numerical data. The decoder then uses that information to predict and generate the image that should appear at the next point in time. The filing also describes configurations that use not only the latest image but also earlier images and "motion vectors," which indicate in which direction and by how much objects on screen have moved.
The AI is trained on temporally continuous video. The model learns to predict the next image from past images, and it is trained so that the difference from the actual video becomes small. Unlike a typical autoencoder, which reconstructs the input image itself, it is trained to generate an image that comes later in time. Because the training video contains the actual next frame, that frame can serve as ground-truth data to compare against the prediction. The publication also describes a configuration that predicts several future frames at once.
The processing flow of the trained model is explained in Figure 4 of the publication. The AI generates a predicted image while ordinary graphics processing renders the next actual frame in parallel; the predicted image is displayed first, followed by the actually rendered frame. Instead of waiting for the GPU's normal rendering to finish, the aim is to have lighter prediction processing bring the display forward. Whether latency can actually be reduced, however, depends on making the AI prediction processing fast enough.
The difference from conventional interpolation: not waiting for the next actual frame
In its official technical explanation of FSR Frame Generation, AMD describes a mechanism that uses machine learning to generate a frame positioned between two consecutive input images. Both approaches use AI to generate new frames, but they differ significantly in the information needed to generate an image and in when that information becomes available.
The biggest difference is whether the system must wait for an actual frame later than the moment being generated to be completed. Comparing the method in Sony's publication with AMD's machine-learning-based FSR Frame Generation gives the following:
| Comparison item | Sony's frame prediction technology | AMD's ML-based FSR Frame Generation |
|---|---|---|
| Images used | Already-rendered images; some configurations use multiple past frames | Two consecutive images before and after the frame to be generated |
| When the image is generated | Before the next actual frame is complete | After the actual frames before and after are available |
| Role of the AI | Predicts a future image from past information | Interpolates intermediate motion from the images before and after |
| Main technical challenge | Must accurately predict scene changes that have not yet been rendered | Must wait for the later actual frame to be completed |
This comparison is based on Figure 4 and paragraphs [0035] to [0037] of Sony's patent publication and on AMD's explanation of "ML-powered frame interpolation." It organizes the required input images and the timing of their use from materials publicly available as of October 19, 2026, and it does not compare the actual performance of the two methods.
Sony's publication also describes a method of predicting the next image without AI. It detects the motion of blocks, which are groups of pixels, from past images, assumes they keep moving in the same direction and at the same speed, and shifts their positions to a later point in time. In other words, whether to predict a future image from past information or to interpolate an intermediate image from actual images before and after, and whether to use AI for that processing, are separate design choices.
It is also necessary to distinguish between an increase in the number of frames shown on screen and improved responsiveness to game controls. For example, if a game renders at 60 frames per second (60 fps) and one generated image is inserted between frames for display at 120 fps, the screen refresh interval shortens from about 16.7 milliseconds to about 8.3 milliseconds. However, this is only a calculated example, obtained by dividing 1,000 milliseconds by the frame rate, and is not a measured value for Sony's method. Even if the video refreshes more often, in-game collision detection and confirmed game state are not necessarily updated at the same rate, so smoothness of display and responsiveness to input must be evaluated separately.
Using controller input to improve prediction accuracy
Sony's publication also describes a configuration that uses controller input in addition to past images. This is to reflect in the next-frame prediction the player's actions, which cannot be determined from images alone. For example, if a player presses the jump button, a jumping motion by the character can be expected to appear in the following few frames. Changes that are hard to predict just by reading motion trends from past images may be estimated more accurately by using operation information that has already been input.
While motion vectors indicate the direction and distance objects have moved so far, controller input supplements this with information about what action the player is currently commanding. However, using controller input is not required and is envisioned as additional information. The mechanism also does not foresee buttons the player will press in the future; it estimates the next image from operations that have already taken place. The publication also describes a method of finding similar blocks within images to calculate motion vectors, so it is not limited to using movement information output by the game engine.
Even when the control input is known, however, it is not always possible to accurately predict everything that will appear on the next screen. For example, if a player moves the camera sideways and a place previously hidden behind a wall comes into view, past images may not contain information about the newly revealed background. Nor is it possible to accurately infer an opponent's latest actions from one's own controller input. Even if AI can generate an image sooner than the actual frame, whether its content matches the actual game state is a separate question.
Such challenges have also been pointed out in earlier research. ExtraSS, presented at SIGGRAPH Asia 2023 by researchers from Intel and the University of California, Santa Barbara, is a technique that combines super-resolution with frame prediction. The research provides dedicated processing for ghosting that occurs when previously hidden regions become visible, and for changes in shadows and reflections that are hard to handle accurately with object motion vectors alone. It shows that simply moving objects in past images cannot correctly reproduce the next screen.
A follow-up study, GFFE (arXiv:2406.18551), also lists as challenges the lack of information for regions that were hidden and the implementation work required to obtain additional rendering information from the game engine. These studies do not show the technology Sony adopted in this patent or its performance. They do, however, offer clues to what problems must be solved when introducing frame prediction into actual games.
AMD also recommends that the frame rate before FSR frame interpolation be at least 60 fps, explaining that image artifacts become more noticeable as the original frame rate drops. This recommendation cannot be applied directly to Sony's prediction method, but it is an example showing that simply increasing generated frames does not necessarily solve the problems that come with a low frame rate.
Reducing latency also depends on controlling display of predicted and actual frames
Sony's patent publication also describes a mechanism for controlling the display order of predicted images and actually rendered frames. Paragraph [0037] describes a method in which, even after rendering of the next actual frame is complete, display of that frame is delayed until the previously generated predicted images have finished being displayed. This is to prevent an actual frame from interrupting partway through the sequential display of multiple predicted images and disrupting the display order. Therefore, even if predicted images can be generated without waiting for the next actual frame, the waiting time associated with display control will not necessarily disappear entirely.
Paragraph [0038] further proposes a method of comparing pixels of the predicted image and the actual frame and not displaying the actual frame if the two match or the difference is smaller than a predetermined threshold. In this case, the actual frame whose display was skipped can be used as input for subsequent predictions. However, this is control over whether to show an already rendered actual frame on screen, not a technique that skips the normal rendering process itself.
What matters in this patent is that, unlike conventional frame interpolation, it can display an image before the next actual frame is complete, and that it also considers how to handle the actual rendering result afterward. However, AI prediction processing also requires computation time, and if it shares computing resources such as the GPU with normal rendering, that load must be taken into account. Moreover, even if a predicted image can be displayed early, it will not necessarily improve the gaming experience if the correct screen is not shown in response to input. Both prediction accuracy and display latency must be evaluated.
The publication as currently available does not include test results on input-to-display latency measured on specific hardware or on the quality of predicted images. Nor is it clear whether the technology is planned for the PS5 Pro or a future PS6, or when it might be put to practical use. The publication of a patent application therefore cannot be taken as grounds for concluding that Sony's consoles will gain a new frame generation feature.
If it is adopted in an actual product, important evaluation items will include comparisons with frame prediction enabled and disabled in the same game and at the same graphics settings, as well as image quality during rapid viewpoint changes and in scenes where predictions are likely to miss, and input-to-display latency. If accurate images can be displayed earlier than before under such conditions, frame prediction could be expected to improve responsiveness to player input, rather than merely making video look smoother.