Add Video Super Resolution for Windows x64 (AMD, Intel and NVIDIA), Windows ARM (Snapdragon), Linux and MacOS#1557
Add Video Super Resolution for Windows x64 (AMD, Intel and NVIDIA), Windows ARM (Snapdragon), Linux and MacOS#1557linckosz wants to merge 1 commit intomoonlight-stream:masterfrom
Conversation
|
Here is a summary of my observations following many tests conducted with different settings and multiple GPUs. AMD delivers the best results, followed by Nvidia, then Intel. VSR (Video Super Resultion) performs exceptionally well in SDR, but HDR setup (in the code) is challenging, and some specifics feature (like sharpening) aren't yet avaible due to drivers limitation. I would advise using the Video Enhancement feature without HDR. 1) Nvidia RTX 4070 Ti: 2) Intel Arc A380: 3) AMD RX 7600: 4) Intel UHD Graphics Xe-LP (iGPU) Does the feature "Video Enhancement" have an impact on the performance? Does HDR work with Video Enhancement? Below are my suggestions for different scenarios with varying settings from host to display (Host->Stream->Display): My suggestion for Office: 1440->1440->1440 My suggestion for Games: 1080->1080->1440 GPU comparison (1440->720/1080->1440 / 150 Mbps / Static picture):
In-motion block artifact comparison at low bitrate (1440->1080->1440 / 20 Mbps):
In-motion GPU comparison: When upscaling (stream at 720 or 1080 to render on a 1440 display), in all scenarios AMD provides the best result. When we use native resolution (stream at 1440 to render on a 1440 display), v5.0.1, AMD and Nvidia are all similar, only Intel still give a pixelized picture.
Resolution comparison (Nvidia Quality 4):
720p GPU comparison (1440->720->1440 / 150 Mbps):
1080p GPU comparison (1440->1080->1440 / 150 Mbps):
|
|
Here is a comparison on a Macbook M1 Pro.
|
|
There is a lot of reformatting going on and it makes it difficult to review the changes. Can you disable whatever auto-formatting it is that you're running? Removing trailing spaces if there are any is OK though. |
|
@andygrundman , |
|
Sure, stuff like this, where there is no actual change, just reformatting. It looks like it's trying to line things up to the brace for some reason. The only time you want to auto-format existing code is is if a project is already using a tool and there is a config file in the repo (like Sunshine which has a .clang-format). And the one exception of trailing spaces which should be fixed (that comes from the olden days when you might send a patch via email where trailing spaces would get lost and cause annoying conflicts). 
|
|
@andygrundman , |
|
Is there any test build with this feature available for Windows platform? Where can I download it? |
Here |
Thanks a lot. I'll check it :) |
|
After downloading the Windows build I've done some rapid tests and it doesn't seems to be working at all in my case. My setup is as follows: Host machine: Running at 1440p HDR (AMD Radeon Adrenaline driver) First I've enabled the new check box to activate the Upscaler enhancer. Connected to host with no issues but however the overlay info doesn't reflect any upscaler feature (pressing CTRL + ALT + SHIFT + S) Then to compare, i've deactivated the checkbox so the stream should be pure 1080p without any enhancement. To my eyes the image quality is identical, there is no difference. So I believe that I must be doing something wrong or the feature is not working. It could be helpful if, at least, the streaming overlay info could display something to know if the enhancer is being used or not. I've also tried lowering the host machine to 1080p while keeping the client working on 1440p and always selecting 1080p in Moonlight settings, but no difference. May be I'm missing something or may be the Intel GPU is not compatible with this experimental feature. |
|
Can't get this working. Is this feature using this library: https://github.com/OpenVisualCloud/Video-Super-Resolution-Library ? or it's only enabling some features already present into the GPU drivers? If this depends of the installed drivers then it should detect it before displaying the checkbox or at least display any warning, because right now I can just enable or disable it and the image quality is the same, it's just identical, no changes. |
Hi @fidoboy , If you are able to check the box, it means that your iGPU has this feature capability, otherwise it will be greyed out. When you stream at a lower resolution than your screen, make sure you use borderless fullscreen or fullscreen, select hardware acceleration and that the overlay stats is displaying "AI-enhanced" like below while streaming. |
The CPU is Intel Celeron N4500 and I can check or uncheck the checkbox, it's not grayed out. But It doesn't display that overlayed info like yours. Mine shows HEVC 10bit only but there is not "AI Enhanced" text when the feature is enabled. What else can I try? |
|
I'm using this build: https://ci.appveyor.com/api/buildjobs/fntsewbjxekye2v2/artifacts/MoonlightPortable-x64-r2725.zip The new checkbox is being displayed, it's not grayed out but I does nothing. The image quality is identical when the feature is activated or when it's not and the overlay stats does display nothing about "AI-Enhanced" I've done my tests with 1440 -> 1080 -> 1440 and also 1080 -> 1080 -> 1440. There is no changes when the feature is activated, image is exactly the same I'm sorry but I doesn't have Chrome to test. I never use that browser |
Yes, that the correct version. |
I forgot to say that I've also tried without HDR enabled in moonlight settings. The result is the same. There is no "AI-Enhanced" text in the overlay info. |
|
This is with HDR enabled:
and this is with HDR disabled:
in both cases the AI Enhacer is checked but it seems that it's not being used |
|
Works on macOS 15.4 with m3 cpu. Tested with 1080p host → 1080p stream → 1080p display, at 150mbps. It would be great to have a slider to select how much the image is sharpened, it looks over sharpened right now imo. Default: AI-enhanced: The added latency might be an issue on fast-paced games. |
Hi @jasperaelvoet , MTLFXSpatialScalerDescriptor documentation |
|
This feature is awesome! |
Great! I'm impatient to test it. I'll report back my findings. |
|
Well, I spent some time doing new tests with this feature and, in short terms, this is a placebo. There is no difference between the image quality when it's enabled or disabled. I've tried the following setups:
In both cases HDR was always enabled and in both cases I've compared between the video upscaler enabled and disabled. I have verified it because the text "AI Enhanced" was being displayed into the overlay stats. There was no any difference between them in the terms of image quality, the clarity, sharpness, etc. are exactly the same. And then I've tried the setup 1440 -> 1440 -> 1440 and the difference is very noticeable. So, in short, now the text is being displayed into the overlayed info but THERE IS NO ANY CHANGE IN IMAGE QUALITY. The image is the same if the feature is activated or not. There is no difference. |
I'm already laughing because your post is a real nonsense. What's the point to use this feature if you are streaming with the same resolution than your display? If the device where moonlight is running already uses 1080p and the host and also the stream is 1080p, where is the purppose for a video upscaler? This only have some sense if the stream is a lower resolution than the display, but it doesn't do anything if the resolution is the same. |
I think it would be beneficial if you substantiated your observations with screenshots on vs off so contributors can see what exactly you see. In the other comparison screenshots there are discernible differences. But whether you notice can be quite subjective. |
linckosz
force-pushed
the
master
branch
9 times, most recently
from
57d8c70 to
09c3dec
Compare
|
@andygrundman , https://www.youtube.com/watch?v=SSlJy0rwpkM&list=PLP_dd27EzwEgdjja5eAHBYtivM6hcG111 |
|
Trying your latest AppImage artifact on my laptop. Don't working( Moonlight log
$ ./Moonlight-09c3de-x86_64.AppImage
|
|
I reread the discussion and realized that Linux support isn't intended. But why? |
@nhths Thanks for your feedback. You are right, Linux is not yet supported, it is just a matter of time for me to get familiar with Vulkan and the way to use shaders. So far, only FSR1 (Shader version) seems to work for Linux. During weekends I try to progress on it, no deadline in sight for the moment, but I know I'll make it work. |
linckosz
changed the title
|
Good news, the Linux version with Video Super Resolution is ready! 🎉 Thanks to Vulkan and Below are screenshots of a 720p stream upscaled to a 1440p display: Linux (original) Linux (with FSR1) |
|
Thank you for your significant and valuable work on Moonlight and implementing FSR support on Linux! The image quality does look noticeably better. I tested with a 720p host resolution and 1080p client, and details definitely appear sharper. But I do not observe any anti-aliasing being applied. Would it be possible to implement it? Also, I hope AMD opens up FSR 3/4 in the future - it would be great to achieve even better results. Tested on Laptop: ██████████████████ ████████
██████████████████ ████████ ------------------
██████████████████ ████████ OS: Manjaro Linux x86_64
██████████████████ ████████ Host: HP Pavilion Laptop 15-eh1xxx
████████ ████████ Kernel: 6.18.12-1-MANJARO
████████ ████████ ████████ Uptime: 20 hours, 37 mins
████████ ████████ ████████ Packages: 1943 (pacman), 25 (flatpak)
████████ ████████ ████████ Shell: zsh 5.9
████████ ████████ ████████ Resolution: 1920x1080
████████ ████████ ████████ DE: GNOME 49.4
████████ ████████ ████████ WM: Mutter
████████ ████████ ████████ WM Theme: Adwaita
████████ ████████ ████████ Theme: adw-gtk3-dark [GTK2/3]
████████ ████████ ████████ Icons: Papirus-Dark [GTK2/3]
Terminal: gnome-terminal
CPU: AMD Ryzen 5 5500U with Radeon Graphics (12) @ 4.056GHz
GPU: AMD ATI 04:00.0 Lucienne (VEGA 6)
Memory: 7496MiB / 15305MiB
|
|
@nhths , For Linux, that will be the most I can do. As for FSR2 (and beyond), the technology is completely different from FSR1. FSR1 is a spatial upscaler, meaning it works on a per-frame basis, which makes it well suited for video streaming. FSR2 is a temporal upscaler, which is far more complex: it requires per-pixel motion vectors and depth buffer information that are generated during the game's rendering pass on the host. Such data is not transmitted over the stream, and I don't think that will be possible anytime soon, this kind of data is significantly heavier than a compressed video frame. |
linckosz
changed the title
|
While i love the quality of this feature, there is a significant performance loss when using it with my setup. I multibox a game with 3 PCs, two moonlight sessions i alt tab between. |
@YakovAU RTX VSR relies on machine learning with an AI model, so it does consume GPU resources. You can try the following instance. 
|
linckosz
force-pushed
the
master
branch
2 times, most recently
from
4b98aa1 to
c3c2ccf
Compare
This commit introduces video super-resolution and upscaling support across Windows, Linux and macOS platforms. DirectX 12 Upscaling Pipeline - Added a full D3D12 renderer to enable advanced GPU upscaling features. - Works on Windows for AMD, NVIDIA, and Intel GPUs, and on macOS (ARM) using Metal. - Supports SDR, HDR, and YUV 4:4:4 pipelines. - Uses vendor-specific driver upscalers by default (e.g., AMD AMF), with automatic fallback to a shader-based solution (typically FSR1) when unsupported or unreliable on older hardware (e.g., NVIDIA GTX). - Automatic algorithm selection based on vendor, GPU model, and available features. - Optimized for low-end GPUs (such as Intel UHD) by combining GPU upscaling with RCAS shader operations. - Uses FFmpeg DX12 hardware decoding by default; includes fallback to DX11 decoding with DX11–DX12 interop in cases where DX12 decoding causes stutters on some AMD GPUs. Rendering remains fully D3D12. - Added detailed upscaler information to the on-screen statistics overlay. - Add Shader FSR1 for Linux macOS Upscaling - Integrated MetalFX upscaler for macOS platforms. Tested Hardware - NVIDIA RTX 4070 Ti (Windows & Linux) - NVIDIA GTX 1050 (Windows & Linux) - AMD Radeon RX 7600 (Windows) - AMD Ryzen 7 780M iGPU (Windows) - Intel Arc A380 (Windows) - Intel UHD Graphics (16 EU, N95 CPU) (Windows) - Snapdragon 7c (Windows ARM) - Apple M1 Pro (macOS) - Apple M3 (macOS)
17 participants
Video Super Resolution (VSR) for Moonlight
Video Super Resolution (VSR) is to video what DLSS is to 3D rendering.
So why not make Moonlight one of the first game streaming solutions to leverage this technology?
AI upscaling significantly reduces bandwidth usage without compromising video quality!
Tested GPUs:
Context
Video Super Resolution (VSR) is to video what DLSS is to 3D rendering.
So why not make Moonlight one of the first game streaming solutions to integrate this technology?
AI-based upscaling drastically reduces bandwidth usage while maintaining excellent video quality.
NVIDIA, Intel (link in French), and more recently AMD have all started promoting their respective AI-powered video enhancement technologies.
Implementation
Implementing VSR was not straightforward. I had to add a Video Processor component to D3D12VA to offload frame processing from the CPU to the GPU, allowing it to take advantage of additional GPU capabilities.
SettingsView.qml.d3d12va.cppandd3d12va_shaders.cpp.Platform-Specific Details:
✅ NVIDIA: Provides VSR, the best overall rendering.
✅ Intel: Is currently using FSR1 Shader, IntelVPL shall be workable but could not make it work yet.
✅ AMD: Released AMF Video Upscaling, a ML model based on FSR1 delivering impressive results.
Additional Development (iOS)
For those interested in VSR for iOS, I have also developed moonlight-ios-MFX, but this is still a work in progress.
On an iPhone 13 Pro, the upscaler works well but is too power-hungry due to the Metal renderer (not the upscaler itself), making it impractical for now.
It may perform better on newer iPhone models, but I haven't tested them yet.
Another approch can be to use a customized shader made to performed on mobile.
I don't have an Apple TV, but it could potentially work with minor adjustments and additional testing.
I won’t be maintaining the iOS version, so feel free to contribute and improve it!
Results & Comparisons
Resolution Test:
Banding Test: