Click to Photon Latency is Useless (and I made something better)
Ok ok hear me out for a second – the whole “click to photon” input lag measurement isn’t as useful as you might think. For those that don’t know, if you’ve ever seen us reviewers report “input lag” or “input latency” results with one of these, NVIDIA’s LDAT – latency display analysis tool – you’ve seen a click to photon input lag result. Hell even when Linus used my open source response time tool they only reported the click to photon latency. So if everyone is doing it, what’s the problem?
The problem is pretty simple – it isn’t measuring anything useful. It’s measuring how long it takes for a click sent from the tester, through the system, into a game engine, then out the graphics card to the monitor, to register that input and do something with it. That sounds like a good measurement though, right? I mean that’s literally what you do when you are playing your games – and it’s useful to know how long that process is taking. That’s true, but there is more to it than that.
Let’s fire up CSGO with LDAT ready to do 20 shots automatically – I’ll start with how I’d normally test this sort of thing with no bots. I line up against this metal sign then let it run. It’s going to run through the full magazine then once that’s done we can take a look at the results. Without bots it’s a little inconsistent, but I got an average of 23.67ms. Not bad for this setup.
Right, now let me restart the lobby and put some bots back in and change where I do the test. LDAT is ready to go again but I’m going to do it down here with the bots all running nearby. Let’s see what this set looks like. Ah, this is higher. Its a little more consistent but it’s higher for sure. It averages at just shy of 27ms, up from 23.7ms.
But if the only thing that changed was what was happening in game, why does the input lag change? Well that’s the thing, the majority of the time being captured here is in how long your system and the game takes to do whatever animation you are asking of it. I ran some results with Cyberpunk 2077 – on the same system and monitor – and got an average of 44ms. That’s wildly different from CSGO, because Cyberpunk runs at a significantly lower FPS and takes longer to register those inputs.
What’s worse is here I am back in CSGO, but this time I’m using a different system. Same monitor, same no bots, same place on the map, and yet the results I’m getting are completely different. 40.6ms. Practically double. The same setup, just a different system.
When you see a reviewer testing a monitor – or even input devices like mice – with something like LDAT, the measurement they are reporting is specific to the exact system they are using, the game they are using, and even what’s happening in the game during their testing too. It’s not something you can then translate to your experience – or even compare to other reviewers’ results.
The problem with all of these measurements is they look like this. A single time block which starts when LDAT sends the mouse click over USB, and ends when it detects the screen getting brighter from the shot. It’s just one big block you get to call the “click to photon” or “total system” latency. That’s it. But it isn’t one single thing – it’s actually made up of three main blocks. The time it takes the USB controller to poll LDAT – which at 1000 Hz can be up to 1ms, the time it takes for the game engine to receive that input and render a frame with the action in it, then the time it takes your monitor to draw that new frame. The important piece of information is really only that last stage though, or for peripheral reviews the first step. The bit in the middle only serves to complicate and ruin the useful information.
Now it is worth noting that you can still extract useful data from a tool like this – but that comes from direct comparisons with controlled variables. A reviewer that uses the same system, the same settings, the same game, the same situation in game CAN directly compare results between different monitors. Sure, you as a viewer can’t then compare those numbers to another review, but the fact that the test methodology stayed consistent means the results can be compared – as in a monitor that reports 22ms is better than one that reports 35ms.
Still, wouldn’t it be good to have more certainty with those measurements? To be able to get rid of that messy bit in the middle and just know how long the monitor itself is taking? I think so. That’s why I bought one of these – this is a Time Sleuth, an open source input lag tester. This hooks up via HDMI direct to your monitor and displays a flashing pattern on screen and reports how long it takes for the monitor to respond. It’s built using an FPGA – a field programmable gate array – which is actually part of the problem. See this caps at 1080p60 – the FPGA can’t handle anything higher resolution or faster in the refresh rate. That’s a bit of a problem for the many, many 1440p and 4K monitors that are so popular – it’s especially bad for 1440p displays as they can’t just natively scale the 1080p signal up. The monitor has to do some computation to turn the 1080p frame into a 1440p one instead, and that means you don’t get all that accurate results.
Ok, there has to be another way then, right? Sure, Simon from TFT Central uses a CRT display and a high speed timer to compare to the LCD he’s testing. This is pretty accurate, although does require you to have a CRT display on hand, a way to output an analogue signal from your GPU (or a DisplayPort to VGA adapter) and a bit of time and effort to capture and calculate the results.
Fine, what other options do we have? Well, I’ve been working on something for months now trying to solve this very problem. Even better, I’m releasing it for free. It’s part of the Open Source Response Time Tool project, which means anyone with one of these – or the new PRO unit that I’m actually building units for right now to go live with (so make sure you sign up to be notified when they go live on OSRTT.com!) – these will all work with the new mode.
In short, I wrote my own DirectX 11 code to replace the Unreal Engine 4 tool I built. I’m still using UE4 for the response time measurements for now but I’ll be swapping to my DirectX code soon once I confirm the transitions are all accurate and reliable. The main reason for using my own code is that I can handle when the inputs come in and record how long it takes for the frame to be rendered with that input. Coupled with the board’s existing ability to time the USB polling delay, it turns out it’s pretty easy to work out how long the display is actually taking to display a new frame.
I also added a new set of options for the Results View, which can now graph both each of the elements on their own, and as a scatter plot for each on display result to see consistency. The variation comes from the new frame taking an extra refresh rate tick to be drawn – but since the DirectX code isn’t doing anything intensive it’s pretty easy to run at 1000 FPS without issue. I even had this running at 1000 FPS while rendering videos using handbrake, meaning my CPU was pinned at 100% already. Nice and easy.
The update is now live on Github, if you’ve already got it installed the auto-updater will do the work for you. If you don’t have an OSRTT unit, you can pick up one of the last standard model units I have left, or sign up to be notified when these even better pro units are available – which should be very soon!
