Four Ways to Detect Vista
Pop quiz. How do you detect Windows version? Simple, call the GetVersionEx API.
And now for a bonus question. How do you detect Windows version without a doubt? Well, that's not so simple. Do you disagree? Read on.
For some time (I think this functionality appeared in Windows XP), you can execute applications under a compatibility layer. Right-click your exe, select Properties and click on the Compatibility tab. Click on the Run this program in compatibility mode for. Now you can select from the list of many operating system versions - from Windows 95 onwards.
When you do this, plenty of things happen. Firstly, Windows starts faking OS version to your program. Secondly, some API functions work more like they did in the selected OS version.
Why would you do this? Maybe the program in question is not well-behaved and needs this kind of help from the OS. Or maybe it just tests for supported OS version too aggressively and you want to circumvent this test.
In all cases, result is the same - what you get from GetVersionEx is not the true answer. So, I have to repeat my bonus question - how could you detect the true OS version?
Four ways to detect OS
I run into that problem few weeks ago. We have an application that is not yet Vista-ready, mostly because of some DirectX incompatibilities. We don't want negative user experience and therefore we don't allow the app to run on Vista at all. Still, some users may be smart enough to enable compatibility mode for this application, which would make OS version test useless (sadly, compatibility mode doesn't help this application to work correctly on Vista).
I found no good solution but luckily my fellow Slovenian Delphi users did. Even more, they found three alternative solutions.
One is to check for system file that was not present in previous OS versions. For example, one can test for presence of %WINDIR%\System32\ndfetw.dll.
One is to check Notepad.exe version. Under Vista, Notepad has version 6.something while it was 5.something in XP.
And the last one (and the best, in my opinion) is to check if specific API is exported. For example, Vista exports GetLocaleInfoEx from kernel32.dll while previous Windowses didn't.
That's it - now you have one way to get simulated OS version (actually, there are two - you can check the HLKM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\CurrentVersion key) and three to get true OS version.
The last approach is so flexible that we (Miha-R from the Delphi-SI forum, who did most of the work, and me) turned it into a function that can detect all 32-bit OS versions simply by checking for exported API functions. After some work, we created a list of OS-version-determining APIs, all exported from kernel32.dll.
Writing the function - DSiGetTrueWindowsVersion - was quite simple; just proceed from newest OS to oldest and check for exported APIs. You can get this function (and many more) by downloading the freeware DSiWin32 unit (another effort of Slovenian Delphi community).
Checking for compatibility layer
It would be also interesting to detect whether the application is running under the compatibility layer. There is a registry key that contains this info - Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers, key name is full path of the executable - but the information itself is not reliable.
First, you have to check both HKLM and HKCU branches, as the compatibility may be set by current user and enforced by the system administrator. Second, on 64-bit Windows, you must check the (true) HKLM64 branch, not the (32-bit compatibility) HKLM branch (see the code below for more detail). And third, all this may be to no avail.
The problem lies in a fact that child processes inherit compatibility layer from parents. If you set compatibility mode for executable A and then run executable B from it, executable B won't have any compatibility data set in the registry but nevertheless it will run under the compatibility layer. Still, information in those keys may be of some use sometime.
Try running it as a normal app and under the compatibility layer.