Skybox position, scale, and orientation incorrect and variable when using d3d9.shaderModel = 0

[BinqAdams]

Description of the issue

When using d3d9.shaderModel = 0;
The skybox (hemisphere) is rendered too small, and changes position / orientation / scale seemingly randomly as the camera position and orientation changes.
This does not occur in the stock base game's fixed function pipeline running without DXVK.

Replication

Run Painkiller Black Edition with d3d9.shaderModel = 0 set in dxvk.conf.
Load a level which uses a skybox (level 1 for example).
Observe the skybox in the incorrect scale and location.
Move around and change viewing angle to see the skybox occasionally teleport to a different location/orientation/scale

Software information

Game: Painkiller Black Edition
Settings: Doesn't matter, as long as the skybox is not disabled in settings

System information

• GPU: RTX 3080 10GB
• Driver: Latest 566.36
• Wine version: Using windows
• DXVK version: 2.5.3

API Trace

I only seem to be able to run apitrace without dxvk, replacing the dxvk d3d9.dll with the apitrace d3d9.dll.
I tried the method with the command line as such:

• apitrace trace --api d3d9 F:\SteamLibrary\steamapps\common\Painkiller Black Edition\Bin\Painkiller.exe
• apitrace trace --api dxgi F:\SteamLibrary\steamapps\common\Painkiller Black Edition\Bin\Painkiller.exe

However with --api d3d9 I received the message:
"\lib\wrappers\d3d9.dll was never used: application probably does not use this API"
With --api dxgi I received no message and no trace was generated.

Here are two screenshots, the first shows the issue clearly, and the second shows the skybox smaller and off in the distance when altering the viewing angle.

As I was unable to find another wrapper which could force the Pain engine into fixed function mode, I used a patched .dll to force the engine to use the fixed function path without using any wrappers - the issue did not occur and the sky rendered correctly in the "stock" fixed function path without DXVK.

Log

Painkiller_d3d9.log

[K0bin]

I don't know if D3D9 even supported hardware that didn't support programmable shaders at all. It's difficult to find documentation because of how long ago this was. D3D9 wasn't even the first version that supported shaders. That was D3D8.

So I don't think shader model 0 is valid at all. It's definitely something I have never tested. The game does seem to try and create a lot of shaders.

[WinterSnowfall]

I don't know if D3D9 even supported hardware that didn't support programmable shaders at all. It's difficult to find documentation because of how long ago this was. D3D9 wasn't even the first version that supported shaders. That was D3D8.

Rare instances of d3d9 games with exclusive fixed-function fallback paths exist, one of the famous ones being Star Wars: Empire at War, but it is very much a rarity. Most d3d8 games however do indeed have such a path intended for older hardware.

So I don't think shader model 0 is valid at all. It's definitely something I have never tested. The game does seem to try and create a lot of shaders.

It's likely native errors out on shader creation when a game tries to create a shader above what's advertised as part of the supported caps, not sure if d9vk currently does (we do that on d8vk side, for what it's worth).

[WinterSnowfall]

@DimplyPickle Can you please share a screenshot of how the skybox is supposed to look when it is correct on the game's fixed function path (without dxvk)? I can get rid of the problem by rejecting any programmable shaders on creation (which would happen on fixed function paths anyway), but that just doesn't render a skybox at all, it seems.

Edit: Nevermind, I can't seem to see any skybox, not even on latest master with d3d9.shaderModel = 0.

[BinqAdams]

Can you please share a screenshot of how the skybox is supposed to look when it is correct on the game's fixed function path (without dxvk)?

Of course! I turned the FOV up a bit so that you could get a better look.

Edit: Also, I am using the Steam version of Painkiller Black Edition, perhaps that makes a difference, but I'm not sure as I have not tried other versions. The sky looks the same in fixed function mode as setting sky quality to low in "normal" mode.

[WinterSnowfall]

I've looked into it a bit more and it seems like the game wasn't really meant to run properly solely on fixed function, as it needs at least a VS 1.1 to render the skybox properly. If I use d3d9.shaderModel = 1 everything looks fine, although I can see some PS 2.0/3.0 shaders being rejected, as one might expect.

I suspect whatever you are doing to force the game to use a fixed function path outside of dxvk has a more pronounced/correct effect on the game engine, and it may end up behaving differently.

I happen to have a D3D7-capable card I can use in a Pentium 4 system to check what it reports in terms of caps and features, so it's not the end of the road just yet, but I doubt there will be any further major revelations on the matter.

[BinqAdams]

Just for additional information;
Using my "force engine itself into fixed function" hack, and then also using DXVK and setting d3d9.shaderModel = 0, results in the issue occurring again, unlike when just using my hack alone. Not sure if this is useful information, but I figured it can't hurt to add.

[WinterSnowfall]

Just for additional information; Using my "force engine itself into fixed function" hack, and then also using DXVK and setting d3d9.shaderModel = 0, results in the issue occurring again

Interesting, thanks for the info. We might be looking at a FF bug after all then.

[BinqAdams]

Also, the game includes a variety of readable shader and fx files; for example:

shader<nv20> sky copy terraintu2 {
	pass {
		depthwrite false
		depthtest true
		blend		none
		cull		ccw
		lighting	false

		texenv[0] = wrap bilinear_nomips
		texenv[1] = wrap bilinear_nomips
		texenv[2] = clamp bilinear_nomips
		texenv[3] =	clamp bilinear_nomips

		map[0] =	$colormap
		map[1] =	$blendmap
		map[2] = 	$alphamap
		map[3] =	$lightmap

		xform[0] =  $blendxform
		xform[1] =  $alphaxform

		vshader =	tu2_blend def tu2
		fshader	=	layer3_blend
	}
}

shader skytrans copy sky {
    pass {
		blend		translucent
		alphatest   true
		alphafunc   greater
		alpharef    0	
    }
}

shader skyntu {
    pass {
		depthtest	true
		depthwrite	false
		blend		none
		cull		ccw

		texop[0] =	texture 
		texop[1] = 	disable

		texenv[0] = wrap bilinear_nomips

		xform[0] =  $identity

		map[0]    = $colormap
		
		vshader =	NTU
		fshader = 	null
	}
}

shader skyntutrans copy skyntu {
    pass {
		blend		translucent
		alphatest   true
		alphafunc   greater
		alpharef    0	
    }
}

[WinterSnowfall]

@DimplyPickle I think I've gotten to the bottom of this thanks to an ancient GeForce 4 MX 440 card I still have from way back when I was in high school. Late D3D7 cards at least, which I guess were the last stand in terms of D3D9 compatibility, actually advertised support for VS 1.1, while not advertising any PS support.

While I don't have an even older card to double check (such as a GeForce 2 for example), if I replicate the GeForce 4 MX way of handling the matter within dxvk, things do indeed start to look as per your screenshots.

You can try out this build: https://github.com/doitsujin/dxvk/actions/runs/12889958588/artifacts/2462303511 with d3d9.shaderModel = 0, of course.