Sign in to follow this  
MajinCry

C# - Detect If Pre-Existing .DDS Contains Alpha Channel?

Recommended Posts

For the past wee while, I've been scouring the net to find a way to tell if a .dds file contains an alpha channel. And I've come up completely dry.

 

I've been working on a tool that sorts through all of the .dds file in a given directory, including it's sub-directories, and resizing all .dds images of a specified size to another. Pretty much just takes text from the gui, reads a couple bytes from the header of the .dds image, and then chucks it all to texconv.exe.

 

Thing is, I need to detect whether or not a .dds image uses transparency. If it does, include the -pmalpha argument when launching texconv.exe

 

How would I go about doing this? I've not much experience with C# (this program is my first foray into the language), and my sum total of programming experience is this and some Pascal for Tes5Edit, so I'll probably need to be explained to as if I'm 5 years old or somethin'.

 

I'm thinking that maybe SharpDX has some use for this? The documentation isn't, err, beginner friendly to say the least, sadly.

 

 

Anywho, if it helps at all, 'eres a couple links:

 

Current release of the program: http://www.nexusmods.com/fallout4/mods/17624/?

Current source code: https://github.com/MajinCry/Fallout-4-Texture-Resizer

Share this post


Link to post
Share on other sites
You could write a pretty simple check by reading the DDS file's headers:

https://msdn.microsoft.com/en-us/library/bb943991.aspx

Note: These documents outline the structures in C style, but you can easily convert that to C#.

DWORD = uint
WORD = ushort
etc.

It looks like the information you want can be found in the DDS_HEADER -> DDS_PIXELFORMAT -> DDPF_ALPHAPIXELS bit in the flags field.

Personally I would just read the file using a BinaryReader and seek to the correct offset in the file that corresponds to that field. Just count the # of DWORDs you need to skip in order to get to the dwFlags field, multiply that by 4, say filestream.Offset = whatever_that_is, then dwFlags = binaryReader.ReadInt32() and then bool containsAlpha = (dwFlags & 1) == 1; Edited by Nypyren

Share this post


Link to post
Share on other sites

The header contains alpha data? I swear I tried comparing the headers between various formats (with and without alpha), and couldn't find any consistency between them. Finding the x & y resolution, as well as the mipmap levels, was easy on the other hand.

 

Me findings can be found here: http://pastebin.com/isBKwaas

 

I think the format is stored in 4 bytes, starting @ offset +54, but that's about it. Comparing the >100 different .dds formats, with the various alpha combinations...Err, hmm.

 

 

The microsoft documentation seems to be really awkward. It was rather good for the simpler functions, but it seems to rather terse about this directx stuff.

DWORD               dwMagic;
DDS_HEADER          header;
DDS_HEADER_DXT10    header10;

dwMagic starts at the beginning of the file, and looks like this in a hex viewer: 44 44 53 20

 

Maybe header starts @ offset +04?

 

And header10 is probably the four bytes @ offset +54.

 

No clue where to find the PIXELFORMAT bytes, though. https://msdn.microsoft.com/en-us/library/bb943984.aspx

Share this post


Link to post
Share on other sites
The way the C++ struct layouts are defined here, each field comes in the order listed, and nested structs exist at that position in the file as well (they aren't pointers to arbitrary locations).

You are correct that DWORDs are 4 bytes (or, 32-bits since 1 byte = 8 bits). The corresponding type in C# to a DWORD is called a uint, or sometimes seen as UInt32. For example the BinaryReader function to read a uint from a file is called ReadUInt32.

so here's how you would find the offset you want:
 
DWORD               dwMagic; = offset 0
DDS_HEADER          header; = offset 0 + sizeof(DWORD) = offset 4
{
  DWORD           dwSize; = offset 4
  DWORD           dwFlags; = offset 8
  DWORD           dwHeight; = offset 12
  DWORD           dwWidth; = offset 16
  DWORD           dwPitchOrLinearSize; = offset 20
  DWORD           dwDepth; = offset 24
  DWORD           dwMipMapCount; = offset 28
  DWORD           dwReserved1[11]; == offset 32, notice this is an array of 11 DWORDs.
  DDS_PIXELFORMAT ddspf; = offset 32 + sizeof(DWORD) * 11 = offset 32 + 44 = offset 76
  {
    DWORD dwSize; = offset 76
    DWORD dwFlags; = offset 80   <============================== This is what you're interested in
    DWORD dwFourCC;
    DWORD dwRGBBitCount;
    DWORD dwRBitMask;
    DWORD dwGBitMask;
    DWORD dwBBitMask;
    DWORD dwABitMask;
  }
  DWORD           dwCaps;
  DWORD           dwCaps2;
  DWORD           dwCaps3;
  DWORD           dwCaps4;
  DWORD           dwReserved2;  
}
dwFlags is a bitfield (i.e. it is a number composed of several other numbers ORed together). the value you care about is:
 
DDPF_ALPHAPIXELS       Texture contains alpha data; dwRGBAlphaBitMask contains valid data.     Value = 0x1 
What this means is, if the texture contains alpha data, the dwFlags should have bits corresponding to the value 0x1 set.

To check whether that value is set, you say:
 
bool isValueSetInBitfield = (bitfield & mask) == value;
where 'bitfield' is 'dwFlags' and both 'mask' and 'value' are 0x1, since it's a one-bit field.


If you discover that you need to also examine the DXT10 header, figuring out which offset it's at works the same; it just comes immediately after the first header. Edited by Nypyren

Share this post


Link to post
Share on other sites
In your existing code, the function GetDDSDimensions already looks like it's using this technique to get the width and height (the offsets are written in hexadecimal in your code, and my previous post is using decimal instead). You could easily add a read of the dwFlags field and check for alpha in there as well, if you want. Edited by Nypyren

Share this post


Link to post
Share on other sites

Sorry for the late reply, been trying to wrap my head around this, trying to get the concept down. Honestly, it's making my brain hurt.

 

Been converting the code you posted to a C# equivalent, and here's what I've got so far:

        public bool IsAlphaPresent(string strDDSFileName)

        {

            using (FileStream fsSourceDDS = new FileStream(strDDSFileName, FileMode.Open, FileAccess.Read))

            using (BinaryReader binaryreaderDDS = new BinaryReader(fsSourceDDS))

            {

                fsSourceDDS.Seek(0x04, SeekOrigin.Begin);

                uint uintSizeOfDDS = binaryreaderDDS.ReadUInt32();

                fsSourceDDS.Seek(0x08, SeekOrigin.Begin);

                uint uintDDSFlags = binaryreaderDDS.ReadUInt32();



                fsSourceDDS.Seek(0x0c, SeekOrigin.Begin);

                ushort ushortDDSHeight = binaryreaderDDS.ReadUInt16();

                fsSourceDDS.Seek(0x10, SeekOrigin.Begin);

                ushort ushortDDSWidth = binaryreaderDDS.ReadUInt16();



                fsSourceDDS.Seek(0x14, SeekOrigin.Begin);

                uint uintDDSPitchOrLinearSize = binaryreaderDDS.ReadUInt32();

                fsSourceDDS.Seek(0x18, SeekOrigin.Begin);

                uint uintDDSDepth = binaryreaderDDS.ReadUInt32();



                fsSourceDDS.Seek(0x1c, SeekOrigin.Begin);

                byte byteNumberOfMipmaps = binaryreaderDDS.ReadByte();

                fsSourceDDS.Seek(0x20, SeekOrigin.Begin);

                byte[] bytearrayDDSReserved = binaryreaderDDS.ReadBytes(44); // Collection of 11 dwords





            }

        }

I've been using hexadecimal values because I've a teensy bit of experience with CheatEngine, in that I've hooked a couple instructions and documented the npc data structure for Dark Souls. Feels more sane to me to use hex, especially since I'm having to reference the header in the .dds file via a hex editor.

 

Is the dwFlags always at a specific offset (i.e, always at the same offset in every BCn format .dds texture), or does it change depending on the format?

Share this post


Link to post
Share on other sites

Is the dwFlags always at a specific offset (i.e, always at the same offset in every BCn format .dds texture), or does it change depending on the format?


As far as I can tell from MSDN, yes, it's always at a specific offset.

Also, you don't need to seek and read all of the intermediate fields if you don't care about them. you can seek directly to the dwFlags field, skipping everything inbetween.
 
fsSourceDDS.Position = 0x50;  // 0x50 == 80; using .Position does the same thing as Seek does.  I pretty much never use Seek anymore.
uint dwFlags = binaryreaderDDS.ReadUint32();
bool containsAlpha = (dwFlags & 1) == 1;
FileStream.Position: https://msdn.microsoft.com/en-us/library/system.io.filestream.position(v=vs.110).aspx Edited by Nypyren

Share this post


Link to post
Share on other sites

Sweet, I'll give it a shot.

 

What I've done, is added the following code:

 public bool IsAlphaPresent(string strDDSFileName)
        {
            using (FileStream fsSourceDDS = new FileStream(strDDSFileName, FileMode.Open, FileAccess.Read))
            using (BinaryReader binaryreaderDDS = new BinaryReader(fsSourceDDS))
            {
                fsSourceDDS.Position = 0x50;
                uint uintDWFlags = binaryreaderDDS.ReadUInt32();
                bool boolIsValueSetInBitfield = (uintDWFlags & 1) == 1;

                if (boolIsValueSetInBitfield)
                {
                    return true;
                }
                else
                {
                    return false;
                }

            }
        }

At the beginning of RunTexConv():

string strAlphaArgument;
            if (IsAlphaPresent(strFinalDDS))
            {
                strAlphaArgument = "-pmalpha ";
                MessageBox.Show(strFinalDDS + " Has an alpha channel!");
            }
            else
            {
                strAlphaArgument = "";
            }

If everything goes as planned, it should just throw up a message for the .dds files that have an alpha channel. Time to see if it works.

 

 

Edit: The code above didn't work, had to do a wee bit of editing to a couple functions, to include the source .dds filepath (due to the destination .dds not existing yet). Alas, no alpha was found in any of the files, despite there being an alpha channel in several of them.

 

Made a new branch with the edited code: https://github.com/MajinCry/Fallout-4-Texture-Resizer/tree/Test---Check-For-Alpha

 

Couple of .dds images I'm using to test: http://www66.zippyshare.com/v/cEJnMsOj/file.html

Edited by MajinCry

Share this post


Link to post
Share on other sites

I think testing DDPF_ALPHAPIXELS is only going to work reliably for uncompressed textures. If it's say a DXT5 texture then it could leave that flag clear and still work correctly. It would also be possible to generate a file where that flag is set, but the alpha channel is actually set to opaque for every pixel.

 

If you want to be certain, the easiest option is probably to convert to an uncompressed format, and read back the alpha channel. Of course for DXT5 you can be fairly certain there is an alpha channel - for opaque textures DXT1 is equivalent quality, and half the file size as it doesn't store full alpha data. However, a DXT1 image may or may not use the single bit alpha channel feature.

 

Of course most of the time game toolchains work the other way round - they start with uncompressed RGBA images and convert to .DDS to reduce the memory cost. When you're doing that you can make sure your converter sets a flag somewhere.

 

Also note that DXT is lossy compression. You will get better quality results if your tools work with uncompressed (or losslessly compressed) textures, and convert to .DDS as the final step.

Edited by Adam_42

Share this post


Link to post
Share on other sites

I think testing DDPF_ALPHAPIXELS is only going to work reliably for uncompressed textures. If it's say a DXT5 texture then it could leave that flag clear and still work correctly. It would also be possible to generate a file where that flag is set, but the alpha channel is actually set to opaque for every pixel.

 

If you want to be certain, the easiest option is probably to convert to an uncompressed format, and read back the alpha channel. Of course for DXT5 you can be fairly certain there is an alpha channel - for opaque textures DXT1 is equivalent quality, and half the file size as it doesn't store full alpha data. However, a DXT1 image may or may not use the single bit alpha channel feature.

 

Of course most of the time game toolchains work the other way round - they start with uncompressed RGBA images and convert to .DDS to reduce the memory cost. When you're doing that you can make sure your converter sets a flag somewhere.

 

Also note that DXT is lossy compression. You will get better quality results if your tools work with uncompressed (or losslessly compressed) textures, and convert to .DDS as the final step.

 

Y'know, I think you might be on to something. I checked a couple of the transparent textures, and they had DXT5 in the header. Checked a few that weren't transparent, and they had DXT1. Hmm.

 

If the image has an alpha channel but doesn't use it for anything, that's of no consequence; I don't plan on "optimizing" textures, as that would be more work and an absolute pain seeing how I can't even check if an alpha is present or not, for now at least. Would be good to have, but I'm aiming to get the base functionality down first.

 

Aye, it'd be great if Bethesda supplied the source textures (and the PSDs while they're at it), but, alas, they've only given us peasants the final .dds textures. 'Tis all I've got to work with, and it's also what the modders supply everyone with. Not ideal for this project, but, well, there's no alternative.

 

 

I wonder if SharpDX would be any use. The documentation is pretty curt, and there aren't any tutorials on how to use it for something like this (identifying pre-existing .dds image properties), though. Are there any users kicking around here who are fairly pro with that? I could try firin' them a wee message.

Share this post


Link to post
Share on other sites

New finding, new post.

 

After doing some reading, I believe it's possible to call a function from a C++ .dll file, from within C#, without rewriting the C++ code.

 

I compiled the [url=https://github.com/Microsoft/DirectXTex/blob/master/DDSTextureLoader/DDSTextureLoader.cpp]DDSTextureLoader.cpp[/url] file to a .dll, and it compiled fine. I think this may be a foxhole worth gettin' stuck in, as DDSTextureLoader has a function that checks the alpha channel of a .dds texture.

 

Thing is, err, I've no clue how to proceed with this. This is what I've gathered so far:

using System.Runtime.InteropServices;

namespace WindowsFormsApplication1
{
    public partial class formProgramWindow : Form
    {

        [DllImport("DDSTextureLoader.dll")]
        public static extern void LoadTextureDataFromFile(string dllDDSFilename);

That "LoadTextureDataFromFile(string dllDDSFilename);" line should call the following function from the .dll:

 

    HRESULT LoadTextureDataFromFile(
        _In_z_ const wchar_t* fileName,
        std::unique_ptr<uint8_t[]>& ddsData,
        DDS_HEADER** header,
        uint8_t** bitData,
        size_t* bitSize)

 

Now, I think that I only need to provide it a string, as there is only one variable (a collection of chars? called fileName) with the _In_ prefix, which I assume is the argument that the function takes, with the other variables being what the function punts out. Hovering my mouse over "HRESULT" causes Visual Studio to say "Typedef long HRESULT". Maybe this causes the function to return a large integer, with subsections that we can reference.

 

There is also the following function in the .dll:

DDS_ALPHA_MODE GetAlphaMode( _In_ const DDS_HEADER* header )

with DDS_ALPHA_MODE being "Typedef enum DirectX::DDS_ALPHA_MODE"

 

 

 

What I think I need to do (no idea how), is do something like:

intDDSTextureData = LoadTextureDataFromFile(string dllDDSFilename);
GetAlphaMode( _In_ const intDDSTextureData.DDS_Header)

But, err, GetAlphaMode returns an enum as well, whatever that is. Really, at this point, I'm just regurgitating what my mouse tells me when I hover over things.

 

 

Yup, no clue what I'm doing. Send help.

Edited by MajinCry

Share this post


Link to post
Share on other sites
Sorry my initial suggestions didn't help earlier on. I can provide more assistance with what you're trying now.

The signature for LoadTextureDataFromFile looks like it will store the results in the four other arguments. In order to call that using DllImport from C#, your C# DllImport line of code would need to have a compatible function signature that .Net can determine is related to the actual function in C++. However, DllImport only works with C functions as far as I know. Besides, that is one *hell* of a complicated function signature to try to get DllImport to accept; I'm not sure what to do about the std::unique_ptr from C#'s point of view even if they did add C++ support to DllImport.

If it were up to me, at this point I would build a "C++/CLI" DLL. C++/CLI is a way of writing code in C++ which can link to normal C and C++ libraries, which you can then call directly from other .Net languages like C# without using the DllImport approach. You would still need to write code that bridges between "unmanaged" code (code that calls the DirectX functions) and "managed" code ('ref classes' which can be passed directly to C#), but at least in C++/CLI you have access to both "C++/CLI" and "plain old C++" simultaneously.

It's potentially simple, but if you're uncomfortable using C++ directly it probably won't be very fun. Then again, you already have a C++ DLL building already, so it's not much of a leap to make a C++/CLI DLL. Another plus is that you probably won't need to write very MUCH C++ code yourself so I wager you could get it working.

In Visual Studio 2015, the project type you want is: Visual C++ -> CLR -> Class Library. The "CLR" part is akin to the "CLI" in C++/CLI.

After you make the C++/CLI DLL, you add it as a reference in the C# project just like you would with other .Net DLLs. Edited by Nypyren

Share this post


Link to post
Share on other sites
The GetAlphaMode function signature implies that it is figuring things out using the DDS_HEADER alone. If we can figure out what it's looking at, we can go back to your original approach of reading the DDS_HEADER directly from the file in pure C#. Perhaps it's looking at several other fields, not just the dwFlags field. Edited by Nypyren

Share this post


Link to post
Share on other sites

The main roadblock is that I've no experience 'n' knowledge with C++ and C#; this project is something I started with literally 0 knowledge of C#, after all. I'm a wee bit more competent with Pascal, but even then, I'm not exactly a pro with that either. Doesn't help that documentation is rather lacking, as well.

 

 

Here's the code for the GetAlphaMode function:

 

    DDS_ALPHA_MODE GetAlphaMode( _In_ const DDS_HEADER* header )
    {
        if ( header->ddspf.flags & DDS_FOURCC )
        {
            if ( MAKEFOURCC( 'D', 'X', '1', '0' ) == header->ddspf.fourCC )
            {
                auto d3d10ext = reinterpret_cast<const DDS_HEADER_DXT10*>( (const char*)header + sizeof(DDS_HEADER) );
                auto mode = static_cast<DDS_ALPHA_MODE>( d3d10ext->miscFlags2 & DDS_MISC_FLAGS2_ALPHA_MODE_MASK );
                switch( mode )
                {
                case DDS_ALPHA_MODE_STRAIGHT:
                case DDS_ALPHA_MODE_PREMULTIPLIED:
                case DDS_ALPHA_MODE_OPAQUE:
                case DDS_ALPHA_MODE_CUSTOM:
                    return mode;
                }
            }
            else if ( ( MAKEFOURCC( 'D', 'X', 'T', '2' ) == header->ddspf.fourCC )
                      || ( MAKEFOURCC( 'D', 'X', 'T', '4' ) == header->ddspf.fourCC ) )
            {
                return DDS_ALPHA_MODE_PREMULTIPLIED;
            }
        }

        return DDS_ALPHA_MODE_UNKNOWN;
    }

 

To break it down, it looks ddspf.flags and the DDS string @ offset 0x54. If it's DX10, it looks in misFlags2 for the DDS_MISC_FLAGS2_ALPHA_MODE_MASK flag. Somehow, that returns four different integers. STRAIGHT returns 1, PREMULTIPLIED returns 2, OPAQUE returns 3, CUSTOM returns 4 and UNKNOWN returns 0.

 

If the texture has DXT2 or DXT4, it just decides that it always has alpha channel data. Bloody thing doesn't seem as robust as I thought. Ergh.

 

Taking a look at a couple variables with "alpha" in their name, I've got to say that C++ makes absolutely no sense with how it handles hexadecimal.

 

#define DDS_ALPHA       0x00000002  // DDPF_ALPHA
 
...
 
enum DDS_MISC_FLAGS2
{
    DDS_MISC_FLAGS2_ALPHA_MODE_MASK = 0x7L,
};

 

I assume that the defined DDS_ALPHA variable is just a dword with a value of 2. Since .DDS files are LittleEndian, it would look like "00 00 00 02" in the hex viewer...Which isn't present in any .dds files with an alpha channel. At least, not with Fallout 4's DXT5 textures.

 

That could mean that the alpha mode mask is at the file offset 0x7L, but that's not even valid hex.

 

 

Urgh. Brain hurt. I also tried looking at the Intel Texture Works source code to figure out how the Photoshop .dds plugin detects if an image has an alpha channel, but it's not any better. It uses a variable called "planes" to check if there is an alpha channel or not. Don't ask me what that means, I've nae idea either.

Share this post


Link to post
Share on other sites
I'm not familiar with the internal workings of DXT5, but I found this table when I searched:
https://en.wikipedia.org/wiki/S3_Texture_Compression#S3TC_Format_Comparison

It seems like DXT5 is always capable of alpha, but doesn't use *premultiplied* alpha. The -pmalpha argument to texconv means "convert to premultiplied alpha". DXT1, 2, and 4 use premultiplied alpha according to that wikipedia page. But just because a format is capable of alpha doesn't mean it USES alpha. Every pixel might be fully opaque. If you need to actually check to see if alpha is actually used or not, you'll probably have to actually load the entire file and check every pixel somehow.


I'm curious: What happens in your tool if you pass -pmalpha for a texture that doesn't have alpha? Does Fallout 4 fail to load the texture? Does it look strange in game? Does the tool report an error? Edited by Nypyren

Share this post


Link to post
Share on other sites

S3TC is the compression used for Direct3D 9 games and older. Far as I know, most Direct3D 10 & 11 games moved to the new BC format (Direct3D 11 added BC6 and BC7 compression). https://msdn.microsoft.com/en-us/library/windows/desktop/bb694531(v=vs.85).aspx

 

At least, BCn compression is what Fallout 4 uses in it's textures. Whilst some BCn files have DXTn in their header, they don't use the older S3TC formats.

 

I'll go give that a try now. As for what happens when you run a texture with alpha, through texconv without passing an alpha argument, it fills the transparent parts of the image with neon yellow.

 

[spoiler]

tGnc66N.png

[/spoiler]

 

The landscape textures have an alpha channel in the original .dds files. However, running them through texconv.exe corrupts the alpha channel, which I believe is due to texconv.exe not having an alpha argument passed.

 

 

Edit: Just tested. I took the textures for the machete weapon, and chucked them into texconv.exe with the -pmalpha flag. Spawned a copy of the weapon, and nothing seems funky.

 

What I think happens, is that texconv.exe just generates an empty alpha channel (all black) when it's passed a texture to calculate the alpha for, when there aren't any transparent pixels in the first place.

Edited by MajinCry

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this