• Advertisement
Sign in to follow this  

Setting photoshop's alpha channel bit depth

This topic is 3456 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

We're using DDSs at work. The artist I'm working with is reluctant to use DXT1 because when he exports a picture from photoshop, the alpha channel gets squashed to 1 bit making the edges of things really ragged. Is there a way to set the bit depth of the transparency channel so that when he's working in photoshop he'll see exactly the alpha he'll get when he saves it as a DDS?

Share this post


Link to post
Share on other sites
Advertisement
If you don't want things to look jagged you really just need to use DXT5.

Otherwise, he should simply be painting the alpha in white and black. Anything else would just be guesswork. Jack up the contrast then use the pencil tool from then on.

Share this post


Link to post
Share on other sites
We could do DXT5s, but the alpha really has no business being a gradation anyway. The fact that it is probably has more to do with his eraser tool being set to brush instead of pencil. And DXT1s save a little bit more space than the DXT5s (DXT5 is twice as large as the DXT1).

I could try and teach him to only use the pencil eraser tool, etc. but it would be kind of an old-dog-new-trick situation. It would be much easier if there was a way to simply force the alpha channel bit depth to 1 bit. Or even just isolate the alpha channel and force a tolerance on it, or something like that.

Share this post


Link to post
Share on other sites
Quote:
Original post by Numsgil
We could do DXT5s, but the alpha really has no business being a gradation anyway. The fact that it is probably has more to do with his eraser tool being set to brush instead of pencil. And DXT1s save a little bit more space than the DXT5s (DXT5 is twice as large as the DXT1).

I could try and teach him to only use the pencil eraser tool, etc. but it would be kind of an old-dog-new-trick situation. It would be much easier if there was a way to simply force the alpha channel bit depth to 1 bit. Or even just isolate the alpha channel and force a tolerance on it, or something like that.

Have him make the alpha as he pleases, but then when he is done have him crank up the contrast to 100%. This will make all pixels absolute white or black. But again, this will still be guess work as it is hard to tell which pixels will go white and which will go black.
As such, i recommend turning off anti-aliasing for the brushes instead.

Share this post


Link to post
Share on other sites
Is there a way to view the alpha channel like I can view the red, green, and blue channels?

Share this post


Link to post
Share on other sites
Quote:
Original post by Numsgil
Is there a way to view the alpha channel like I can view the red, green, and blue channels?

Yeah... unless you're doing alpha channeling different, the alpha channel should appear right under the rgb channels. Simply hide the rgb channels and select the alpha channel and you should be able to view it.

Share this post


Link to post
Share on other sites
I see RGB, Red, Green, Blue. No alpha. Is there some sort of mode I have to go in to?

Share this post


Link to post
Share on other sites
Quote:
Original post by Numsgil
I see RGB, Red, Green, Blue. No alpha. Is there some sort of mode I have to go in to?

Nope, you should see it, simply create a new channel under that tab and it should be an alpha channel.
How is the artist doing them?

Share this post


Link to post
Share on other sites
Quote:
Original post by Jarrod1937
Quote:
Original post by Numsgil
I see RGB, Red, Green, Blue. No alpha. Is there some sort of mode I have to go in to?

Nope, you should see it, simply create a new channel under that tab and it should be an alpha channel.


Ah, yes. But while it's called an alpha channel, it's not really a channel to show transparency, is it? Near as I can tell it's just an extra data channel?

Quote:

How is the artist doing them?


I'm not sure, actually. He tends to get impatient when he tries explaining this stuff to me. We really have a whole different vocabulary for things.

Share this post


Link to post
Share on other sites
Quote:
Original post by Numsgil
Quote:
Original post by Jarrod1937
Quote:
Original post by Numsgil
I see RGB, Red, Green, Blue. No alpha. Is there some sort of mode I have to go in to?

Nope, you should see it, simply create a new channel under that tab and it should be an alpha channel.


Ah, yes. But while it's called an alpha channel, it's not really a channel to show transparency, is it? Near as I can tell it's just an extra data channel?

Quote:

How is the artist doing them?


I'm not sure, actually. He tends to get impatient when he tries explaining this stuff to me. We really have a whole different vocabulary for things.

Yeah, the alpha channel is simply another extra 8 bit grayscale data channel. It can hold grayscale maps for a variety of things, including shaders.
However, it is most used for alpha testing (1 bit transparency) and alpha blending (8 bit transparency). Checkout the "Bit depth" section of my guide for more information.

Share this post


Link to post
Share on other sites
When I say "alpha channel", I mean transparency specifically. As in the "a" component of the RGBA 32 bit DDS format. Which is what we're using right now. DXT1 uses a single bit for that transparency component, and since our textures are supposed to be cutouts anyway, it seems like a good format.

But when photoshop tries to save that 8 bit transparency channel in to a 1 bit channel, it makes some suboptimal decisions that result in jagged edges. So my ultimate goal here is to force photoshop to use 1 bit for its transparency so we can examine and correct any jagged edges before exporting.

Share this post


Link to post
Share on other sites
Quote:
Original post by Numsgil
When I say "alpha channel", I mean transparency specifically. As in the "a" component of the RGBA 32 bit DDS format. Which is what we're using right now. DXT1 uses a single bit for that transparency component, and since our textures are supposed to be cutouts anyway, it seems like a good format.

But when photoshop tries to save that 8 bit transparency channel in to a 1 bit channel, it makes some suboptimal decisions that result in jagged edges. So my ultimate goal here is to force photoshop to use 1 bit for its transparency so we can examine and correct any jagged edges before exporting.

I'm not sure if you're following me.
If i understand your problem correctly, refer to the link i posted above on how to disable the brush anti-aliasing. This will prevent any type of "in-between" pixels and allow you to paint a 100% white or black pixel range, aka, 1 bit color range. Which should work for the 1 bit alpha testing (transparency).

Share this post


Link to post
Share on other sites
Yes, I'm aware of that trick, but it doesn't help with existing images. What I want to be able to do is take an existing psd which has various levels of transparency, and "flatten" that transparency to 1 bit.

Basically I'm trying to get photoshop to be as close to WYSIWYG for DTX1s as I can.

Share this post


Link to post
Share on other sites
Quote:
Original post by Numsgil
Yes, I'm aware of that trick, but it doesn't help with existing images. What I want to be able to do is take an existing psd which has various levels of transparency, and "flatten" that transparency to 1 bit.

Basically I'm trying to get photoshop to be as close to WYSIWYG for DTX1s as I can.

Then just go to the alpha channel and crank up the contrast to 100%, as stated above. That will force all pixels to be either 100% blank or white, aka, 1 bit color range.

Share this post


Link to post
Share on other sites
Yes, that is the part I do not know how to do... I can create a new channel, and it will name itself alpha1, but it will just be an arbitrary data channel. It won't actually be the transparency channel. Here is the view I have when I open one of the psds and go to the channels tab (click for better res versions:)

Free Image Hosting at www.ImageShack.us

Then, if I create a new channel, it calls it "Alpha 1", but as you can see it does not actually store transparency information. It's just treated as an extra data channel:

Free Image Hosting at www.ImageShack.us

So forgive me if I'm just being exceptionally obtuse (I'm a programmer by trade, and we seem to have different vocabulary from artists), but is there a way to get the channel tab to display the transparency information as a separate channel?

Share this post


Link to post
Share on other sites
Oh i see. Your artist is utilizing photoshops built in "transparency", where you erase part of the image to reveal the checkered pattern underneath.
While this may be recognized by .dds, it generally is not the correct way to produce transparency in games.
After a quick google i found a tutorial that explains alpha channel transparency quite well:
Hair alpha Tutorial

You can use basic/simple transparency, like that found in .gif and what you're doing. But you generally want to use the alpha channel for finer control, and more advanced 8 bit transparency.
The alpha channel generally provides an 8 bit grayscale map to define what pixels will be transparent. Black will be transparent while white will be visible.
I would recommend using 1 bit alpha, as the fill rate penalty is less, and only use 8 bit for things thta may require it like hair (if your engine supports 8 bit alpha).

Share this post


Link to post
Share on other sites
Heh, okay, I think we're on the same page now.

Now, I created a new alpha channel, calling it "alpha", and copied it from the red channel. Then I went in and contrasted it so that it was black and white only. I can shift+click on the alpha channel and the rgb channel and have them overlay at the same time, so I can see where the alpha channel is in relation to the underlying rgb channels. And I can even double click on my alpha channel and toggle between masked/selected.

How do I now export that alpha channel and eventually get it as the transparency in a DDS? We use the NVidia Photoshop DDS plugin to export textures, but it refuses to even try when there's an alpha channel (Error: too many channels to export (5)). Saving it as PNG just causes the extra alpha channel to get stripped out. If I could even export just that channel as it's own texture, I could probably rig up an automated merge step that combined an rgb texture and an alpha texture into a single dds.

Share this post


Link to post
Share on other sites
Try flattening the image first.

Also, if you want to copy photoshops built-in transparency data from a regular layer into an actual alpha channel, you can ctrl+click on the layer to use, and then click the 'Save selection as channel' button (circle in a rectangle) on the channels toolbar.

Share this post


Link to post
Share on other sites
Quote:
Original post by Hodgman
Try flattening the image first.


Flatten lets me save it as DDS, but it seems the actual alpha channel is just ignored.

Quote:

Also, if you want to copy photoshops built-in transparency data from a regular layer into an actual alpha channel, you can ctrl+click on the layer to use, and then click the 'Save selection as channel' button (circle in a rectangle) on the channels toolbar.


As near as I can tell, this does not actually copy the built in transparency. It instead selects everything that satisfies some magic-wand type tolerance and copies that? Close enough for most cases, but not quite the same thing.

Share this post


Link to post
Share on other sites
Quote:
Original post by Numsgil
As near as I can tell, this does not actually copy the built in transparency. It instead selects everything that satisfies some magic-wand type tolerance and copies that? Close enough for most cases, but not quite the same thing.
No it does. The selection outline may not look quite correct, but it really does respect any semi-transparent pixels - they will be written into the alpha channel as the correct value of grey.

We've been using this technique at work to get images from non-technical artists into production games for years ;)

If you want to copy multiple layers to an alpha channel, then ctrl+shift+click on each of them in turn, then press the 'Save selection as channel' button.

[EDIT]
Just to clarify, in your Channels toolbar, you've only got Red, Green, Blue and Alpha?

Share this post


Link to post
Share on other sites
Quote:
Original post by Hodgman
Just to clarify, in your Channels toolbar, you've only got Red, Green, Blue and Alpha?


Yes. The rgb channels were there originally, and I created the alpha by doing "new channel".

Share this post


Link to post
Share on other sites
I'm still having problems. How do I actually save out my alpha channel to a file format I can use?

Share this post


Link to post
Share on other sites
Quote:
Original post by Numsgil
I'm still having problems. How do I actually save out my alpha channel to a file format I can use?

Well, the only requirement is that the format supports 32 bit images, like .tga.

Share this post


Link to post
Share on other sites
Okay... tell me where in my process I'm making the mistake:

I have my image, with its four channels. RGB, and a new one I created called alpha. Alpha contains the transparency information I want to use, different from what's encoded in the RGB channels. I save as... and change the type to PNG. It then asks me for interlaced/interleaved (can't remember exactly) and I just hit OK. I then reload the image in photoshop to check and see if it saved off my extra alpha channel, or used it instead of the built in transparency, etc.

It has not, the channel was apparently just stripped of that extra data channel. The transparency is the transparency that was encoded in the rgb channels, not my modified version.

Share this post


Link to post
Share on other sites
Quote:
Original post by Numsgil
Okay... tell me where in my process I'm making the mistake:

I have my image, with its four channels. RGB, and a new one I created called alpha. Alpha contains the transparency information I want to use, different from what's encoded in the RGB channels. I save as... and change the type to PNG. It then asks me for interlaced/interleaved (can't remember exactly) and I just hit OK. I then reload the image in photoshop to check and see if it saved off my extra alpha channel, or used it instead of the built in transparency, etc.

It has not, the channel was apparently just stripped of that extra data channel. The transparency is the transparency that was encoded in the rgb channels, not my modified version.

I don't believe Photoshop's native .png support actually supports 32 bit images. Png does however support basic web graphic transparency like .gif, which really is not accurate enough for games.
If you're not locked into that format try .tga, which is my favorite to use. You need to research and make sure your format supports 32 bits for alpha transparency, and to make sure the format just doesn't use 24 bits with standard web graphic transparency.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement