vertex shader with HLSL - no colour SOLVED

Started by
6 comments, last by mattm 19 years, 1 month ago
Hi, First thanks for everyones help in the past, but i am once again stuck...I have searched for this but cannot find an answer. Basically, i have my terrain drawn and want to use a vertex shader to change its color. But, i cannot get the vertex shader to output anything but grey, no matter what i try. My terrain is the standard vertex/index buffer with no materials, textures vertex colors etc. I am using a FVF for this example of the following format
Vertex::FVF = D3DFVF_XYZ|D3DFVF_NORMAL;

struct Vertex
{
	Vertex(){};
	Vertex(float x, float y, float z,float nx, float ny, float nz);
	float	mx;
	float	my;
	float	mz;
	float	mnx;
	float	mny;
	float	mnz;;
	static const DWORD FVF;
};

 

My shader is as follows (i have used a number of different values for color (including the "blue" vector defined, but no luck). I am definately using the shader!
//terrain shader
//only maps to clip space for now


//matrix to hold the required transform

matrix ViewProjMatrix;
matrix ViewMatrix;


//some vectors to hold our values

vector blue = {0.0f,0.0f,1.0f,1.0f};

//input and output structures

struct VS_INPUT
{
	vector position : POSITION;
	vector normal : NORMAL;
};

struct VS_OUTPUT
{
	vector position : POSITION;
 	vector diffuse : COLOR;
};

//Main

VS_OUTPUT Main(VS_INPUT input)
{
	//zero output
	VS_OUTPUT output = (VS_OUTPUT) 0;
	
	//transform position to clipspace
	//input.position.y =0;
	output.position = mul(input.position, ViewProjMatrix);

	output.diffuse = 0xffff00;
	return output;
} 

I have not set any settings using the SetRenderState method as i don't beleive these apply when using vertex shaders (if i'm wrong please say!) Any help would be greatly appreciated! Thanks, Matt [Edited by - mattm on March 3, 2005 5:14:07 PM]
Advertisement
your output diffuse should be a float4 with each float representing a a color channel (RGBA). No idea what it does when you output a single float value. The pixel shader forces you to this output and if you wrote your own pixel shader you would have to translate there. Assuming that you are using Fixed Function for the pixel pipeline you should probably output like this from your vertex shader.
SetTextureStageState(0, D3DTSS_COLOROP, D3DTOP_SELECTARG1);
SetTextureStageState(0, D3DTSS_COLORARG1, D3DTA_DIFFUSE);
Even with the floats it is still black or grey.Could it be one of my D3DPRESENT_PARAMETERS ; would any of them produce this? The 2 commands above also have no effect.

Thanks for you help,

Matt
Ok. I've still had no luck with this. I would really appreciate it if someone could calrify some points for me please:

1) Do i need to assign a material or texture or vertex color to my object prior to putting it through the vertex shader or should the color in the vertex shader be the color which is applied to my mesh?

2) Are there any obvious settings thay may prevent my mesh having any color. For instance, even when using the vertex shader (and fixed function pixel pipeline) do i still need a light (i don't beleive so).

3) Do i need a diffuse setting in my FVF for me to apply a color

4) Do i need to send any render states for me to see my mesh in color?

Thanks,

Matt
Wierd. The following works

//terrain shader//only maps to clip space for now//matrix to hold the required transformmatrix ViewProjMatrix;matrix ViewMatrix;//some vectors to hold our values//input and output structuresstruct VS_INPUT{	float3 position : POSITION;	float3 normal : NORMAL;	float4 color: COLOR;};struct VS_OUTPUT{	float4 position : POSITION; 	float4 diffuse : COLOR;};//MainVS_OUTPUT Main(VS_INPUT input){	//zero output	VS_OUTPUT output = (VS_OUTPUT) 0;		//transform position to clipspace	//input.position.y =0;	output.diffuse = float4 (0.0f, 0.0f, 1.0f, 1.0f);	output.position = mul(float4(input.position,1), ViewProjMatrix);	return output;} 


so it appears it did't like by definition of blue. No idea but thanks to turnpast for putting me on the right track.

EDIT: Found out why - i hadn't set TransformConstantTable->SetDefaults(..); so i guess the variables had not bee intitialised and therefore were neve been set. oops, but hopefully someone else will learn from my mistake!

Matt
I have never had to do a call SetDefaults for my effect files fill out the constant values (but if it works...). I also have never used the 'vector' type in my HLSL code, but the docs sugges that when you use it you have to specify its type:

vector <float, 4> bigEnoughForAColorVector = {0,0,1,1};

looks like you have switched everything over to float3 and float4 now anyway.

Glad things are working for you now.

PS.
If you want to see something interesting run 'fxc.exe /Cc /Tfx_2_0 youreffect.fx' on your effect file. I use it regularly myself to help debug my effects.
I changed the blue from a vector to floats and it still didn't work before setting the SetDefaults. It may be that this is a coincidence; i will have to take it out now it is working to see :)

I followed up your original advice and saw that only floats are guranteed to be supported (or other types may be converted into them) so i changed them all.

Thanks,

Matt

This topic is closed to new replies.

Advertisement