I'd like to have the possibility to 3D-paint an object texture using the mouse as a brush.
I've tried ray-casting mouse click, getting the triangles inside the brush area, calculate barycentric coordinates for each point intersection, set the colors on the uv corresponding values, saving and reloading the texture. Too slow.
I'd like to have a glsl solution for this. I've google around but couldn't find much.
If someone would be so kind to develop a small application that paints a sphere, or a cube, using a glsl approach I'd like to pay for it (I use Paypal).
Looking forward to any possible help, thanks.
1
3D painting
Started by Alessandro, Jun 23 2012 07:22 AM
22 replies to this topic
Sponsor:
#5 Members - Reputation: 284
Posted 23 June 2012 - 02:40 PM
Hello Adam, I have an application that load an .obj file, and I'd like to be able to paint the model texture directly using the mouse.
The painting to be done is really basic, I just need to color pixels so that I can use that information to mask parts of the model later for further operations.
Does this explain a little better?
The painting to be done is really basic, I just need to color pixels so that I can use that information to mask parts of the model later for further operations.
Does this explain a little better?
Edited by Alessandro, 23 June 2012 - 02:57 PM.
#7 Crossbones+ - Reputation: 1523
Posted 23 June 2012 - 06:12 PM
hmm, this is a rather complex problem, for convex objects, this is a bit simpler, but for complex objects, it gets a bit more tedious, and their are most likly many approachs, but i can only think of one way that i'd do it:
create a large texture to be the "brushed" texture, each polygon/triangle would get a small unique patch of this texture(hence why it'd be very large), i'd then fire a ray at the polygon, and figure out where the brush is intersecting which faces, after that, i'd figure out all the faces that the brush would affect, and from their, figure out the pixels to write to in the client side version of the texture.
using glSubImage2D i'd update this texture once every tenth of a second, or once every quater of a second(whichever one doesn't create too much of a noticable lag), by spacing out the updates, this should allow the program to not slow down very much, and by evaluating where in the texture was last updated, i can minimize the amount being uploaded.
in the shader, i'd simple pass x texture coordinates where x is the number of textures the model already has overlayed+1 for the brush texture, finally i'd add whatever the final color is, with the brush's texture, and bam, you have brushed object.
creating the texture can be done in multiple ways, if you want a 1:1 per texel/polygon, then the texture size would have to be equal to the sum of the area of all triangles, and you can increase/decrease precision by the ratio.
anyway, that's my approach, unfortuantly i'm working on other things, so i'm not going to write up an sample, but i feel you are probably well-able.
and again, if speed of constantly submitting a new texture is your concern, simply decrease the rate of uploads into a per second, instead of per frame.
create a large texture to be the "brushed" texture, each polygon/triangle would get a small unique patch of this texture(hence why it'd be very large), i'd then fire a ray at the polygon, and figure out where the brush is intersecting which faces, after that, i'd figure out all the faces that the brush would affect, and from their, figure out the pixels to write to in the client side version of the texture.
using glSubImage2D i'd update this texture once every tenth of a second, or once every quater of a second(whichever one doesn't create too much of a noticable lag), by spacing out the updates, this should allow the program to not slow down very much, and by evaluating where in the texture was last updated, i can minimize the amount being uploaded.
in the shader, i'd simple pass x texture coordinates where x is the number of textures the model already has overlayed+1 for the brush texture, finally i'd add whatever the final color is, with the brush's texture, and bam, you have brushed object.
creating the texture can be done in multiple ways, if you want a 1:1 per texel/polygon, then the texture size would have to be equal to the sum of the area of all triangles, and you can increase/decrease precision by the ratio.
anyway, that's my approach, unfortuantly i'm working on other things, so i'm not going to write up an sample, but i feel you are probably well-able.
and again, if speed of constantly submitting a new texture is your concern, simply decrease the rate of uploads into a per second, instead of per frame.
Edited by slicer4ever, 23 June 2012 - 06:14 PM.
Check out https://www.facebook.com/LiquidGames for some great games made by me on the Playstation Mobile market.
#8 Members - Reputation: 21
Posted 24 June 2012 - 08:01 PM
i can post a code here but it will be native opengl and glu its simple that you find a point in 3d where your mouse is and then draw a blended or not quad on it
it is REALLY SIMPLE
it is REALLY SIMPLE
Master & Mentor
https://sites.google.com/site/imoomohomozumo/
https://sites.google.com/site/imoomohomozumo/
#9 Members - Reputation: 284
Posted 25 June 2012 - 01:39 AM
i can post a code here but it will be native opengl and glu its simple that you find a point in 3d where your mouse is and then draw a blended or not quad on it
it is REALLY SIMPLE
Hi, thanks but this wouldn't help. I need to draw pixels, not vertices.
#10 Members - Reputation: 220
Posted 25 June 2012 - 04:26 AM
I've done this before. First you need to auto unwrap the mesh, to generate proper UVs. If you are using DirectX it has some utility functions for it, otherwise you have do it yourself. After having the mesh unwrapped, you just need to do a ray test when the user 'paints' the mesh, to get the UV, then update the texture.
#11 Members - Reputation: 21
Posted 25 June 2012 - 06:57 AM
but you save output image as single image, there is nothing special to obtain a pixel coordinate in image from any perspective and do whatever you like
Edited by ___, 25 June 2012 - 06:59 AM.
Master & Mentor
https://sites.google.com/site/imoomohomozumo/
https://sites.google.com/site/imoomohomozumo/
#12 Members - Reputation: 551
Posted 25 June 2012 - 11:16 AM
lol, he wants to paint an object. You are suggested something like a decal system. How does a single quad wrap around a sphere? That is not painting and it is a broken decal system as well.
I still don't get why you can't use a normal artist pipeline. 3D artists do this all the time and then its all good to get loaded in at startup. If you really need it to be inside of your project then it is a bit more hard to explain. A basic overview that should work:
UV unwrap the object (you have to do this).
In game, you need run all your vertices through the modelviewprojection to get the vertices final 2D Screen pos.
Use those 2D texture coords with your "paint brush" image, and you have to draw those on top of the original texture coords.
That's as in depth as I can go, I haven't seen many articles about it but you should decide why you need this. There are plenty of other 3D paint tools. If the user/gamer is not going to be painting objects in the simulation/game, then use an existing tool.
I still don't get why you can't use a normal artist pipeline. 3D artists do this all the time and then its all good to get loaded in at startup. If you really need it to be inside of your project then it is a bit more hard to explain. A basic overview that should work:
UV unwrap the object (you have to do this).
In game, you need run all your vertices through the modelviewprojection to get the vertices final 2D Screen pos.
Use those 2D texture coords with your "paint brush" image, and you have to draw those on top of the original texture coords.
That's as in depth as I can go, I haven't seen many articles about it but you should decide why you need this. There are plenty of other 3D paint tools. If the user/gamer is not going to be painting objects in the simulation/game, then use an existing tool.
#14 Members - Reputation: 284
Posted 07 July 2012 - 10:13 AM
Well, I'm still at trouble with this feature I'd like to add to my application.
I have the unwrapped mesh and I know how to get reference to it, so that if I click a point on a model in the 3D viewport, I can get the corresponding point in UV space.
So in practice I am able to draw a pixel directly on the uv texture map (and apply it on the model so that it shows the painting).
However, I don't understand how I could actually do the same for all the pixels inside the cursor brush area (the attached image should explain better than my words).
How do I calculate the UV points for all the pixels inside the green cursor brush area? Using glUnproject would be and option but it would be slow as hell. Any idea?
I have the unwrapped mesh and I know how to get reference to it, so that if I click a point on a model in the 3D viewport, I can get the corresponding point in UV space.
So in practice I am able to draw a pixel directly on the uv texture map (and apply it on the model so that it shows the painting).
However, I don't understand how I could actually do the same for all the pixels inside the cursor brush area (the attached image should explain better than my words).
How do I calculate the UV points for all the pixels inside the green cursor brush area? Using glUnproject would be and option but it would be slow as hell. Any idea?
#17 Members - Reputation: 551
Posted 07 July 2012 - 03:34 PM
It would be much easier with shaders yes.
The idea would be this:
1Render the brush to say a stencil buffer.
2While drawing the current model, if a pixel on the current triangle passes the "inside brush/stencil", then it knows it needs to be written back to the UV textures
3For given pixel, you already know the 2D UV coordinate in the image you are about to paint. Paint it based on the brush applied.
Given that you don't have shaders working then I dont know. As always my suggestion is: Open up maya/blender/zbrush. Paint, export, import into your game.
The idea would be this:
1Render the brush to say a stencil buffer.
2While drawing the current model, if a pixel on the current triangle passes the "inside brush/stencil", then it knows it needs to be written back to the UV textures
3For given pixel, you already know the 2D UV coordinate in the image you are about to paint. Paint it based on the brush applied.
Given that you don't have shaders working then I dont know. As always my suggestion is: Open up maya/blender/zbrush. Paint, export, import into your game.
#18 Members - Reputation: 551
Posted 07 July 2012 - 03:41 PM
I came up with a genius, non-shader way:
To start, you need a texture that holds pixel coordinates. What this means, pixel 0,0 is not red or green or blue, but it is 0,0,0 (its position is 0,0. If your texture is say 512x512. the pixel in the middle (256,256) holds again not a color but 256,256,0. Ok in gl everything is scaled 0 to 1. So divide by 512, and the middle pixel = .5,.5,0.
With me yet?
How to use it:
When drawing your model to the screen, you will write to the framebuffer, the model with that texture applied. What this means is every pixel you see on the screen currently, you have their positions in the UV map. You can then call glReadBuffer, read the pixels back and if then apply that brush by getting its window position on the current image. And applying that over your array of pixels.
Will this be slow? No. You only need to readback the current framebuffer if the camera has moved. Well sending the new texture every frame might be slow if it is big, but for 512x512 should be no problem.
Get it?
If it wasn't obvious, you would then just clear the screen and draw the model with the painted texture applied so that the user never sees the other stuff you painted.
To start, you need a texture that holds pixel coordinates. What this means, pixel 0,0 is not red or green or blue, but it is 0,0,0 (its position is 0,0. If your texture is say 512x512. the pixel in the middle (256,256) holds again not a color but 256,256,0. Ok in gl everything is scaled 0 to 1. So divide by 512, and the middle pixel = .5,.5,0.
With me yet?
How to use it:
When drawing your model to the screen, you will write to the framebuffer, the model with that texture applied. What this means is every pixel you see on the screen currently, you have their positions in the UV map. You can then call glReadBuffer, read the pixels back and if then apply that brush by getting its window position on the current image. And applying that over your array of pixels.
Will this be slow? No. You only need to readback the current framebuffer if the camera has moved. Well sending the new texture every frame might be slow if it is big, but for 512x512 should be no problem.
Get it?
If it wasn't obvious, you would then just clear the screen and draw the model with the painted texture applied so that the user never sees the other stuff you painted.
Edited by dpadam450, 07 July 2012 - 03:44 PM.
