I managed to calculate slip angle based on angle of wheelhubvel and wheeldir. thing is, the car jiggles a bit even when it should stand still. but when moving it's quite fine. how do I solve the stability problem[/quote]
You don't compute the slip angle using the wheel hub velocity. The slip angle is found using the orientation of the wheel, and the direction of travel. These are all normalised direction vectors.

The jiggling problem is due to the integrator alternating the force vector direction when the body starts to come to rest. You will see also this when you start implementing the longitudinal traction force calculations. Naturally this will also affect the vector used to indicate direction of travel for the lateral stuff. To minimise this oscillation problem, I simply I apply manual damping on the system when the velocity drops below a threshold. You can do this via a rolling resistance function, or something equivalent. It's a hack but it works well.

Umm I'll try that, ugh hacks.

heheh seems the lateral force is working. I need to get my hands on the longitudinal version now. It behaves so good, I even compared it to the marco monster cardemo. I somehow wonder how his car can 'power slide', it even slides too much, you have any idea?[/quote]
Marco's implementation is a just a rough approximation of the Pacejka. His version is basically a two step function, where it has a linear adhesion region (the ramp in the plot), then the plot is capped to a maximum. That region is what causes the car to slide.

You mean I should reduce the lateral force and favor the longitudinal? [/quote]
You raised a good question. To clarify things, the 'damping' aspect is applied on the longitudinal component of your system. However, this has an indirect implication for the lateral calculations. Ultimately you will compute the net force vector acting on the car body, which has contributions both from the long and lateral calculations. From this you also compute the velocity vector (in the integrator), and the direction of travel. This direction vector is eventually fed back into your slip angle calculations. Basically if you stabilise your longitudinal calculations, the lateral system should also stabilise. Does this make sense?

Does it mean the sum of long and lat force should lie within the circle? [/quote]
To further clarify the oscillation problem has nothing to do with the traction circle. It's a numerical instability problem caused by your integrator. Do not confuse this with 'car stability', that is, keeping the vehicle under control while driving.

The traction circle simply refers to the car's maximum traction in any direction of travel. If you think about it, when you contribute the lateral and longitudinal traction forces on the body, the combination can be considered as a traction circle, or ellipse. The perimeter of the circle represents the peak traction of your system in any direction along 2D plane of the car body. If you look at one of Brian Beckman's articles, he illustrates this concept well with this image. The graph shows the lateral and the longitudinal Pacejka formulas plotted together. The base of the graph signifies the lateral and longitudinal (i.e. 2D) direction of the car body. Inside that hole is the 'adhesion region' of the tyres, rim of that plot is the maximum traction circle, and slope outside of that is the region where the car loses traction control.