I noticed that on the ASSIMP forums there were many requests for an animation loader– even more specially, one that pre calculated the animation transforms. There are huge benefits to pre calculating anything, especially animation data like this. I had a test mesh with 56 bones (yeah, its alot i know, but the models animations are great) and a single texture for the entire model. So the whole model took a single draw call.
All the numbers below were taken from using NVIDIA PerfHUD, which is a great profiling tool!
To Traverse the model and recalculate the transforms each frame took about .083 ms CPU time (this is includes the total model setup time as well), GPU draw time was .083ms. You see the problem?
Afterwards, my CPU time went to .013ms, and of course my gpu time was the same. This might not seem like big savings, but if you have 20-30 animated characters on the screen the total cpu time is 25* .083 = 2.075 ms, and 25* .013 = .325ms. As you can see, this is a huge savings if I plan on having multiple animations running in the scene.
So, to use it, you must be using the ASSIMP library of course! The code also has a save to disks and load from disk that works as well. So, if you want to save the animation, use
void Save(std::ofstream& file);
void Load(std::ifstream& file);
The functions expect the file stream to be opened somewhere else in the program, then passed a reference to the stream. In this way, animations can be saved where ever you like within a custom model file, or to a separate file.
Before any Animations can be played,
void Init(const aiScene* pScene);
has to be called, which will extract the skeleton and all animations from the pScene object.
After that, to get the matrix transforms, simply call
std::vector& GetTransforms(float dt)
Now, this function will wrap your dt, so dont worry about going out of bounds. If the time is 5656565, it will wrap it correctly. So just keep a running total of the dt and pass it to the function. The return will be the array with the transforms, which you then pass to the shader.
Note: I did not comment the code completely, and there is a small reminent of my dual quaternion implementation that will work if you uncomment it and use it. You would need to check out the NVIDIA SDK10, Quaternion Skinning example for the shader code and their implementation. It is pretty straight forward. All you would need to do is change the shader. Also, I use my mat4 class, but it is complely compatible with the D3DXMATRIX class, so if you replace mat4 with D3DXMATRIX and their equivalent calls, there will be no problems. One last thing, this builds transforms for a LEFT HANDED coordinate system. ASSIMP’s matrix4x4 is a right handed matrix, which is why I do a couple of transposes in the code when converting to my mat4.
The code is free for use for anyone for any purpose.
Hi! I would like to ask, how should be quats returned from std::vector& GetTransforms applied on mesh. For example, GetTransforms[0] should be applied on which part of mesh? GetTransforms[1] on which and so on … Thanks in advance for response (btw, your work is great, it’s pitty i didn’t find this earlier
Hi Juti. Each index in the returned std::vector corresponds to the bone index in the model. So, the vector should be bound to an array in your vertex shader, then you use the bone index of your model to index straight into the shader array that you just set.
The function GetTransforms() should be called each frame and the array should be rebound to the shader.
Does that answer your question?
The bone indices are the same as what you will get when you extract them from the ASSIMP loader.
thanks for fast reply,
I would like to ask one more question, if you don’t mind.
Assimp’s every bone has array of “aiVertexWeight* mWeights”, and every single aiVertexWeight has “float mWeight”, which can be different from bone to bone and differnt from vertex to vertex affected by bone(if i understand correctly, for 1. vertex affected by bone it can be 0,5f and for 2. vertex affected by the same bone it can be 0,8f), and i have to do a vertexPos * weight * TransformationMatrix;
So, if i want to do a GPU skinning, i have to send to vertex shader: Vertex, float mWeight and then i have to update for every bone a constant buffer with TransformationMatrix. Well, my problem is, updating constant buffer per bone isn’t such a deal, but sending float with every vertex, it would be slow, wouldn’t it?… how did you solve that problem? (I’m assuming that you don’t have a constant buffer (or global float in shader) that you are updating per vertex basis.)
If this is maybe too much offtopic or I failed to explain it properly, then I’m sorry, but thanks for response anyway!
Juti,
Each Vertex should have with it the bone index and weight. For example, normally in a model you have Position, Texcoords, Normal, and Tangent. In an animated model you extend the vertex format to add BoneIndices, and BoneWeights — each having 4 components i.e. xyzw.
So, when extracting your information from ASSIMP, you should pack all the data for each vertex in a struct that looks like this:
struct Vertex {
float4 Pos;
float2 TexCoords;
float3 Normal;
float3 Tangent;
float4 BoneIndices;
float4 BoneWeights;
};
These are all static data. So, you bind your vertex buffer like normal, and in the vertex shader would do something like this
struct PSPTNIn {
float4 position : SV_Position;
float2 texcoord : TexCoord0;
float3 normal : Normal;
};
cbuffer animationvars {
float4x4 g_matrices[60]; // the bone transforms
}
PSPTNIn AnimatedVS(float3 vPos : POSITION, float2 vTexCoord0 : TEXCOORD0, float3 vNormal : NORMAL, float3 vTangent : TANGENT, float4 vBones : BONE, float4 vWeights : BLENDWEIGHT){
PSPTNIn output;
float3 vNormalWorldSpace;
matrix finalMatrix;
finalMatrix = vWeights.x * g_matrices[vBones.x];
finalMatrix += vWeights.y * g_matrices[vBones.y];
finalMatrix += vWeights.z * g_matrices[vBones.z];
finalMatrix += vWeights.w * g_matrices[vBones.w];
float4 vAnimatedPos = mul(float4(vPos.xyz,1),finalMatrix);
float4 vAnimatedNormal = mul(float4(vNormal.xyz,0),finalMatrix);
// Transform the position from object space to homogeneous projection space
output.position = mul(mul(vAnimatedPos,ObjectToWorld), CameraViewProjection);
// Transform the normal from object space to world space
output.normal = mul(vAnimatedNormal.xyz, (float3x3)InvTransposeObjectToWorld); // normal to world space
output.texcoord.xy = vTexCoord0 ;
return output;
}
I figured it out
(bone weights are limited by assimp, so vertex can be affected by 4 bones)
Feel free to remove the previous comment
This is akward … Sorry for double post,
I just didn’t refresh a page before i posted a comment, sorry!
Thanks for the code and your animation class! You are saving me a lot of time
I wish you best for your mmo, it’s looking very promising, keep it up!
Ask any questions you want, I will do my best to answer them.
Good Luck
Hi, again
I have been stuck on a problem since friday, I want to ask you if you do everything the same way as I do (loading to constant buffer, init of animClass and so on)
This is how it looks (don’t mind those white triangles behind, it’s just tech demo): http://dl.dropbox.com/u/22928727/SomethingIsWrong.jpg
You said that “Also, I use my mat4 class, but it is complely compatible with the D3DXMATRIX class, so if you replace mat4 with D3DXMATRIX and their equivalent calls, there will be no problems. “, it is not completly true, your mat4 class has a .Transpose() function which If i understand correctly switches right-handed system to left-handed. So i had to wrote my own function .(D3DXMATRIX SwapRightToLeftMatrix(D3DXMATRIX& in) in pastebin)
I took shader from Skinning10 directx example, and I verified that it works correctly by setting for one bone/more bones/all bones a transformation, and it happend properly (test function)
Please, if you have time, look at void loadAnimation() and void getAndUpdateBoneMatrices(ID3D11DeviceContext* pImmediateContext), and tell me if you are doing everything like me.
pastebin: http://pastebin.com/ESs1BEMe
118. line, and 131. line should be not there
Thank you very much in advance
I’m really hopeless, I spend whole weekend over this and I can’t get it work.
The right hand to left hand function is not correct. The directx function call for transpose is
D3DXMATRIX* D3DXMatrixTranspose(
__inout D3DXMATRIX *pOut,
__in const D3DXMATRIX *pM
);
Try using that and see what happens.
You sir, you are genius.
http://www.youtube.com/watch?v=wess7VHgi54
You can see that som vertices are still doing what they want, but it seems that it’s caused by my internal problem with loadModel() function.
Once again, thank you very much!
Hahah, thanks Junti. I was very tired when I responded last night to your post. I wanted to tell you why the function was not working, before I was unable too, but it didnt happen that way.
So, a matrix transpose is like this (I am leaving out the diagonal because they dont matter, i.e. 11 =11, 22=22)
std::swap( _12, _21);
std::swap( _13, _31);
std::swap( _14, _41);
std::swap( _23, _32);
std::swap( _24, _42);
std::swap( _34, _43);
If you notice in your swap function you also did this
out._12 = in._13;
out._13 = in._12;
you mean to swap each value, but I think this was an accident. Everyone makes mistakes–especially me!
I just like to say thank you very much for this. I was stuck on getting animations to work with assimp but this project has helped so much and now have my dx11 animations up and running!
has anyone gotten dual quaternions to work, I’m a little stumped on what i have to change in this project to actuate it. I’ll be fine on the shader side I think, at least from looking at the nvidia example, but really need some insight on the dx side. thanks if anyone could share some example code or advice.
anyway thanks again!
David,
I haven’t used the dual quaternion code in a while so I cant say exactly what is needed to get it to work. But any code changes should be made in the GetTransforms function — I know that much. The rest of the code doesnt need to be changed.
Thank you Scott, I’ll start there and see what i can come up with.
I’ve gotten pretty close just a few small issues to figure out.
here is where I’m at if anyone is interested that comes along here.
http://devmaster.net/forums/topic/16358-dual-quaternion-skinning/
I know your probably busy but i just had one more question, but i understand if you can’t answer for any reason.
I was wondering how or what transform to use to get the bones location in 3d space. so i could for instance attach a mesh to the bone location for say weapons or attaching ragdoll rigid bodies.
thanks if any light can be shed on this, it’s just the last thing i need to find out for my animation framework.
David,
If you want to place a weapon in the hand of an model here ss how you do it.
When you run an animation, you call GetTransform(float dt) each frame and pass that to the shaders. You need to find out which bone equal which index in the array returned in the function.
In the class SceneAnimator, there is a protected
std::vector Bones
That vector holds all the bones of the model, with the bone name, (cBone::Name). So, if the modeler did his or her job right, the bones should be named properly: hand, forearm, bicep. So, you can create a function to tell you which Index the bone you want is and use that to get the transform to it.
So, if you want to get the transform form just the hand.
Call GetTransform(float dt)
Pass that to your shader like normal.
Create a function like this in SceneAnimator
unsigned int GetBoneIndex(const string& bonename);
the function should return the index of the bone with that name. Now, you can add code to the Init functions so that an std::map is created to make the search faster, otherwise, you will have to run over the entire std::vector Bones to find the matching bone.
When you find the matching bone, return the index.
Use that index to map directly to the transforms returned from GetTransform( float dt) There is a 1 to 1 relationship
Use that matrix to correctly place the object in the hand.
Here is some pseudo code
In the drawfunction of the model. Assuming EACH model has a Scaling, Rotation, and Translation matrix, and using a left handed system.
void Draw(){& temp = Sceneanim.GetTransforms(dt);
std::vector
//Pass temp to shader here
unsigned int handindex = Sceneanim.GetBoneIndex(“Hand”);
mat4& handmat = temp[handindex];// this is where the hand is relative to the model
//To get a weapon or other objects correct transform to the hand of the model it would be something like this. I am appending a model. onto the characters transform
mat4 weaponinhand = handmat * model.scaling * model.Roation * model.Translation ;
weaponinhand can now be used to move the weapon into place in the shader. I hope that makes sense, if not, please respond and I will do my best to explain.
Thank you very much for the reply and information Scott, that will work great for attaching meshes as i hadn’t thought to do that in the shader, very cool. I was hoping to be able to get a vec3 location though so i could attach other objects to the bones that aren’t rendered but i might be able to extract them another way just won’t be as streamline as your class. thanks very much again you saved me heaps of time and frustration in getting animations up and running!
You can get the exact position by taking the
weaponinhand from the above post and get the translation part of it, but that would only give you the position of the hand. You would need to correctly orient the object in the hand. For example, a running animation, the hand of a person will move around, so the position will not cut it.
Unless you were using something that was symmetrical, like a sphere, where orientation doesn’t matter.
Hi Scott, thanks for the update the new functions work great. But i think the problem is the assimp matrices, as far as animation goes everything is great, but when i decompose the matrix to rip out the location it seems to be well out of place. i get the same results if i do as you suggested by positioning in the shader as in your first set of instructions on this. really not sure why this happens. will have a better look tommorow to make sure it’s not something silly i’ve done.
Hi Scott, I wish to use your code to get my animation up and running but I am using opengl vs directx and i was wondering other then having to omit the transposes necessary for the left handed coordinate system, what else might i have to change in your code to get it to work with a right handed coordinate system. also i am curious, in one of your responses to juti you lay out what looks to be a vertex shader for DirectX am i right in that assumption. it seems fairly straight forward. any ways any words on the matter would help greatly. thanks
I also wanted to ask if the “MyMath.h” is something that was included with the files you have here?
It has been a while since I looked over the code.
In the cAnimationController.cpp file
look for any matrix multiplication and reverse the order. Then any transposes should be removed.
The mymath files are in the repository on google.com. Goto nolimitsdesigns.com and check out my latest post. All my code is available now on a google repository. My math is in the Utilities_Lib and AnimationController is in Graphics_Lib
Holy moly you reply very quick thanks. also yes i forgot that with directx the matrix multiply is swapped thanks. and i will check out that repository. thanks again
No problem. . . any issues that you have, just make a post. I will respond quickly if I am able.
Ok with the CalcBoneMatrices() function would i reverse the globalTransform with Offset when doing this multiply?
such that
before:
void SceneAnimator::CalcBoneMatrices(){
for( size_t a = 0; a Offset * Bones[a]->GlobalTransform;
}
}
after:
void SceneAnimator::CalcBoneMatrices(){
for( size_t a = 0; a GlobalTransform * Bones[a]->Offset;
}
}
I thought that I answered this questions, but I can see that I didnt.
The answer to your question is, yes.
Hello Scott! Thanks for sharing this, it saved me a lot of time. I got one problem however.
When I fill out the SceneAnimator object with the SceneAnimator ::Init(aiScene* scene) function everything works like it should and I get the correct animation. But when I use the SceneAnimator::Load(ifstream& file) function something weird happens. Only half of the model gets animated (the legs) and the rest is static. I have checked that my bone indices and weights are correct and when I compare the result from the SceneAnimator::GetTransforms(float dt) between the usage of Init(…) and Load(…) they are different at the same time stamp.
Maybe you know what’s going on. If you need some code let me know.
// Axel
Axel,
I just got back from a vacation so I couldn’t respond until now.
Just to make sure I am on the same page as you. The Load(stream) function takes a file that has already been opened and loads the animation data from it, instead of through the SceneAnimator::Init(aiScene* scene) function.
So the first time you load an animation you would call SceneAnimator::Init(aiScene* scene), then Save(stream) sometime after it has been loaded. The next time you load the object, you can use the Load(stream) instead of the SceneAnimator::Init(aiScene* scene). This is used because I save all my models to my own format so I save the model vertices, then save the animation data to the same file. But, since the Save and Load take an already opened stream, it could be the same file, or a different one.
So you did all of that and the animation is not playing correctly? The animation does play correctly if you use the Init function to load the animations though?
Thanks for everyone’s posts. This code has been a huge help. I’m running into problems on 2 fronts:
I can’t figure out why the model is not rendering correctly. I have triple checked that all data is flowing into the shader properly. One clue which I found interesting was:
I tried setting all the bone matrices passed in to the shader to the identity matrix. This way I could see if at least the basic loading works properly. The interesting thing there is that some portions of the model do not render. It made me realize that the sum of bone weights for a given vertex don’t have to equal 1, did anyone else notice that?
If I not set it to identity, and just use the actual bone matrices, the model seems to get all screwed up. Does anyone have the AI model loading settings they used?
Hi cris,
I just double checked my model rendering example in the code repository and it works correctly, so you can look at that. An example is in the Model_Loading_Example project.
Most of the good stuff is inside a couple of files though. For the animated model loading code, you can check out
https://github.com/smasherprog/Destination_Toolkit/tree/master/Graphics_Lib
The Animated_mesh files contain my code for loading a model from assimp.
But, I would suggest making sure that the model loads and draws correctly as a static mesh. Once you get the model to load and draw properly as a non-animated mesh, create a new class for your animate mesh types.
My hlsl shader code is in the https://github.com/smasherprog/Destination_Toolkit/tree/master/Assets/Shaders folder under animated_mesh.fx
between those, you should be able to get everything working.
If you are still having problems, post some pictures, and link your code.
Thanks for posting that code. I looked at your shader and loader logic – and while I did it a little differently, it looks very similar. I can’t post a picture right now, but basically the effect is the model (a wild boar) turns into a jumble of planes stretching in random directions basically out to infinity. I can try to post my code somewhere official, but the snippets I compared with yours are below, and I’ll post a link to a full set of code soon. Thanks for your help.
////////////////////////////////////////////////////////////////////////////////
// Filename: light.vs
////////////////////////////////////////////////////////////////////////////////
/////////////
// GLOBALS //
/////////////
cbuffer MatrixBuffer
{
matrix worldMatrix;
matrix viewMatrix;
matrix projectionMatrix;
matrix g_matrices[60];
};
//////////////
// TYPEDEFS //
//////////////
struct VertexInputType
{
float4 position : POSITION;
float2 tex : TEXCOORD0;
float3 normal : NORMAL;
float4 boneindices : BONEINDICES;
float4 boneweights : BONEWEIGHTS;
};
struct PixelInputType
{
float4 position : SV_POSITION;
float2 tex : TEXCOORD0;
float3 normal : NORMAL;
};
////////////////////////////////////////////////////////////////////////////////
// Vertex Shader
////////////////////////////////////////////////////////////////////////////////
PixelInputType LightVertexShader(VertexInputType input)
{
PixelInputType output;
// Change the position vector to be 4 units for proper matrix calculations.
input.position.w = 1.0f;
// Calculate animation if present
matrix animation = worldMatrix;
if(input.boneindices.x != -1)
animation = input.boneweights.x * g_matrices[input.boneindices.x];
if(input.boneindices.y != -1)
animation += input.boneweights.y * g_matrices[input.boneindices.y];
if(input.boneindices.z != -1)
animation += input.boneweights.z * g_matrices[input.boneindices.z];
if(input.boneindices.w != -1)
animation += input.boneweights.w * g_matrices[input.boneindices.w];
// Calculate the position of the vertex against the world, view, and projection matrices.
if(input.boneindices.x != -1)
output.position = mul(input.position, animation);
else
output.position = input.position;
output.position = mul(output.position, worldMatrix);
output.position = mul(output.position, viewMatrix);
output.position = mul(output.position, projectionMatrix);
// Store the texture coordinates for the pixel shader.
output.tex = input.tex;
// Calculate the normal vector against the world matrix only.
output.normal = mul(input.normal, (float3x3)worldMatrix);
// Normalize the normal vector.
output.normal = normalize(output.normal);
return output;
}
Snippet from model loading
m_vertexCount = scene->mMeshes[0]->mNumVertices;
m_indexCount = scene->mMeshes[0]->mNumFaces * 3;
// Create the model using the vertex count that was read in.
m_model = new ModelType[m_vertexCount];
m_indices = new int[m_indexCount];
if(!m_model)
{
return false;
}
for(int i=0; imMeshes[0]->mVertices[i].x;
m_model[i].y = scene->mMeshes[0]->mVertices[i].y;
m_model[i].z = scene->mMeshes[0]->mVertices[i].z;
m_model[i].tu = scene->mMeshes[0]->mTextureCoords[0][i].x;
m_model[i].tv = scene->mMeshes[0]->mTextureCoords[0][i].y;
m_model[i].nx = scene->mMeshes[0]->mNormals[i].x;
m_model[i].ny = scene->mMeshes[0]->mNormals[i].y;
m_model[i].nz = scene->mMeshes[0]->mNormals[i].z;
// initialize bones to -1
m_model[i].boneIndices.x = -1;
m_model[i].boneIndices.y = -1;
m_model[i].boneIndices.z = -1;
m_model[i].boneIndices.w = -1;
// bones weights don't matter
}
for(int j = 0; j mMeshes[0]->mNumBones; j++) {
for(int weights = 0; weights mMeshes[0]->mBones[j]->mNumWeights; weights++) {
int vertexId = scene->mMeshes[0]->mBones[j]->mWeights[weights].mVertexId;
float weight = scene->mMeshes[0]->mBones[j]->mWeights[weights].mWeight;
int boneId = j;
if(m_model[vertexId].boneIndices.x == -1) {
m_model[vertexId].boneIndices.x = boneId; m_model[vertexId].boneWeights.x = weight;
} else if(m_model[vertexId].boneIndices.y == -1) {
m_model[vertexId].boneIndices.y = boneId; m_model[vertexId].boneWeights.y = weight;
} else if(m_model[vertexId].boneIndices.z == -1) {
m_model[vertexId].boneIndices.z = boneId; m_model[vertexId].boneWeights.z = weight;
} else if(m_model[vertexId].boneIndices.w == -1) {
m_model[vertexId].boneIndices.w = boneId; m_model[vertexId].boneWeights.w = weight;
} else {
// error
}
}
}
for(int i=0; i mMeshes[0]->mFaces[i].mIndices[0];
m_indices[i * 3 + 1] = scene->mMeshes[0]->mFaces[i].mIndices[1];
m_indices[i * 3 + 2] = scene->mMeshes[0]->mFaces[i].mIndices[2];
}
aiReleaseImport( scene);
Sorry, it looks like the html formatting on my code above got screwed up and the text color is particularly bad.
I am off to work, Ill look at it when i get back
Hey Scott, thanks. You can delete from last comments with that code – I posted my project on github. The files that I compared against your shader and loader logic are in:
https://github.com/stauffec/survival/blob/master/Survival/light.vs
https://github.com/stauffec/survival/blob/master/Survival/modelclass.cpp
(Note, in my modelclass.cpp, I am using the InitializeOpenModel method for loading the assimp model).
I really appreciate any feedback, and I’m going to continue to dig through your code.
I guess one other thing worth mentioning is I’m not using your latest code on code.google.com, but instead I’m using the 2 files from your first post and modified them to use the D3DXMATRIX and transpose ( you can see it in my project).
First check to see if the model is correct by removing the animation transform in your shader to look like this
if(input.boneindices.w != -1)
animation += input.boneweights.w * g_matrices[input.boneindices.w];
//comment these two lines out
//if(input.boneindices.x != -1)
//output.position = mul(input.position, animation);
output.position = mul(output.position, worldMatrix);
try the code out. It should draw your model in its bind pose. if its all screwed up, the the problem is elsewhere.
I looked over your code and its difficult to tell whether there is a problem anywhere.
If you have an amd video card like I do download gpuperfstudio
http://developer.amd.com/tools/graphics-development/gpu-perfstudio-2/
when you run the program, it will ask to to select an exe. The file should be your program. Once you launch the program, press the pause button, then goto the menu, select Window -> Frame debugger.
This will give you a breakdown of all your draw calls, you can see the vertex buffer, and all sorts of information. This might help you find the problem as well.
let me know what happens
Scott, I’m getting very close to figuring it out. I made sure the static model rendered fine. I then also confirmed that if I send identity matrix, that works – except if I try and apply a bone weight of 1.0. It appears that the weights are somehow getting lost on there way to the pipeline. I’ll hunt it down, but I’m very close.
try changing your vertex format postition to be 4 component and the same for normal.
Also, check to make sure your weights all add up to one.
So when you load the model, run a loop over your weights to see if any will be over 1. In fact, the four vertex weights should always add up to 1
Thanks Scott, so I did lookat the 4 component vertex format and there was a bug in there. That is fixed. I’m going to check the weight, although last time I checked there appeared to definitely be summed weights that were less than 1. I’ll test it again now.
So, have you got the model to draw without animations– just a static mesh in the bind pose?
Yes. Also, I’m seeing something interesting. It appears that more than 4 bones are being bound to a single vertex. That is why things are getting messed up. Was 4 an assimp limit?
Yeah. Assimp has a max of 4 weights per vertex
Interesting, so until I actually put the postprocessing step in: aiProcess_LimitBoneWeights, it didn’t respect the 4 and was using higher than four. So that issue is resolved. However, the model appears to be stretched out into some point in the distance still – so it looks like the animation is working, but clearly some transformation is screwing it up. The hunt continues.
Mystery Solved! I didn’t realize there was a transpose that you made on the bone matrices that were returned. Once I did the transpose, all worked well.
Excellent! Yeah, the matrix need to be transposed before being sent to the video card.
If you add any features to the animation code let me know what you have done so I can add them to the main code.
Also, if there are any suggestions, let me know so I can make them.