physical audio

[Dylan Menzies]

Hi, I am new on this forum. I have developed a library for integrating physically modelled sound with rigid body engines, which goes back a few years, although its been gathering dust. The motivation for physically modelled audio is that its a lot more realistic and interesting than sample based sound, and is a natural match for macro-physical behaviour. Together these two dynamics cover the perceptual bandwidth.

Here are some demos with an old engine. They will probably run too fast..
http://www.zenprobe.com/dylan/project/phya

I'm considering turning this into an open source project, bullet looks like a good candidate for providing some new demos, and maybe later more integration. First thing that grabbed my eye was the comprehensive contact coverage, including proper cylinders, and then mention of persistent contacts, and seperate impacts.

Anyways my experience is that engines vary quite a bit in their usability with physical audio, so I would like some opinions. The main requirement is to find when engine contacts are made or broken, what their current dynamics are. If they are persistent great, if not this is not usually a disaster, provided the body pair lists for contacts can be scanned.

Any comments appreciated

Dylan Menzies

[Dylan Menzies]

To follow up the previous message, I have some specific questions-

- The btBroadphasePair passed to a customNearCallback appears persistent. Would it be unreasonable to request a userdata option on this object to allow cross referencing to acoustic contact objects?

- Can persistent contact data also be found from a btPersistentManifold in the customNearCallback? There doesn't seem to be a userdata option here either.

The alternative would appear to build a linked list of (btBroadphasePair, acoustic contact).

thanks for any help.

[Erwin Coumans]

Hi,

Thanks for sharing this info, that sounds pretty cool. It would be great to have a demo combinding physics-based sound and Bullet collision/rigidbody dynamics.

The btBroadphasePair passed in a customNearCallback is persistent, and has a user info pointer. However, under certain conditions this user-info pointer is used by the rigidbody dynamics system internally (and destroyed during the gContactDestroyedCallback, see around line 110 of bullet\src\BulletDynamics\ConstraintSolver\btSequentialImpulseConstraintSolver.cpp.

It should be possible to either use this user-info pointer in btBroadphasePair , or extend the system to make it easier to add a user pointer (with added/destruction callback).

Are you using BulletCollision collision detection only, or do you combine it will BulletDynamics rigid bodies?
Thanks,
Erwin

[Dylan Menzies]

I think btBroadphasePair user pointer would be the best from an integration viewpoint. It would be need to be maintained for the duration of the contact, so I guess that would not be possible as it stands. I'm not clear when a btBroadphasePair begins. It seems to be just before contact. This is ok, but its better to avoid using acoustic contacts that are not needed, so maybe something using the persistent contact manifold might be better.

At the moment I am just trying to figure out the best approach. I would intend to use the dynamics in bullet initially anyway. Visual inspection of the cylinders spinning down shows they behave well, and should give good acoustic results. They seem to switch off a bit too suddenly though, and also sometimes the spin down near the end is too slow, ie the rolling friction isn't totally working. In reality the spin down tends to accelerate in this last phase. Still, I don't think this would be a big deal. Physical audio is good with smooth surfaces - you can really 'hear out' the dynamics. If a surface is approximated with triangles you can get bumpy results unless you filter the dynamics. One way is to put filter switches on surfaces to describe the degree of smoothing.

[stenyak]

Are there any news? I'm pretty interested on realtime generated audio support for use with physic libraries.

[Dylan Menzies]

I have committed to doing this, but I have a lot of other things to do.. Hopefully won't be long.

[stenyak]

Ok, thanks for the update!

[Dylan Menzies]

Ok, I 've been doing some integration and its going ok, I have physical audio running in bullet. The problem I have right now is how to reset a reference to an audio contact from a dynamic pair, when the pair is no longer appearing in the collision call back, because objects have moved apart. For instance is there a callback for pairs that are being released?

Dylan

[Dylan Menzies]

It looks like I have a choice of two appraoches for handling contacts- generating audio contact interactions on a per body pair basis or per manifold point pasis. Both appear to be persistent and have user data.

The manifold point approach seems better supported with callbacks, but potentially could lead to uneccessary costly audio detail, or maybe better detail.

The pair approach seems to have less support for callbacks, such as pair removed, but provides a way of condensing physical audio interactions.

mmm. Doese this make sense? Are there things I am overlooking here before I blindly start hacking around? With other engines I have taken the pair approach, but here the manifold point may be better.

[Erwin Coumans]

Each overlapping pair (btBroadphasePair) can have one or more persistent contact manifolds (btPersistentManifold) created by its btCollisionAlgorithm. Each contact manifold has up to 4 contact points.

But right now it is hard to iterate/traverse those manifolds, given a btBroadphasePair, so it is easier to use contact point callbacks.

If necessary we can make it easier to get access to contact manifolds/contact points from a btBroadphasePair.
Obviously, you can modify the code to suit your audio needs, and let us know what improvements you recommend,

Hope this helps,
Erwin

[Dylan Menzies]

thanks for your reply.

I will try tracking manifold points -mps first.

I think I have found a bug. I have been listing mps using iteration each step and comparing with callbacks set with gContactAddedCallback gContactDestroyedCallback. I find calls to AddedCallback are completely inconsistent with the iteration monitor, although DestroyedCallback works fine once user data is set in the monitor. DestroyedCallback is the most important, so should be able to make this work without too much fuss.

I take it the user data on mps is never used by dynamics?

With complex objects what happens with multiple areas of contact, does this generate multiple manifolds?

I guess m_appliedImpulse acts along m_normalWorldOnB. Is there a friction force perpendicular to this or not?

Dylan

[Dylan Menzies]

I have it working now with audio contact on each manifold point, however this could be costly. To make it work with an audio contact per manifold, I would need a callback for manifold deletion, and user data in the manifold. Maybe you can point me where I should add the callback in bullet.

Dylan

[Erwin Coumans]

I would need a callback for manifold deletion, and user data in the manifold. Maybe you can point me where I should add the callback in bullet.

You can derive your 'audio' version of btCollisionDispatcher, and override the following virtual methods (and call the base class):

Code: Select all

	virtual btPersistentManifold*	getNewManifold(void* b0,void* b1);
	
	virtual void releaseManifold(btPersistentManifold* manifold);

I think I have found a bug. I have been listing mps using iteration each step and comparing with callbacks set with gContactAddedCallback gContactDestroyedCallback. I find calls to AddedCallback are completely inconsistent with the iteration monitor, although DestroyedCallback works fine once user data is set in the monitor. DestroyedCallback is the most important, so should be able to make this work without too much fuss.

Can you explain what you exactly mean by 'iteration monitor'? btPersistentManifold is a contact point cache, and it can contain 0,1,2,3 or 4 points. Those contact points are added, removed and updated every simulation step.

I take it the user data on mps is never used by dynamics?

Indeed. You can use m_userPersistentData in btManifoldPoint for your audio purposes.

With complex objects what happens with multiple areas of contact, does this generate multiple manifolds?

Indeed, it is possible that complex or composite objects generate multiple contact manifolds. If you need it, we can add a manifold iterator given a btBroadphasePair.

I guess m_appliedImpulse acts along m_normalWorldOnB. Is there a friction force perpendicular to this or not?

Yes, it is the applied impulse along the contact normal. In the past we also cached the applied impulse along the two friction directions. We can easily add this if you need it.

Hope this helps,
Erwin

[Dylan Menzies]

I would need a callback for manifold deletion, and user data in the manifold. Maybe you can point me where I should add the callback in bullet.

You can derive your 'audio' version of btCollisionDispatcher, and override the following virtual methods (and call the base class):

Code: Select all

	virtual btPersistentManifold*	getNewManifold(void* b0,void* b1);
	
	virtual void releaseManifold(btPersistentManifold* manifold);

Ok, thanks.

I think I have found a bug. I have been listing mps using iteration each step and comparing with callbacks set with gContactAddedCallback gContactDestroyedCallback. I find calls to AddedCallback are completely inconsistent with the iteration monitor, although DestroyedCallback works fine once user data is set in the monitor. DestroyedCallback is the most important, so should be able to make this work without too much fuss.

Can you explain what you exactly mean by 'iteration monitor'? btPersistentManifold is a contact point cache, and it can contain 0,1,2,3 or 4 points. Those contact points are added, removed and updated every simulation step.

I just mean taht I iterate to find out the current mps each step, and compare with calls to AddedCallback and DeletedCallback. Calls to AddedCallback are occuring even when the mps are remaining unchanged, and the mps passed by the callback are different.

I guess m_appliedImpulse acts along m_normalWorldOnB. Is there a friction force perpendicular to this or not?

Yes, it is the applied impulse along the contact normal. In the past we also cached the applied impulse along the two friction directions. We can easily add this if you need it.

Its not needed right now, I just wanted to understand the layout.
Thanks
Dylan

[Dylan Menzies]

A question about m_appliedImpulse : If the same simulation is run at twice the time step size, the corresponding values of m_appliedImpulse should be twice as big, so the average force over a step is found by dividing by the step size?