Simulator Statistics Testing, Planning for Patches, Next Steps

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Simulator Statistics Testing, Planning for Patches, Next Steps

Maxwell, Douglas CIV USARMY RDECOM ARL (US)

Good Morning All,  in previous email I described the simulator statistics reporting adjustments we have been working on.  This group requested that we avoid sending a major code drop, so we have staggered the code updates into three different stats patches.  You could call the first patch a dry run of sorts to allow us to ensure proper formatting and integration on your side.  Looks like all we need to do is sort out the white spaces issue in the patch and we will be good to go.

 

For your planning purposes, here is the roadmap for all three stats patches:

 

Patch 1: Simulator FPS, Physics FPS, Total Frame Time, Physics Frame Time, and Simulation Time. 

Patch 2: correct the broken user login freezes, XEngine Thread Count, Util Thread Count, System Thread Count, Proc Mem, Frame Dilation (previous the proposed Time Dilation).

Patch 3: network reporting:  UDPIn, UDPOut, UDPInError, PktsIn, PktsOut, Avatar to IP list, Server-to-Client Ping, and Average Server Ping (External).

 

The attached screenshot shows Patch 1 in action on the MOSES grid.  Based on your feedback, we did not affect how the script engine reports Time Dilation, however the simulator frame rate is reported accurately.  This is demonstrated via a scripted object (green box).  The statistics window reports Time Dilation accurately and is a constantly changing value based on simulator load, specifically physics interactions.  (Forgive the low client frame rate, I took the screenshot on an older laptop.)  Some of you have seen this screenshot, I'm providing it again for context.

 

What's next?  Physics & Networking.  Just tossing physical balls into a pit or logging in a few dozen bots isn't an adequate test.  There are too many variables at play and even simulator<->physics engine integration code is wickedly complex.  We need to be able to rely on the reporting of the simulator's behavior so we can know what adjustments are meaningful.  We don't yet know if PhysX is better than Bullet, but thorough and accurate testing is the only way to find out.

 

What's the potential payoff for you?  Imagine being able to hold an event like the OSCC in a single large var region.  Imagine having lots of people being able to log into a region simultaneously, rather than in serial like today.  Wouldn't it be nice to be able to actually plan for what hardware and network support you need for a grid, rather than just guessing and buying the best you can afford - not what you actually need?

 

This is what we are working towards.  If we can agree that these changes are valid and necessary, then I invite you to inspect our work.  If you object to the way we have implemented the changes, then we welcome feedback and will work with you to find a more compatible path. 

 

In approximately 9 months, we will begin testing some new simulation based training material that involves large numbers of people and I would like to have a version of MOSES ready in time to support.  My only alternative is to continue licensing a very expensive proprietary engine.  It is my desire to cancel that license and devote those funds to open simulator development, but only if the MOSES can handle the work.

 

 

Douglas Maxwell, MSME
Science and Technology Manager
Virtual World Strategic Applications
U.S. Army Research Lab
Human Research & Engineering Directorate
Simulation & Training Technology Center
(c) <a tabindex="0" style="WHITE-SPACE: normal; WORD-SPACING: 0px; TEXT-TRANSFORM: none; COLOR: rgb(17,85,204); FONT: 13px arial,sans-serif; LETTER-SPACING: normal; BACKGROUND-COLOR: rgb(255,255,255); TEXT-INDENT: 0px" href="tel:%28407%29%20242-0209" target="" value="&#43;14072420209">(407) 242-0209

_______________________________________________
Opensim-dev mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev

ABWIS MOSES Prerelease_001.jpg (229K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Simulator Statistics Testing, Planning for Patches, Next Steps

Kevin Cozens
On 2015-04-26 07:28, Maxwell, Douglas CIV USARMY ARL (US) wrote:
> Patch 1: Simulator FPS, Physics FPS, Total Frame Time, Physics Frame
> Time, and Simulation Time.

My main computer is down and in need of repair or upgrade so I can't
test out the suggested code changes at this time. The one thing I wanted
to comment on is the use of a hard-coded variable in the patch. There is
a value of 10 in the patch that is buried in the code. I would suggest
you consider defining a variable at the top of the file to hold the
value. Should the value ever need to be changed, or needed in some other
file, it would be easier to update or use it if it was more readily
visible near the top of the file.

I have some other comments I was going to make but they will have to
wait as I have to go out in just a few minutes.

--

Cheers!

Kevin.

http://www.ve3syb.ca/           |"Nerds make the shiny things that
distract
Owner of Elecraft K2 #2172      | the mouth-breathers, and that's why
we're
                                 | powerful!"
#include <disclaimer/favourite> |             --Chris Hardwick

_______________________________________________
Opensim-dev mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
Reply | Threaded
Open this post in threaded view
|

Re: Simulator Statistics Testing, Planning for Patches, Next Steps

Dahlia Trimble
In reply to this post by Maxwell, Douglas CIV USARMY RDECOM ARL (US)
Thank you Doug, those indeed do look like desirable enhancements. I'm sure many of the developers will do what we can to help you and the MOSES team bring them into core.

On Sun, Apr 26, 2015 at 4:28 AM, Maxwell, Douglas CIV USARMY ARL (US) <[hidden email]> wrote:

Good Morning All,  in previous email I described the simulator statistics reporting adjustments we have been working on.  This group requested that we avoid sending a major code drop, so we have staggered the code updates into three different stats patches.  You could call the first patch a dry run of sorts to allow us to ensure proper formatting and integration on your side.  Looks like all we need to do is sort out the white spaces issue in the patch and we will be good to go.

 

For your planning purposes, here is the roadmap for all three stats patches:

 

Patch 1: Simulator FPS, Physics FPS, Total Frame Time, Physics Frame Time, and Simulation Time. 

Patch 2: correct the broken user login freezes, XEngine Thread Count, Util Thread Count, System Thread Count, Proc Mem, Frame Dilation (previous the proposed Time Dilation).

Patch 3: network reporting:  UDPIn, UDPOut, UDPInError, PktsIn, PktsOut, Avatar to IP list, Server-to-Client Ping, and Average Server Ping (External).

 

The attached screenshot shows Patch 1 in action on the MOSES grid.  Based on your feedback, we did not affect how the script engine reports Time Dilation, however the simulator frame rate is reported accurately.  This is demonstrated via a scripted object (green box).  The statistics window reports Time Dilation accurately and is a constantly changing value based on simulator load, specifically physics interactions.  (Forgive the low client frame rate, I took the screenshot on an older laptop.)  Some of you have seen this screenshot, I'm providing it again for context.

 

What's next?  Physics & Networking.  Just tossing physical balls into a pit or logging in a few dozen bots isn't an adequate test.  There are too many variables at play and even simulator<->physics engine integration code is wickedly complex.  We need to be able to rely on the reporting of the simulator's behavior so we can know what adjustments are meaningful.  We don't yet know if PhysX is better than Bullet, but thorough and accurate testing is the only way to find out.

 

What's the potential payoff for you?  Imagine being able to hold an event like the OSCC in a single large var region.  Imagine having lots of people being able to log into a region simultaneously, rather than in serial like today.  Wouldn't it be nice to be able to actually plan for what hardware and network support you need for a grid, rather than just guessing and buying the best you can afford - not what you actually need?

 

This is what we are working towards.  If we can agree that these changes are valid and necessary, then I invite you to inspect our work.  If you object to the way we have implemented the changes, then we welcome feedback and will work with you to find a more compatible path. 

 

In approximately 9 months, we will begin testing some new simulation based training material that involves large numbers of people and I would like to have a version of MOSES ready in time to support.  My only alternative is to continue licensing a very expensive proprietary engine.  It is my desire to cancel that license and devote those funds to open simulator development, but only if the MOSES can handle the work.

 

 

Douglas Maxwell, MSME
Science and Technology Manager
Virtual World Strategic Applications
U.S. Army Research Lab
Human Research & Engineering Directorate
Simulation & Training Technology Center
(c) <a style="WHITE-SPACE:normal;WORD-SPACING:0px;TEXT-TRANSFORM:none;COLOR:rgb(17,85,204);FONT:13px arial,sans-serif;LETTER-SPACING:normal;BACKGROUND-COLOR:rgb(255,255,255);TEXT-INDENT:0px" href="tel:%28407%29%20242-0209" value="+14072420209" target="_blank">(407) 242-0209

_______________________________________________
Opensim-dev mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev



_______________________________________________
Opensim-dev mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
Reply | Threaded
Open this post in threaded view
|

Re: Simulator Statistics Testing, Planning for Patches, Next Steps

Melanie-2
In reply to this post by Maxwell, Douglas CIV USARMY RDECOM ARL (US)
I would like to use the opportunity to toss in my $.02 regarding
time dilation in scripts.

Time dilation as a LL function is defined to return a value between
0 and 1, which, while not as informative as the new dilation stat,
is useful to make scripts perform better under bad simulator conditions.

I would postulate that the only case in which any script in OpenSim
uses this function is if it was copy/pasted from SL. Since the
return value is a constant 1, I doubt any software written for
OpenSim specifically even uses it.

A SL compatible return value would improve the copy/pasted scripts
from SL and not adversely affect OpenSim scripts which don't use it
anyway.

I would therefore like to propose to change the return value of the
LL function to return an SL-compatible value and, should there be
interest, maybe introduce a companion OS function returning the raw
value, including the positive values greater 1.

- Melanie

On 26/04/2015 13:28, Maxwell, Douglas CIV USARMY ARL (US) wrote:

> Good Morning All,  in previous email I described the simulator statistics reporting adjustments we have been working on.  This group requested that we avoid sending a major code drop, so we have staggered the code updates into three different stats patches.  You could call the first patch a dry run of sorts to allow us to ensure proper formatting and integration on your side.  Looks like all we need to do is sort out the white spaces issue in the patch and we will be good to go.
>
>
>
> For your planning purposes, here is the roadmap for all three stats patches:
>
>
>
> Patch 1: Simulator FPS, Physics FPS, Total Frame Time, Physics Frame Time, and Simulation Time.
>
> Patch 2: correct the broken user login freezes, XEngine Thread Count, Util Thread Count, System Thread Count, Proc Mem, Frame Dilation (previous the proposed Time Dilation).
>
> Patch 3: network reporting:  UDPIn, UDPOut, UDPInError, PktsIn, PktsOut, Avatar to IP list, Server-to-Client Ping, and Average Server Ping (External).
>
>
>
> The attached screenshot shows Patch 1 in action on the MOSES grid.  Based on your feedback, we did not affect how the script engine reports Time Dilation, however the simulator frame rate is reported accurately.  This is demonstrated via a scripted object (green box).  The statistics window reports Time Dilation accurately and is a constantly changing value based on simulator load, specifically physics interactions.  (Forgive the low client frame rate, I took the screenshot on an older laptop.)  Some of you have seen this screenshot, I'm providing it again for context.
>
>
>
> What's next?  Physics & Networking.  Just tossing physical balls into a pit or logging in a few dozen bots isn't an adequate test.  There are too many variables at play and even simulator<->physics engine integration code is wickedly complex.  We need to be able to rely on the reporting of the simulator's behavior so we can know what adjustments are meaningful.  We don't yet know if PhysX is better than Bullet, but thorough and accurate testing is the only way to find out.
>
>
>
> What's the potential payoff for you?  Imagine being able to hold an event like the OSCC in a single large var region.  Imagine having lots of people being able to log into a region simultaneously, rather than in serial like today.  Wouldn't it be nice to be able to actually plan for what hardware and network support you need for a grid, rather than just guessing and buying the best you can afford - not what you actually need?
>
>
>
> This is what we are working towards.  If we can agree that these changes are valid and necessary, then I invite you to inspect our work.  If you object to the way we have implemented the changes, then we welcome feedback and will work with you to find a more compatible path.
>
>
>
> In approximately 9 months, we will begin testing some new simulation based training material that involves large numbers of people and I would like to have a version of MOSES ready in time to support.  My only alternative is to continue licensing a very expensive proprietary engine.  It is my desire to cancel that license and devote those funds to open simulator development, but only if the MOSES can handle the work.
>
>
>
>
>
> Douglas Maxwell, MSME
> Science and Technology Manager
> Virtual World Strategic Applications
> U.S. Army Research Lab
> Human Research & Engineering Directorate
> Simulation & Training Technology Center
> (c) (407) 242-0209<tel:%28407%29%20242-0209>
>
>
>
> _______________________________________________
> Opensim-dev mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
_______________________________________________
Opensim-dev mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
Reply | Threaded
Open this post in threaded view
|

Re: Simulator Statistics Testing, Planning for Patches, Next Steps

Mike Chase
+1.  I think thats the most sensible approach.  Time Dilation is the one
case where the changes really changed the *meaning* of a value as
opposed to just the number.   I would keep it SL compatible for LLXXX
lsl functions and define a new one to return the new value that was defined.

Mike

On 04/26/2015 04:11 PM, Melanie wrote:

> I would like to use the opportunity to toss in my $.02 regarding
> time dilation in scripts.
>
> Time dilation as a LL function is defined to return a value between
> 0 and 1, which, while not as informative as the new dilation stat,
> is useful to make scripts perform better under bad simulator conditions.
>
> I would postulate that the only case in which any script in OpenSim
> uses this function is if it was copy/pasted from SL. Since the
> return value is a constant 1, I doubt any software written for
> OpenSim specifically even uses it.
>
> A SL compatible return value would improve the copy/pasted scripts
> from SL and not adversely affect OpenSim scripts which don't use it
> anyway.
>
> I would therefore like to propose to change the return value of the
> LL function to return an SL-compatible value and, should there be
> interest, maybe introduce a companion OS function returning the raw
> value, including the positive values greater 1.
>
> - Melanie
>
> On 26/04/2015 13:28, Maxwell, Douglas CIV USARMY ARL (US) wrote:
>> Good Morning All,  in previous email I described the simulator statistics reporting adjustments we have been working on.  This group requested that we avoid sending a major code drop, so we have staggered the code updates into three different stats patches.  You could call the first patch a dry run of sorts to allow us to ensure proper formatting and integration on your side.  Looks like all we need to do is sort out the white spaces issue in the patch and we will be good to go.
>>
>>
>>
>> For your planning purposes, here is the roadmap for all three stats patches:
>>
>>
>>
>> Patch 1: Simulator FPS, Physics FPS, Total Frame Time, Physics Frame Time, and Simulation Time.
>>
>> Patch 2: correct the broken user login freezes, XEngine Thread Count, Util Thread Count, System Thread Count, Proc Mem, Frame Dilation (previous the proposed Time Dilation).
>>
>> Patch 3: network reporting:  UDPIn, UDPOut, UDPInError, PktsIn, PktsOut, Avatar to IP list, Server-to-Client Ping, and Average Server Ping (External).
>>
>>
>>
>> The attached screenshot shows Patch 1 in action on the MOSES grid.  Based on your feedback, we did not affect how the script engine reports Time Dilation, however the simulator frame rate is reported accurately.  This is demonstrated via a scripted object (green box).  The statistics window reports Time Dilation accurately and is a constantly changing value based on simulator load, specifically physics interactions.  (Forgive the low client frame rate, I took the screenshot on an older laptop.)  Some of you have seen this screenshot, I'm providing it again for context.
>>
>>
>>
>> What's next?  Physics & Networking.  Just tossing physical balls into a pit or logging in a few dozen bots isn't an adequate test.  There are too many variables at play and even simulator<->physics engine integration code is wickedly complex.  We need to be able to rely on the reporting of the simulator's behavior so we can know what adjustments are meaningful.  We don't yet know if PhysX is better than Bullet, but thorough and accurate testing is the only way to find out.
>>
>>
>>
>> What's the potential payoff for you?  Imagine being able to hold an event like the OSCC in a single large var region.  Imagine having lots of people being able to log into a region simultaneously, rather than in serial like today.  Wouldn't it be nice to be able to actually plan for what hardware and network support you need for a grid, rather than just guessing and buying the best you can afford - not what you actually need?
>>
>>
>>
>> This is what we are working towards.  If we can agree that these changes are valid and necessary, then I invite you to inspect our work.  If you object to the way we have implemented the changes, then we welcome feedback and will work with you to find a more compatible path.
>>
>>
>>
>> In approximately 9 months, we will begin testing some new simulation based training material that involves large numbers of people and I would like to have a version of MOSES ready in time to support.  My only alternative is to continue licensing a very expensive proprietary engine.  It is my desire to cancel that license and devote those funds to open simulator development, but only if the MOSES can handle the work.
>>
>>
>>
>>
>>
>> Douglas Maxwell, MSME
>> Science and Technology Manager
>> Virtual World Strategic Applications
>> U.S. Army Research Lab
>> Human Research & Engineering Directorate
>> Simulation & Training Technology Center
>> (c) (407) 242-0209<tel:%28407%29%20242-0209>
>>
>>
>>
>> _______________________________________________
>> Opensim-dev mailing list
>> [hidden email]
>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
> _______________________________________________
> Opensim-dev mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev

_______________________________________________
Opensim-dev mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev