If this is the case, then there seem to be a very serious problem.
I run our agents with same configuration several times. To give you an
example,I obtained 47.672 point in one run and 58.401 point in another.
This is a very big difference.
If this difference was a little, this might not be a concern but if
such a system is used in a competition environment we should pay
attention on this issue. This is not fortune game! By looking at
these results I may be ranked as last or first by chance. Of course,
there are several runs and it has very low possibility that a bad
implementation get good ranks but this is the case. Among good
implementation chance factors become very important. If team A slighly
perform better than team B in a configuration in one run, in another
run team B may perform better.
Another interesting point that I want to mention about is this. I
implemented empty agents that just connect to kernel and do nothing else
during simulation. I believed that this should be the worst case for the
configuration. However,when I run these agents and some other team's
code and our code, I saw that even empty agents perform better in some
cases although very rare! In fact, they don't perform better, but
simulation environment performs differently!
Are we missing some point or is this the case?
-----Özgün İleti-----
Kimden: owner-r-resc-l@usc.edu [mailto:owner-r-resc-l@usc.edu] Yerine s
post
Tarih: Tuesday, December 16, 2003 2:23 PM
Kime: r-resc-l@usc.edu
Konu: Re: A question related with 2003 rules
Hellow everybody,
As far as I know, messages from the agents & simulators to the kernel do
not always arrive in the same order. This is because agents & simulators
sometimes may run a little faster or slower depending on the load of the
system. Therefore it would not seem logical to think about `the same
seed
for each run', since communication issues prevent repeatable simulations
anyway.
I am currently in the process of writing a detailed document on these
issues. In a later stage my master thesis will become available at
http://www.science.uva.nl/~arnoud/research/roboresc/
For the moment however I have made the chapter about the RCRSS system
itself available at http://carol.science.uva.nl/~sbmpost/RCR/rcrss.ps
I think this chapter may prove a useful resource for all RoboCupRescue
participants.
S.post, UvA university, Netherlands
On Mon, 15 Dec 2003 eker@uekae.tubitak.gov.tr wrote:
> Hi all,
>
> It says in 2003 rules that:
> "We have to ensure that the competition field for competing teams is
> identical not only parameters but also the seeds of random numbers
used in
> simulators, -- the miscsimulator, collapse simulator, fire simulator."
>
> I'm using version 0.43. Although I'm using same
> configuration I get different scores in different
> runs. This means that most probably some simulators
> starts from different initial seeds in different runs.
> How I can ensure that simulators start always from
> same initial seed?
>
> Best regards,
> Baris Eker
> RoboAkut
>
>
>
This archive was generated by hypermail 2.1.3 : Tue 16 Dec 2003 - 13:37:36 GMT