<div dir="ltr"><br><br><div class="gmail_quote">On Mon, Aug 4, 2008 at 10:21 PM, Ceki Gulcu <span dir="ltr"><listid@qos.ch></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Joern,<br>
<br>
Thanks for your input. Instead of forcing the test to sleep, another approach is</blockquote><div><br>The sleep was just meant as a placeholder for the code that will take some time, i.e. the actual code of the test. So I constructed one case where the test will work for sure, and one test that would always fail because it takes longer than the expected amount of time.<br>
I probably should have added some comments :)<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
to adjust the expected performance according to the capabilities of the host<br>
CPU. This is the approach adopted in BogoPerf. See<br>
<br>
<a href="http://svn.qos.ch/viewvc/logback/trunk/logback-classic/src/test/java/ch/qos/logback/classic/LoggerPerfTest.java" target="_blank">http://svn.qos.ch/viewvc/logback/trunk/logback-classic/src/test/java/ch/qos/logback/classic/LoggerPerfTest.java</a><br>
<a href="http://svn.qos.ch/viewvc/logback/trunk/logback-classic/src/test/java/ch/qos/logback/classic/util/BogoPerf.java" target="_blank">http://svn.qos.ch/viewvc/logback/trunk/logback-classic/src/test/java/ch/qos/logback/classic/util/BogoPerf.java</a><br>
<div><div></div><div class="Wj3C7c"></div></div></blockquote><div><br>I'll take a look for sure.<br><br>Joern. <br><br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div class="Wj3C7c"><br>
Joern Huxhorn wrote:<br>
> Ceki Gulcu wrote:<br>
>> Hello all,<br>
>><br>
>> Logback contains a small number of time-sensitive tests. The time needed to<br>
>> execute these tests depend on the capabilities of the host CPU. For example, if<br>
>> on my machine a test executes in 1 millisecond, I would expect it to<br>
>> finish in say 1*10 milliseconds on most machine -- allowing for 10 fold variation.<br>
>><br>
>> Does anyone know of a junit extension which can help me normalize the<br>
>> coefficient, 10 in the previous example, so that it adapts to the<br>
>> speed of the host CPU? There is also the problem of the JIT<br>
>> compiler...<br>
>><br>
>> Any recommendations on the subject?<br>
>><br>
>><br>
> Well, I don't have a real solution to your problem...<br>
> The only performance-related JUnit framework I'm aware of is<br>
> <a href="http://clarkware.com/software/JUnitPerf.html" target="_blank">http://clarkware.com/software/JUnitPerf.html</a> but I think it's outdated<br>
> and I never used it.<br>
> It could be useful for load-testing, though.<br>
><br>
> If you are using JUnit 4.x I'd suggest something like the following:<br>
><br>
> import org.junit.Test;<br>
> import org.junit.BeforeClass;<br>
> import org.junit.Before;<br>
> import static junit.framework.Assert.assertEquals;<br>
> import static junit.framework.Assert.fail;<br>
><br>
> public class TimingTest<br>
> {<br>
><br>
> private static long referenceTime;<br>
><br>
> @BeforeClass<br>
> public static void initReferenceTime()<br>
> {<br>
> long time=System.currentTimeMillis();<br>
><br>
> // lets waste some time...<br>
> StringBuffer msg=new StringBuffer();<br>
> for(int i=0;i<10000;i++)<br>
> {<br>
> msg.append(i);<br>
> }<br>
> System.out.println(msg.toString());<br>
> referenceTime=System.currentTimeMillis()-time;<br>
> System.out.println("ReferenceTime: "+referenceTime+" millis");<br>
> }<br>
><br>
> @Test(timeout = 1000)<br>
> public void works()<br>
> throws Exception<br>
> {<br>
> long time=System.currentTimeMillis();<br>
> Thread.sleep(referenceTime*5);<br>
> time=System.currentTimeMillis()-time;<br>
> long maxTime=referenceTime*10;<br>
> if(time>maxTime)<br>
> {<br>
> fail("Test was not expected to take more than "+maxTime+"<br>
> millis but took "+time+" millis!");<br>
> }<br>
> }<br>
><br>
> @Test(timeout = 1000)<br>
> public void fails()<br>
> throws Exception<br>
> {<br>
> long time=System.currentTimeMillis();<br>
> Thread.sleep(referenceTime*15);<br>
> time=System.currentTimeMillis()-time;<br>
> long maxTime=referenceTime*10;<br>
> if(time>maxTime)<br>
> {<br>
> fail("Test was not expected to take more than "+maxTime+"<br>
> millis but took "+time+" millis!");<br>
> }<br>
> }<br>
> }<br>
><br>
><br>
> The problem with this approach, however, is that referenceTime isn't<br>
> very stable. It fluctuates between 15ms and 40ms on my development<br>
> system. I guess this fluctuation could be reduced by using an even<br>
> higher number in the initReferenceTime loop.<br>
> The time-wasting would have to be changed depending on the<br>
> problem-domain of the real test, e.g. something like the above if String<br>
> operations are performed in the actual code that is tested.<br>
><br>
> The timeout I've given in @Test is supposed to be a definitive upper<br>
> level that should never be reached. This could be left out, though.<br>
><br>
> I have some doubts what the Hotspot-JIT will be doing if the above<br>
> initReferenceTime is used the same way in more than one class. I have<br>
> some hope that it does NOT realize that it's the same code but I'm not sure.<br>
><br>
> The code above isn't perfect, neither is it especially elegant, but it's<br>
> probably good enough for your use case if initReferenceTime and the<br>
> factors are tuned a bit...<br>
> I don't know.<br>
><br>
> Joern.<br>
</div></div><div class="Ih2E3d">> _______________________________________________<br>
> logback-dev mailing list<br>
> logback-dev@qos.ch<br>
> <a href="http://qos.ch/mailman/listinfo/logback-dev" target="_blank">http://qos.ch/mailman/listinfo/logback-dev</a><br>
<br>
</div><font color="#888888">--<br>
Ceki Gülcü<br>
<br>
QOS.ch is looking to hire talented developers located in Switzerland<br>
to work on cutting-edge software projects. If you think you are<br>
qualified, then please contact ceki@qos.ch.<br>
</font><div><div></div><div class="Wj3C7c">_______________________________________________<br>
logback-dev mailing list<br>
logback-dev@qos.ch<br>
<a href="http://qos.ch/mailman/listinfo/logback-dev" target="_blank">http://qos.ch/mailman/listinfo/logback-dev</a><br>
</div></div></blockquote></div><br></div>