<br><div class="gmail_quote">On Wed, Mar 18, 2009 at 10:25 PM, Ceki Gulcu <span dir="ltr"><ceki@qos.ch></span> wrote:<br><div>Â </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Are you using the standard corpus with 50'000? If so, 12MB for 50'000 elements translates to about 240 bytes per message which is pretty good. The CPU times, e.g. 1883 ms, are fantastic.</blockquote><div><br>yep,<br>
<br>ILoggingEvent[] eventArray = Corpus.makeStandardCorpus(); <br><br>Â Â Â for (int i=0; i<10; i++) {<br>Â Â Â Â Â ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("/tmp/logback-java.ser." + i));<br>
     OutputStream os = new BufferedOutputStream(new FileOutputStream("/tmp/logback-protobuf.ser" + i));<br>     GZIPOutputStream gos = new GZIPOutputStream(new FileOutputStream("/tmp/logback-proto." + i + ".gz"), 8000);<br>
     <br>     DeflaterOutputStream dos = new DeflaterOutputStream(<br>         new FileOutputStream("/tmp/logback-protobuf.def." + i),<br>         new Deflater(Deflater.BEST_SPEED));<br><br>     serialize(eventArray, oos);<br>
     serializeWithProtobuf("normal", eventArray, os);<br>     serializeWithProtobuf("gzip ", eventArray, gos);<br>     serializeWithProtobuf("def  ", eventArray, dos);<br>     oos.close();<br>
     os.close();<br>     gos.close();     <br>   }<br><br>Still have to create a multi-threaded test, but I suspect it won't make much difference.<br><br>Maarten<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<br>
Maarten Bosteels wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div class="h5">
Hi Ceki,<br>
<br>
Keep in mind that I still have to double check my LoggingEvent-to-protobuf converter.<br>
<br>
Serializing the corpus with different formats:<br>
java serialization  => 16 MB  (16108602)              protobuf         => 33 MB  (34385867 bytes)<br>
protobuf + gzip    => 9.9 MB  (10354646 bytes)<br>
protbuf + deflater  =>  12 MB  (12006206 bytes)<br>
<br>
speed:<br>
<br>
java serialization: Â Â 4330 ms<br>
protobuf          928 ms<br>
protobuf + gzip    3146 ms protobuf + deflater  1883 ms<br>
<br>
So I would choose for protobuf + deflater : Â 25% smaller than java serilaization and twice as fast.<br>
<br>
Deflater means using  java.util.zip.Deflater.BEST_SPEED<br>
<br>
regards,<br>
Maarten<br>
<br>
<br>
On Wed, Mar 18, 2009 at 9:49 PM, Ceki Gulcu <ceki@qos.ch> wrote:<br>
<br>
<br>
  Hello Maarten,<br>
<br>
  It's interesting to learn that protobuf is faster than plain old<br>
  java serialization. Thanks.<br>
<br>
  How about the size of the stored data? How many bytes does a logging<br>
  event use on average?<br>
<br>
  Maarten Bosteels wrote:<br>
<br>
    Hi,<br>
<br>
    No objections at all.<br>
<br>
    Looking at my .proto file it's indeed pretty obvious that<br>
    CallerData and StackTraceElement are quite similar :-)<br>
    <a href="http://code.google.com/p/firewood/source/browse/trunk/compare-formats/src/main/java/com/googlecode/firewood/protobuf/logging.proto" target="_blank">http://code.google.com/p/firewood/source/browse/trunk/compare-formats/src/main/java/com/googlecode/firewood/protobuf/logging.proto</a><br>
<br>
    I've done some benchmarking with protobuf (using the Corpus),<br>
    and it seems to be at least 4 times faster than Java serialization.<br>
    Will soon post the results and the code.<br>
<br>
    regards,<br>
    Maarten<br>
<br>
<br>
  --   Ceki Gülcü<br>
  Logback: The reliable, generic, fast and flexible logging framework<br>
  for Java.<br>
  <a href="http://logback.qos.ch" target="_blank">http://logback.qos.ch</a><br>
  _______________________________________________<br>
  logback-dev mailing list<br>
  logback-dev@qos.ch<br>
  <a href="http://qos.ch/mailman/listinfo/logback-dev" target="_blank">http://qos.ch/mailman/listinfo/logback-dev</a><br>
<br>
<br>
<br></div></div>
------------------------------------------------------------------------<div class="im"><br>
<br>
_______________________________________________<br>
logback-dev mailing list<br>
logback-dev@qos.ch<br>
<a href="http://qos.ch/mailman/listinfo/logback-dev" target="_blank">http://qos.ch/mailman/listinfo/logback-dev</a><br>
</div></blockquote><div><div></div><div class="h5">
<br>
-- <br>
Ceki Gülcü<br>
Logback: The reliable, generic, fast and flexible logging framework for Java.<br>
<a href="http://logback.qos.ch" target="_blank">http://logback.qos.ch</a><br>
_______________________________________________<br>
logback-dev mailing list<br>
logback-dev@qos.ch<br>
<a href="http://qos.ch/mailman/listinfo/logback-dev" target="_blank">http://qos.ch/mailman/listinfo/logback-dev</a><br>
</div></div></blockquote></div><br>