Preliminary Performance Tests: mruby & V8 | ArangoDB Blog 2012
I’m still investigating the possibility to use mruby as embedded language for AvocadoDB, see me last post. I managed to create an interactive shell to play with mruby. Now am trying to do some performance tests.
Note that mruby is still “alpha”, so DO NOT take these test to seriously. They are basically just for me to learn, how to use mruby.
Exposing a C function to MRUBY
The first function I need is a timing function. I’ve implemented such a function in C using gettimeofday
. Now in order to make it accessible from mruby I’ve used
static mrb_value MR_Time (mrb_state* mrb, mrb_value self) {
return mrb_float_value(TRI_microtime());
}
void TRI_InitMRUtils (mrb_state* mrb) {
struct RClass *krn;
krn = mrb->kernel_module;
// timing function
mrb_define_method(mrb, krn, "time", MR_Time, ARGS_NONE());
}
mrb_define_method
defines a new method in a class or module. I’m not sure what the correct module or class is, so for my tests I’ve chosen the Kernel module. ARGS_NONE()
is used for functions that take no arguments.
MRuby
Testing it as follows works
fceller@huerth:~/Arango-DB-MRUBY> ./avocirb
_ _
__ ___ _____ ___(_)_ __| |__
/ _` \ \ / / _ \ / __| | '__| '_ \
| (_| |\ V / (_) | (__| | | | |_) |
\__,_| \_/ \___/ \___|_|_| |_.__/
Welcome to avocirb 0.3.13. Copyright (c) 2012 triAGENS GmbH.
Using MRUBY 0.0.0 engine. Copyright (c) 2012 mruby developers.
Using READLINE 6.1.
avocirb> def fact n; (n > 1) ? n*fact(n-1) : 1; end
avocirb> $a = time; $b = fact(100); p time - $a
9.08374786376953e-05
As expected it is too fast to be usable. So, I’ve used a loop
avocirb> $a = time; for i in 1 .. 10000000; fact(10.0); end; p time - $a
20.9153339862823
avocirb> $a = time; for i in 1 .. 10000000; $b = fact(10.0); end; p time - $a
21.3838360309601
avocirb> p $b
3628800.0
So there is also no big difference between integer and floats. Both of with are stored as immediate values in mruby. Note that there is no Bignum in mruby.
JavaScript
The same in JavaScript V8 give:
_
__ ___ _____ ___ ___| |__
/ _` \ \ / / _ \ / __/ __| '_ \
| (_| |\ V / (_) | (__\__ \ | | |
\__,_| \_/ \___/ \___|___/_| |_|
Welcome to avocsh 0.3.13. Copyright (c) 2012 triAGENS GmbH.
Using Google V8 3.9.4 JavaScript engine.
Using READLINE 6.1.
avocsh> function fact (n) { return 1 < n ? n * fact(n-1) : 1; }
avocsh> a = internal.time(); for (var i = 0; i < 10000000; ++i) { b = fact(10); } print(internal.time() - a);
0.6311039924621582
avocsh> a = internal.time(); for (var i = 0; i < 10000000; ++i) { b = fact(10.0); } print(internal.time() - a);
0.6052999496459961
avocsh> print(b);
3628800
JavaScript is currently faster – but note that mruby is still in an very early stage. BTW, I’ve used “-O2” for both V8 and mruby.
Ruby 1.8.7
Using “irb” one gets
irb(main):001:0> def fact n; (n > 1) ? n*fact(n-1) : 1; end
=> nil
irb(main):011:0> $a = Time.now; for i in 1 .. 10000000; $b = fact(10); end; p Time.now - $a
49.044638
=> nil
irb(main):010:0> $a = Time.now; for i in 1 .. 10000000; $b = fact(10.0); end; p Time.now - $a
61.320039
So in this particular example, mruby is a lot faster than Ruby 1.8. Again, it is too early to draw any conclusions.
Ruby 1.9
Using “irb” one gets
irb(main):001:0> def fact n; (n > 1) ? n*fact(n-1) : 1; end
=> nil
irb(main):002:0> $a = Time.now; for i in 1 .. 10000000; $b = fact(10); end; p Time.now - $a
7.81386388
=> 7.81386388
irb(main):003:0> $a = Time.now; for i in 1 .. 10000000; $b = fact(10.0); end; p Time.now - $a
22.378524013
=> 22.378524013
Next Steps
I need to do more tests to see how easy it is to really embed the language in to ArangoDB. And I also need to do some more tests.
2 Comments
Leave a Comment
Get the latest tutorials, blog posts and news:
I was interested in whether the huge performance difference you measured was due to MRuby being (probably) still unoptimized or due to V8 being much faster than other dynamic language executors. So I measured myself.
Results from a 32 bit laptop follow:
MRuby (0.0.0):
avocirb> def fact n; (n > 1) ? n*fact(n-1) : 1; end
avocirb> $a = time; for i in 1 .. 10000000; fact(10.0); end; p time – $a
57.8161611557007
avocirb> $a = time; for i in 1 .. 10000000; $b = fact(10.0); end; p time – $a
59.1052560806274
avocirb> p $b
3628800.0
Javascript (V8 3.9.4):
avocado> function fact (n) { return 1 a = internal.time(); for (var i = 0; i a = internal.time(); for (var i = 0; i print(b);
3628800
The results are in line with what you measured. V8 was about the same magnitude faster than MRuby as in your test.
For curiosity I also cross checked the results against PHP.
PHP (5.3.6):
cli> php -r ‘function fact($n) { return ($n > 1) ? $n * fact($n – 1) : $n; } $a = microtime(true); for ($i = 1; $i php -r ‘function fact($n) { return ($n > 1) ? $n * fact($n – 1) : $n; } $a = microtime(true); for ($i = 1; $i < 10000000; $i++) { $b = fact(10.0); } print microtime(true) – $a; print "n"; print $b; print "n";'
36.732476949692
3628800
So MRuby and PHP execution times at least have the same order of magnitude, which was what I expected because I think they have the same code execution model.
Comparing Ruby1.9 and MacRuby (another JITted Ruby) on my MBA (which is slightly slower than the other machines used here):
ruby1.9 ~/tmp/ben1.rb11.44724430.074118macruby ~/tmp/ben1.rb4.6390053.818616Obviously the unboxed floating point values of MacRuby seriously win, but also the JIT seems to win (but not as much as V8, unsurprisingly).V8 version 2.2.4.2> function fact (n) { return 1 a = new Date; for (var i = 0; i a = new Date; for (var i = 0; i (these are milliseconds).