25

In this test, we can see that the performance of golang is sometimes much slower than scala. In my opinion, since the code of golang is compiled directly to c/c++ compatible binary code, while the code of scala is compiled to JVM byte code, golang should have much better performance, especially in these computation-intensive algorithm the benchmark did. Is my understanding incorrect?

http://benchmarksgame.alioth.debian.org/u64/chartvs.php?r=eNoljskRAEEIAlPCA48ozD%2Bb1dkX1UIhzELXeGcih5BqXeksDvbs8Vgi9HFr23iGiD82SgxJqRWkKNctgkMVUfwlHXnZWDkut%2BMK1nGawoYeDLlYQ8eLG1tvF91Dd8NVGm4sBfGaYo0Pok0rWQ%3D%3D&m=eNozMFFwSU1WMDIwNFYoNTNRyAMAIvoEBA%3D%3D&w=eNpLz%2FcvTk7MSQQADkoDKg%3D%3D

8
  • 2
    That chart shows nothing about performance and everything about memory usage. You're asking the wrong question.
    – wheaties
    Jan 2, 2014 at 2:12
  • @user2864740 no, it shows memory used.
    – wheaties
    Jan 2, 2014 at 2:24
  • 1
    What is worse is that someone editted the post to take away the URL link to the test from which the graph was gotten. It's true that more context should have been written around it but now there's no chance of any of that context being reachable.
    – wheaties
    Jan 2, 2014 at 2:26
  • 2
    @Tyr looking at that link I see Go beating Scala most of the time. I'm not a Go fanboy, I write mostly Scala code but "benchmarks" like these aren't very useful in judging a language. You have to ask if the code really was optimized, if it was run long enough to capture real GC activity, etc.
    – wheaties
    Jan 2, 2014 at 2:37
  • 2
    In general: The JVM is not slow - has not been for a long time now. Go also does Garbage collection, boundary checks etc. Also they don't obviously do very extensive compile time optimizations in favor of extremely fast compile times. To speculate about the reasons why one or another benchmark is fast or slow does not make any sense in that general way: If you want to know, just grab a profiler and find out.
    – 0x434D53
    Jan 2, 2014 at 14:33

5 Answers 5

58

Here's what I think's going on in the four benchmarks where the go solutions are the slowest compared to the scala solutions.

  1. mandelbrot: the scala implementation has its internal loop unrolled one time. It may be also that the JVM can vectorise the calculation like this, which I think the go compiler doesn't yet do. This is good manual optimisation plus better JVM support for speeding arithmetic.
  2. regex-dna: the scala implementation isn't doing what the benchmark requires: it's asked to """(one pattern at a time) match-replace the pattern in the redirect file, and record the sequence length""" but it's just calculating the length and printing that. The go version does the match-replace so is slower.
  3. k-nucleotide: the scala implementation has been optimised by using bit-twiddling to pack nucleotides into a long rather than use chars. It's a good optimisation that could also be applied to the Go code.
  4. binary-trees: this tests gc performance by filling RAM. It's true that java gc is much faster than the go gc, but the argument for this not being the top priority for go is that usually one can avoid gc in real programs by not producing garbage in the first place.
1
  • 2
    The Scala regex-dna #5 program has been removed.
    – igouy
    Jan 7, 2014 at 1:40
20

This chart is from the Programming Shootout. You should read the disclaimers on the Shootout page before taking the benchmarks as gospel. At best these benchmarks are only useful for indicating broad expectations of performance.

That said, the JVM has a decade of well-funded optimization and apart from startup time, provides excellent performance for running code. Go is still a young language. The fact that Go comes within spitting distance of a JVM language is impressive. If you enjoy programming in Go, you should not reject it over one benchmark.

3
  • I'm not rejecting go. I just feel curious about the benchmark result.These cases should be pure computing intensive algorithm which JVM may not help a lot. As go is compiled directly to binaries it should have similar performance as C. Even how fast JIT in JVM is, it should not be in the same magnitude with native code. I also don't quite buy in that go is still young - it has been developed for several years. Anyway, seems there's no better explanation to this.
    – Tyr
    Jan 2, 2014 at 8:18
  • 3
    Not all compiled languages run at the same speed. Go possibly won't be as fast as C (but the gap will be reduced in the future). Garbage collector, boundary checking, etc. are useful features of the language that will slowdown the execution time.
    – siritinga
    Jan 2, 2014 at 8:41
  • >>You should read the disclaimers on the Shootout page before taking the benchmarks as gospel.<< -- You should take the benchmarks game measurements as gospel! They show exactly what they say they show. You should take the home page exhortations as gospel -- "Measurement is not prophesy."
    – igouy
    Jan 7, 2014 at 1:49
15

This is discussed in the go FAQ:

One of Go's design goals is to approach the performance of C for comparable programs, yet on some benchmarks it does quite poorly, including several in test/bench/shootout. The slowest depend on libraries for which versions of comparable performance are not available in Go. For instance, pidigits.go depends on a multi-precision math package, and the C versions, unlike Go's, use GMP (which is written in optimized assembler). Benchmarks that depend on regular expressions (regex-dna.go, for instance) are essentially comparing Go's native regexp package to mature, highly optimized regular expression libraries like PCRE.

Benchmark games are won by extensive tuning and the Go versions of most of the benchmarks need attention. If you measure comparable C and Go programs (reverse-complement.go is one example), you'll see the two languages are much closer in raw performance than this suite would indicate.

Still, there is room for improvement. The compilers are good but could be better, many libraries need major performance work, and the garbage collector isn't fast enough yet. (Even if it were, taking care not to generate unnecessary garbage can have a huge effect.)

As an aside, consider the 10x (!) speed difference between the different versions of a benchmark for a given programming language. C gcc #7 is 8.3 times slower than C gcc #5, and Ada #3 almost 10 times slower than Ada #5. These benchmarks provide a rough idea of how language compare, but the difference between Go and Scala is within one order of magnitude, which means any 'intrinsic' variation between the runtimes is likely to be dwarfed by differences in the implementation: this post describes how they sped up a program 11x by performing smarter memory allocation. Maybe the compiler/runtime should be handling this kind of optimisations automatically (as the JVM does, to a certain level), but I am not sure you can really draw the conclusion that 'Go is slower (resp. faster) than Scala' in the general case from these figures. Just my opinion though :)

1
  • >>any 'intrinsic' variation between the runtimes is likely to be dwarfed by differences in the implementation<< -- Not "likely to be", just "might be".
    – igouy
    Jan 7, 2014 at 1:54
11

Since you seem to be keen in looking at these biased benchmarks. Let's take a real example for real scenario not some Fibonacci implementations.

Take a look at these rankings for web frameworks benchmarks, the testing was done using native client if available and sometimes using OSS web frameworks, they also use many packages for testing with the same language. The tests vary from requests for raw strings to using ORM to query a database.

It is clear that Scala performance is no where close to Go, in all of the tests Scala was below Go. Having said this, benchmarks are nothing close to reality and I suggest you look at a language from tools/features perspective or simply what would be best to solve your problem.

3
  • 2
    Not "Fibonacci implementations". Not "biased".
    – igouy
    Jan 7, 2014 at 1:55
  • 2
    On better hardware (Dual Xeon E5 v2), go is slower than scala in 5 of 6 tests. JSON serializaion: go 31.7%, scala plain 18.7%. Single query: go 34.9%, scala 54.2%. Multiple queries: go 37.4%, scala plain 78.2%. Fortunes: go 8.8%, scala plain 52.1%. Data updates: go 72.4%, scala plain 91.6%. Plaintext: go 7.1%, scala plain 32.7%. techempower.com/benchmarks/previews/round9/…
    – rofrol
    Mar 11, 2014 at 10:20
  • not sure where are you seeing this, when I check the performance results Go is way ahead Scala: go 441,978 20.1% play-scala-anorm 181,316 8.2% techempower.com/benchmarks/… Mar 1, 2016 at 10:22
4

As Brad pointed out, these results are from one particular benchmark suite. This provides some information, but don't assume it's the whole picture. It would be helpful to know whether the source code is well enough written in each case to give the fastest speed, the least memory use, or some other target goal.

Perhaps we might compare with another website that ranks languages. Take a look at http://www.techempower.com/benchmarks/ in which web service codes are compared. In spite of being a young language, Go is one of the best in some of these benchmarks.

As in all benchmarks, it always depends what you strive for and how you measure it.

Not the answer you're looking for? Browse other questions tagged or ask your own question.