Shootout benchmarks

[krono]

These are the shootout benches

• ackermann
• ary
• binarytrees
• chameneos samth/pycket#116 (threads)
• cheapconcurrency samth/pycket#116 (threads)
• echo needs threads (threads)
• except
  • samth/pycket#122 (call-with-current-continuation-prompt)
• fannkuch
• fannkuch-redux
• fasta samth/pycket#112
• fibo
• hash samth/pycket#90
• hash2 samth/pycket#90
• heapsort samth/pycket#120
• k-nucleotide
  • samth/pycket#90
  • samth/pycket#134
• lists
• mandelbrot
• mandelbrot-generic
• matrix
• meteor
• moments
• nbody
• nbody-generic
• nbody-vec
• nbody-vec-generic
• nestedloop
• nsieve
• nsievebits
• partialsums
• pidigits
• pidigits1
• random samth/pycket#115
• recursive
• /regexmatch/
  • samth/pycket#121(regex)
• /regexpdna/
  • samth/pycket#117 samth/pycket#121 (regex)
  • samth/pycket#135
• /reversecomplement/
  • samth/pycket#121
• reversefile
• sieve
• spellcheck samth/pycket#90
• spectralnorm
• spectralnorm-generic
• strcat samth/pycket#119
• sumcol
• thread-ring no output
• wc samth/pycket#118
• wordfreq

[cfbolz]

Should we focus on the current shootout benchmark set? In the above benchmarks a lot of older, currently phased out benchmarks are listed. the current set is:

binary-trees
chameneos-redux
fannkuch-redux
fasta
k-nucleotide
mandelbrot
meteor-contest
n-body
pidigits
regex-dna
reverse-complement
spectral-norm
thread-ring

[samth]

An unfortunately high percentage of these involve regular expressions. If
they just need something simple that's not performance critical, we could
maybe support that, but not the others.

The set I'd most like to focus on is those in my pldi 11 paper, which
Racket has previously been compared on.

[krono]

The set from this paper is (shootout part):

• binarytrees
• fannkuch
• fasta
• k-nucleotide
• mandelbrot
• nbody
• pidigits
• regex- dna
• rev.comp
• spectralnorm

[cfbolz]

Mandelbrot* seem to not really work for me, I get
primitive regexp-match? is not yet implemented

[samth]

I think @krono changed that to error instead of always returning true.

[krono]

Yes, thanks for the catch.
I’ll try to patch the benches where regexes are not actually necessare

[samth]

I think the benchmark "uses" regexps by requiring a library that loads
them. We just need to put that hack back in.
​

[krono]

well, loading does not trigger the not-implemented-error, calling does.
Probably someone in racket/port does this, so, yes for regex-match? the hack may be justified

[samth]

@krono can you update the status here?

[krono]

will do

[krono]

status is up-to-date