Luke J Crook wrote: > How many implementations of Ocaml are there? One. So every developer > working on Ocaml is hammering that one version. If you want to compare > Ocaml to an open source language then choose a language for which there > is but a single implementation; Perl, Python, Parrot, PHP, Ruby etc.
That doesn't really make any sense.
Firstly, the language comparison is with C++, Java and SML as well, all of which have multiple implementations. Indeed, two implementations of SML are already on the language comparison. Secondly, of the languages you listed, at least Perl and Python have multiple implementations. Finally, the number of implementations is irrelevant.
> If you want to compare Ocaml to a specific implementation of Lisp, then > target CMUCL, SBCL, CLISP, Lispworks etc. However, if you want to > compare the performance of Lisp to the performance of Ocaml then choose > the fastest implementation of Lisp available and run that. Allego, > Lispworks and Corman (Corman is Windows only) are all free to you for > this purpose.
We're Linux only, so Corman is out the window. Which of the other Lisp implementations do you expect to do well at this task?
Jon Harrop wrote: > Ulrich Hobelmann wrote: > > I wouldn't consider 5 times as slow as a *functional* language very > > competitive, but it might be fast enough for many problems.
> Well, it's relative. Most of the other Lisp/Scheme implementations were two > orders of magnitude slower. Stalin gets even closer than SBCL.
> Also, MLton often beats g++, so functional languages aren't slow coaches any > more...
Jon: You may not forget that you are testing for micro benchmarks. Surely, a Python will never become faster in the long run. However, all the things will change if we were benchmarking really complicated code of many thousand of lines.
When I was younger (now I am 31) I also believed benchmarking is a must. However, in the meantime it is even this: if your ray-tracer in OCaml were 100000 times faster than the Bigloo-Scheme version I would not opt for OCaml quickly. Perfromance is important, no question, but in a code project it is a rather tiny part.
Förster vom Silberwald <chain_l...@hotmail.com> wrote:
> When I was younger (now I am 31) I also believed benchmarking is a > must. However, in the meantime it is even this: if your ray-tracer in > OCaml were 100000 times faster than the Bigloo-Scheme version I would > not opt for OCaml quickly. Perfromance is important, no question, but > in a code project it is a rather tiny part.
Depends on the code project, doesn't it? If you're doing primarily heavy duty (but medium complexity) mathematical manipulations, then speed is a big factor.
I agree that in most problem domains, constant time speed factors are pretty much irrelevant for 95% of code, and we should be very skeptical about judging "language speed" by simple benchmarks like this, because an easier to program language, often leads to better algorithms, as well as a lot less programming time (and less downside variation in program speed -- cf: Erann's lisp-java-c experiment: the fastest C programs just beat the fastest lisp programs, but many of the C programs were *much* slower than the slowest lisp programs).
But those tradeoffs aren't the same for all problem domains. In those rare problem domains where there's always lots time to optimize relative to the runtime available, bare speed matters.
The problem is that it's still hard to tell the difference between a faster language and better writers of optimized code, as the almabench thread demonstrates. How much time do you spend on optimization before you abandon the project and say "It's easier to make this fast in X". And obviously some things will be easier to optimize in language Y than others. All this makes any given benchmark or set of benchmarks suspect as an absolute measure of speed, but it doesn't completely invalidate their potential usefulness, as long as one understands what they mean and what they do not mean.
Michael Sullivan wrote: > Förster vom Silberwald <chain_l...@hotmail.com> wrote: >> When I was younger (now I am 31) I also believed benchmarking is a >> must. However, in the meantime it is even this: if your ray-tracer in >> OCaml were 100000 times faster than the Bigloo-Scheme version I would >> not opt for OCaml quickly. Perfromance is important, no question, but >> in a code project it is a rather tiny part.
> Depends on the code project, doesn't it? If you're doing primarily > heavy duty (but medium complexity) mathematical manipulations, then > speed is a big factor.
Yes, absolutely. I am primarily interested in the middle ground between slow but hugely expressive languages like Mathematica and "fast" but archaic languages like Fortran. Functional languages are great for this middle ground and they are improving at such a rate that they've been eating into the remits of Mathematica and Fortran a lot.
> I agree that in most problem domains, constant time speed factors are > pretty much irrelevant for 95% of code, and we should be very skeptical > about judging "language speed" by simple benchmarks like this, because > an easier to program language, often leads to better algorithms, as well > as a lot less programming time (and less downside variation in program > speed -- cf: Erann's lisp-java-c experiment: the fastest C programs just > beat the fastest lisp programs, but many of the C programs were *much* > slower than the slowest lisp programs).
This is exactly why I measure LOC. From my results, the LOC measurements indicate that code size will become a limiting factor much more quickly for Java than for OCaml, for example. Although this is a very crude measure (there are a huge number of language features, like a decent type system, that aid productivity), my quantitative result reflect my experience.
However, LOC overly penalises Lisp and Scheme, IMHO. Specifically, Lisp and Scheme programs are virtually unreadable unless the parentheses are staggered by spreading expressions over several lines and using an automatic indenter. So if I were to put a Lisp implementation of the ray tracer on my site then I'd either state that, or I'd give results using some other measure of verbosity, like characters. I do think Lisp deserves to be somewhat penalised in this way, but LOC goes too far.
> But those tradeoffs aren't the same for all problem domains. In those > rare problem domains where there's always lots time to optimize relative > to the runtime available, bare speed matters.
I think it is also important to measure the performance of "natural" code. In Lisp, natural code is often understood very generically by the compiler. In most other languages, you have to work to get that generality.
> The problem is that it's still hard to tell the difference between a > faster language and better writers of optimized code, as the almabench > thread demonstrates. How much time do you spend on optimization before > you abandon the project and say "It's easier to make this fast in X". > And obviously some things will be easier to optimize in language Y than > others. All this makes any given benchmark or set of benchmarks suspect > as an absolute measure of speed, but it doesn't completely invalidate > their potential usefulness, as long as one understands what they mean > and what they do not mean.
Yes, absolutely. I do think my ray tracer is a suboptimal benchmark, of course, but I was astonished that so many people put so much effort into that almabench benchmark when (IMHO) it was an appalingly narrow and badly coded test that isn't representative of any real programs, in my experience as a computational physicist. I'm glad that this hasn't descended into a "Lisp sux0rz" conversation though.
In contrast, my ray tracer involves integer and floating point arithmetic, 3D vectors, trees, tagged unions, recursion/iteration, naturally functional (e.g. vector ops and ray_sphere) and imperative (e.g. for loops) style. So it tests quite a lot using very little code and it is fun to play with. This diversity also makes the collection of ray tracers a better educational tool - people can glance at the code and see how different things are implemented in different languages.
Jon Harrop wrote: > ... > However, LOC overly penalises Lisp and Scheme, IMHO. Specifically, Lisp and > Scheme programs are virtually unreadable unless the parentheses are > staggered by spreading expressions over several lines and using an > automatic indenter. So if I were to put a Lisp implementation of the ray > tracer on my site then I'd either state that, or I'd give results using > some other measure of verbosity, like characters.
i doubt lisp or scheme will gain anything there: the language defined words tend to be quite lengthy, and afaict that seems to encourage programmers to use pretty length identifiers for their own identifiers, so character count might penalize lisp even worse. otoh, the lengthy identifiers make lisp code quite easy to read and understand.
robert.tho...@antenova.com wrote: > Raffael Cavallaro wrote: > > On 2005-08-13 07:18:40 -0400, Jon Harrop <use...@jdh30.plus.com> said:
> > > Here is Nathan Baum's port for CMUCL and SBCL:
> > just as an additional data point, this code runs in just over 6 seconds > > in sbcl 0.9.3 on a dual 2.0 GHz G5 (though sbcl only uses one > > processor).
> I've also written a version of Jon's raytracer benchmark. Unlike the > one given here I've used "simple-vector", to do the vectors.
> I've also being using GCL to compile it so far.
> It will be interesting to see how the performance compares.
Here is my version. For some reason it doesn't work, the "g" variable sticks at 0.5. Have I missed something obvious, can anyone see what's wrong with it?
; Jon Harrops little raytracer ; With bits from Jan Van Lint's version and Nathan Baum's version ; Currently not working
(defun v* (s r) (map 'simple-vector #'(lambda (x) (* s x)) r)) (defun v+ (a b) (map 'simple-vector #'+ a b)) (defun v- (a b) (map 'simple-vector #'- a b)) (defun dot (a b) (apply #'+ (map 'list #'* a b))) (defun unitise (r) (v* (/ (sqrt (dot r r))) r))
(defstruct ray orig dir) (defstruct sphere center rad) (defstruct group sphere scenes)
(defun ray-sphere (ray sph) (let* (t1 t2 (v (v- (sphere-center sph) (ray-orig ray))) (b (dot v (ray-dir ray))) (disc (+ (- (* b b) (dot v v)) (expt (sphere-rad sph) 2)))) (if (minusp disc) infinity (progn (setq disc (sqrt disc)) (if (minusp (setq t2 (+ b disc))) infinity (progn (setq t1 (- b disc)) (if (plusp t1) t1 t2)))))))
(defun intersect (ray scene) (labels ((lp (hit scene) ; car hit is a lam, cdr a simple-vector normal (if (sphere-p scene) (let ((lam (ray-sphere ray scene))) (if (>= lam (car hit)) hit (cons lam (unitise (v- (v+ (ray-orig ray) (v* lam (ray-dir ray))) (sphere-center scene)))))) (if (group-p scene) (if (>= (ray-sphere ray (group-sphere scene)) (car hit)) hit (reduce #'lp (group-scenes scene) :initial-value hit)) (error "not a group or sphere in intersect"))))) (lp (cons infinity #(0.0 0.0 0.0)) scene)))
JH>> However, LOC overly penalises Lisp and Scheme, IMHO. Specifically, Lisp and JH>> Scheme programs are virtually unreadable unless the parentheses are JH>> staggered by spreading expressions over several lines and using an JH>> automatic indenter. So if I were to put a Lisp implementation of the ray JH>> tracer on my site then I'd either state that, or I'd give results using JH>> some other measure of verbosity, like characters.
Hmm. What would you (JH) be measuring here?
a) Keystrokes required to produce the code (see below, though) b) Some kind of 'intrinsic verbosity', which would require some *serious* thinking about idiomaticity, relevance of formatting and massive, massive sampling to make it statistically relevant.
> i doubt lisp or scheme will gain anything there: the language defined > words tend to be quite lengthy, and afaict that seems to encourage > programmers to use pretty length identifiers for their own identifiers,
Yes
> so character count might penalize lisp even worse. otoh, the lengthy > identifiers make lisp code quite easy to read and understand.
Yes, and using a decent editor with auto-completion (Emacs) means that I hit less keys to produce the token 'DESTRUCTURING-BIND' ( DE-B <META-TAB> ) than you might think.
Oh, and all the ')))))' you see probably didn't get typed by hand ( <META-RET> closes all open parens).
> token count probably would be better
Yep, although (because I am biased) I would like to see 'keystroke/mouse-click' count instead. I think that with the requirement for idiomatic variable naming, CL might not come out as 'verbose' as you think...
"Rob Thorpe" <robert.tho...@antenova.com> writes: > Here is my version. For some reason it doesn't work, the "g" variable > sticks at 0.5. Have I missed something obvious, can anyone see what's > wrong with it?
Try setting delta to something *much* larger than long-float-epsilon.
> "Hartmann Schaffer" <h...@hartmann.schaffernet> wrote in message > news:WqfMe.1588$Dd.6727@newscontent-01.sprint.ca... >> Jon Harrop wrote: >>> ... > JH>> However, LOC overly penalises Lisp and Scheme, IMHO. Specifically, > Lisp and > JH>> Scheme programs are virtually unreadable unless the parentheses are > JH>> staggered by spreading expressions over several lines and using an > JH>> automatic indenter. So if I were to put a Lisp implementation of the > ray > JH>> tracer on my site then I'd either state that, or I'd give results > using > JH>> some other measure of verbosity, like characters.
> Hmm. What would you (JH) be measuring here?
> a) Keystrokes required to produce the code (see below, though) > b) Some kind of 'intrinsic verbosity', which would require some *serious* > thinking about idiomaticity, relevance of formatting and massive, massive > sampling to make it statistically relevant.
>> i doubt lisp or scheme will gain anything there: the language defined >> words tend to be quite lengthy, and afaict that seems to encourage >> programmers to use pretty length identifiers for their own identifiers,
> Yes
>> so character count might penalize lisp even worse. otoh, the lengthy >> identifiers make lisp code quite easy to read and understand.
> Yes, and using a decent editor with auto-completion (Emacs) means that I > hit less keys to produce the token 'DESTRUCTURING-BIND' ( DE-B > <META-TAB> ) than you might think.
> Oh, and all the ')))))' you see probably didn't get typed by hand ( > <META-RET> closes all open parens).
^^^^^^^^^^^^ at least at the slime REPL anyway <oops>
> Yep, although (because I am biased) I would like to see > 'keystroke/mouse-click' count instead. I think that with the requirement > for idiomatic variable naming, CL might not come out as 'verbose' as you > think...