Chris Smith wrote: > Jon Harrop <use...@jdh30.plus.com> wrote: >> Can you elaborate on this? All of my tests indicate that Java is many >> times slower than most other modern languages, even stereotypically slow >> languages like SML and OCaml.
> You're making statements that are far too broad to be useful. Java is > far slower doing what?
Running my ray tracer.
> Most computational tasks tend to run within about 10% to 20% of optimal > time for the hardware platform, and compare about evenly with modern > compiled languages.
If you're trying to say that equivalent programs written in different languages like C, C++, Java, SML, OCaml and Fortran will be within 20% of each other's performance then that definitely isn't true. Any programs which stress allocation/deallocation (e.g. balanced binary trees) are likely to do significantly worse in garbage collected languages. Any programs which stress indirected data structures will do significantly worse in OCaml, particularly when pointers cost 64 bits.
> There are specific tasks for which significant > general performance differences may be measured (floating point > calculations being one, but I can't recall whether Java is typically > faster or slower here, and it may depend on the platform), but this > wouldn't be described as "many times slower".
Here are the times for running my ray tracer on x86 (1.2GHz Athlon T-bird) with n=128, level=6 and ss=4:
Mlton 1.250s mlton ray.sml IFC 1.361s ifort -O3 -u -static-libcxa -o raytracer raytracer.f90 C++ 1.555s g++-3.4 -O3 -march=athlon-tbird -ffast-math ray.cpp -o ray ocamlopt 1.932s ocamlopt -inline 100 ray.ml -o ray SML-NJ 2.085s sml ray.sml G95 3.351s g95 -O3 -ffast-math ray.f90 -o ray C 4.125s gcc-3.4 -lm -std=c99 -O3 -march=athlon-tbird -funroll-all-loops -ffast-math ray.c -o ray Java 6.492s javac ray.java GCJ 20.316s gcj-3.4 --main=ray -Wall -O3 -march=athlon-tbird -funroll-all-loops -ffast-math ray.java -o ray ocamlc 41.047s ocamlc ray.ml -o ray
as you can see, Java run under Sun's J2SE is over 5x slower than the fastest implementation (Mlton-compiled SML).
> I strongly suspect that the issue is startup time.
I have taken many measurements for different running times (<2s to >20s) and, assuming Java's startup time to be constant, the startup time is insignificant.
> If you're measuring > it with your ray tracer, are you measuring externally so as to include > the startup time, or internally once the VM has started? There's no > "right" way to do this; it depends on whether your target audience will > care about startup time, or only how the appo performs once it's > running. However, if you're including startup time, you ought to at > least add a note that this is the reason for the results. The great > majority of applications run for orders of magnitude longer than the > average benchmark, and just saying that Java is slower is very > misleading.
Right, well I've accounted for that and I can still only conclude that Java is very slow.
I've also run the ray tracer on AMD64 with JDK 1.5 and Java is still significantly slower than other languages, albeit "only" 2.4 times slower.
Jon Harrop <use...@jdh30.plus.com> writes: > Can you elaborate on this? All of my tests indicate that Java is many times > slower than most other modern languages, even stereotypically slow > languages like SML and OCaml.
Since Java code (unless forced to run interpreted) runs 98% of its time as run-time optimized native code, how can it be slower than compile-time optimized C code? Are you perchance taking startup time into consideration?
Tor Iver Wilhelmsen wrote: > Jon Harrop <use...@jdh30.plus.com> writes: >> Can you elaborate on this? All of my tests indicate that Java is many >> times slower than most other modern languages, even stereotypically slow >> languages like SML and OCaml.
> Since Java code (unless forced to run interpreted) runs 98% of its > time as run-time optimized native code, how can it be slower than > compile-time optimized C code?
Many possible reasons:
1. Java's optimiser isn't as good. 2. The time spent optimising is longer than the time saved executing. 3. Deficiencies in the language (of which there are many in Java, e.g. run-time checks due to OO, boxing). 4. Poor implementations of performance critical constructs by the compiler, e.g. recursion.
> Are you perchance taking startup time into consideration?
I have taken startup time into consideration. Java is still consistently many times slower than most other languages.
Jon Harrop <use...@jdh30.plus.com> writes: > I have taken startup time into consideration. Java is still consistently > many times slower than most other languages.
Is this using the original code that was posted here? Because it can be written significantly better.
Just making the Vector class static will reduce its number of fields by 25%, which should make a significant cut in the initialization time for each Vector.
Lasse Reichstein Nielsen wrote: > Jon Harrop <use...@jdh30.plus.com> writes: >> I have taken startup time into consideration. Java is still consistently >> many times slower than most other languages.
> Is this using the original code that was posted here? Because it can > be written significantly better.
Great, tell me how (provided it doesn't get too much longer).
> Just making the Vector class static will reduce its number of fields > by 25%, which should make a significant cut in the initialization time > for each Vector.
One my machine, the original took 31s for:
$ javac ray.java $ java ray 256>image.pgm
With the vector class declared static (what exactly does this do?) it takes 29s, which is not significantly faster.
In contrast, Mlton-compiled SML is much shorter (79 vs 110 LOC) and runs about 5x as fast, taking only 6s to ray trace exactly the same image.
So we really need much better optimisations if Java is going to perform anything like as well as the other languages. Am I missing something simple, like JIT or optimisation flags?
Jon Harrop <use...@jdh30.plus.com> writes: > With the vector class declared static (what exactly does this do?)
This alone tells us you really do not know Java.
The point is that you really should not take startup time into consideration when considering how "fast" a language is. It would be akin to testing whether the Windows "calc" program is faster than using pencil + paper to add two numbers, and then take Windows startup time into account. Java is designed to run apps over a period of time, you don't and draw the wrong conclusion.
Tor Iver Wilhelmsen wrote: > Jon Harrop <use...@jdh30.plus.com> writes: >> With the vector class declared static (what exactly does this do?)
> This alone tells us you really do not know Java.
Yes, I do not know Java, as I stated in my original post. That's why I'm here, asking for advice.
I'm not actually getting any useful advice though. So far, lots of people have said that Java has a startup time, suggested minor performance tweaks which fail to make the program 5x faster and told me that I don't know Java.
> The point is that you really should not take startup time into > consideration when considering how "fast" a language is.
For the umpteenth time, I have taken startup time into account. I am doing measurements over an order of magnitude variation in total running time and startup appears to be insignificant (~0.2s out of 30s total running time). Java is still vastly slower than everything else. Is this representative of Java or am I doing something daft?
> It would be > akin to testing whether the Windows "calc" program is faster than > using pencil + paper to add two numbers, and then take Windows startup > time into account. Java is designed to run apps over a period of time, > you don't and draw the wrong conclusion.
Yes, I am trying very hard to draw the appropriate conclusion. Currently, the conclusion is that Java is the second most verbose, slowest to compile and the slowest to run out of the languages that I have tested (C, C++, OCaml, SML, Fortran).
Jon Harrop <use...@jdh30.plus.com> writes: > Lasse Reichstein Nielsen wrote: >> Jon Harrop <use...@jdh30.plus.com> writes: >>> I have taken startup time into consideration. Java is still consistently >>> many times slower than most other languages.
>> Is this using the original code that was posted here? Because it can >> be written significantly better.
> Great, tell me how (provided it doesn't get too much longer).
I posted a link to a slightly optimized version. I don't know if I am changing the algorithm too much for comparison (e.g., moving loop invariant code out of the loop) and doing all the I/O at the end instead of during the computation.
I admit to having performed "optimizations" without profiling first, so there is no telling which change does what, but it does give me an ~30% increase in speed for size=100.
> With the vector class declared static (what exactly does this do?)
It makes the class more independent of the surrounding class, so each Vector doesn't have to carry around a reference to a "ray" object that it never uses.
In a real program, I would put all the nested classes into their own files.
> it takes 29s, which is not significantly faster.
My guess is that a lot of the time comes from I/O (ok, just O, it only does output :).
> In contrast, Mlton-compiled SML is much shorter (79 vs 110 LOC) and runs > about 5x as fast, taking only 6s to ray trace exactly the same image.
Brevity of code is not in itself a quality. Java is often more verbose than other languages, but I also often find it more readable, because differet programmers can't invent their own dialect.
> So we really need much better optimisations if Java is going to perform > anything like as well as the other languages. Am I missing something > simple, like JIT or optimisation flags?
Can't help you there. I rarely fiddle with flags :)
Btw, is the generated image correct? It looks odd. /L -- Lasse Reichstein Nielsen - l...@hotpop.com DHTML Death Colors: <URL:http://www.infimum.dk/HTML/rasterTriangleDOM.html> 'Faith without judgement merely degrades the spirit divine.'
Tor Iver Wilhelmsen wrote: > Jon Harrop <use...@jdh30.plus.com> writes:
>>Can you elaborate on this? All of my tests indicate that Java is many times >>slower than most other modern languages, even stereotypically slow >>languages like SML and OCaml.
> Since Java code (unless forced to run interpreted) runs 98% of its > time as run-time optimized native code, how can it be slower than > compile-time optimized C code? Are you perchance taking startup time > into consideration?
You know I've been indoctrinated in the Java is as fast as C as well, but someone was pointing out benchmarks that show it is twice as slow as compiling the code with gcj. I've not had time to look into the actual results (and I haven't found any comparing to 1.5), but it makes me wonder if maybe we've believed something that isn't true.
Dale King wrote: > You know I've been indoctrinated in the Java is as fast as C as well, > but someone was pointing out benchmarks that show it is twice as slow as > compiling the code with gcj. I've not had time to look into the actual > results (and I haven't found any comparing to 1.5), but it makes me > wonder if maybe we've believed something that isn't true.
Depends on what you're doing -- obviously. There have been various reports of differences in performance with various JVMs vs. various "other" languages. The results are varied as well.
I do suspect, though, that applications that show Java to be in the 50%-80% as fast as C/C++/Fortran are probably under-reported. Java turning out to be as fast or faster than the old-school languages is news; Java turning out to be significantly slower is also news; but Java turning out to be slower by an unimportant margin... <yawn/>
In this particular case, though, I suspect that the results are not representative -- I did measure the OP's earlier version of the Java code vs. the C++ code he had on his web page (I had to fix some minor differences), and found that on my machine the 1.5 server JVM from Sun was getting on for twice as fast as the C++ code compiled with MS VC6 or VS2003 [*]. He's since offered modified C++ code, but I haven't attempted to see what the differences are, nor whether the Java code could still be considered to be a fair equivalent.
-- chris
([*] to be honest, that surprised me -- I'd have expected the overhead of heap-allocating hundreds of millions of objects to have more impact than that)
Percentage shows the ratio of compiled and optimized native code speed vs. JVM speed. What is interesting: a better written source code is better optimized by a compiler -- see that increasing the job size takes less and less time in the optimized program...
Ah, yep, this was about another way to accelerate your program ;)
> Tor Iver Wilhelmsen wrote: > > Jon Harrop <use...@jdh30.plus.com> writes:
> >>Can you elaborate on this? All of my tests indicate that Java is many times > >>slower than most other modern languages, even stereotypically slow > >>languages like SML and OCaml.
> > Since Java code (unless forced to run interpreted) runs 98% of its > > time as run-time optimized native code, how can it be slower than > > compile-time optimized C code? Are you perchance taking startup time > > into consideration?
> You know I've been indoctrinated in the Java is as fast as C as well, > but someone was pointing out benchmarks that show it is twice as slow as > compiling the code with gcj. I've not had time to look into the actual > results (and I haven't found any comparing to 1.5), but it makes me > wonder if maybe we've believed something that isn't true.
> -- > Dale King
Claiming java code runs as fast as native C code is naive at best, it simply isnt. There are numerous sites with benchmarks on the net that prove my point. There are certain scenarios where Java will be as fast (or faster) than native C code, but such scenarios are fairly rare in applications such as raytracers. I have personally written a real-time raytracer in Java, and have found performance to be dramatically lower than it's C counterpart, while the code was almost the same.
Note that you can improve performance by using the server VM (-server), which almost doubled the performance of my raytracer. The server VM does it's best to get peak-performance (most likely by aggressively JIT-ing/inlining a lot of code), the client VM is more concerned with fast start-up and reducing memory footprints.
> To the issues of JAVA performance: > I have compiled both original version (Ray) and this optimized version > (RayTracing): > > <URL:http://www.infimum.dk/privat/RayTracing.java>
> Percentage shows the ratio of compiled and optimized native code speed > vs. JVM speed. What is interesting: a better written source code is > better optimized by a compiler -- see that increasing the job size > takes less and less time in the optimized program...
> Ah, yep, this was about another way to accelerate your program ;)
Remon van Vliet wrote: > Did you run the JVM code with the -server option on? If not, try that and > see if it makes a difference, it should.
I did and it makes no difference because it is the default. However, running with -client actually appears to be slightly faster (17.8s vs 18.3s). That might not be a significant difference though.
Dale King wrote: > Tor Iver Wilhelmsen wrote: > You know I've been indoctrinated in the Java is as fast as C as well, > but someone was pointing out benchmarks that show it is twice as slow as > compiling the code with gcj. I've not had time to look into the actual > results (and I haven't found any comparing to 1.5), but it makes me > wonder if maybe we've believed something that isn't true.
In my experience Java is slower than C. Java has many qualities that make it nicer to program in than C, but raw speed isn't one of them.
> Remon van Vliet wrote: > > Did you run the JVM code with the -server option on? If not, try that and > > see if it makes a difference, it should.
> I did and it makes no difference because it is the default. However, running > with -client actually appears to be slightly faster (17.8s vs 18.3s). That > might not be a significant difference though.
Remon van Vliet wrote: > > I did and it makes no difference because it is the default. However, > > running with -client actually appears to be slightly faster (17.8s vs > > 18.3s). That might not be a significant difference though. > That's a really odd result actually, i've never seen tight loops in java > code run faster on the -client than -server. Any explanation for this?
I'd guess that a small-ish part of it is that the server JVM has a longer startup time (especially with the fast-startup stuff in Java 1.5 which I assume (without checking) that only the client JVM bothers with). I can imagine that making a couple of seconds difference.
A possible explanation for the rest of the difference would be that the specific benchmark is structured so that the server JVM cannot optimise it without doing on-stack replacement of running methods. I don't know what the real story on that is, but my own experiments indicate that it is unlikely to optimise a benchmark that consists of a single set of nested loops.
The easy, and obvious, way to find out is to re-structure the benchmark so that it:
(a) times the relevant operation /itself/, rather than timing the execution of the whole program.
(b) runs the benchmark in a loop. Probably (since the loop is long) timing each iteration independently.
Without those changes, I don't see that the benchmark can be saying anything very interesting. Of course, /with/ the changes it might still turn out that the server JVM was lagging badly behind C++ on this application. I wouldn't be surprised myself, since the way it is written involves creating infeasible large numbers of intermediate objects -- which would obviously put Java at a disadvantage. (Not that I'm suggesting that it's an invalid benchmark for that reason.)
Chris Uppal wrote: > Remon van Vliet wrote: >> > I did and it makes no difference because it is the default. However, >> > running with -client actually appears to be slightly faster (17.8s vs >> > 18.3s). That might not be a significant difference though.
>> That's a really odd result actually, i've never seen tight loops in java >> code run faster on the -client than -server. Any explanation for this?
Most of the time is spent in mutually recursive functions, rather than "tight loops".
> I'd guess that a small-ish part of it is that the server JVM has a longer > startup time (especially with the fast-startup stuff in Java 1.5 which I > assume > (without checking) that only the client JVM bothers with). I can imagine > that making a couple of seconds difference.
The difference (of 0.5s) is well within random noise.
> A possible explanation for the rest of the difference would be that the > specific benchmark is structured so that the server JVM cannot optimise it > without doing on-stack replacement of running methods. I don't know what > the real story on that is, but my own experiments indicate that it is > unlikely to optimise a benchmark that consists of a single set of nested > loops.
Then what can it optimise?
> The easy, and obvious, way to find out is to re-structure the benchmark so > that it:
> (a) times the relevant operation /itself/, rather than timing the > execution of the whole program.
If the "start up" time is proportional to the running time of the program (e.g. because it is the time spent compiling to native code on-the-fly) then it should be counted as it is. There is no evidence of a large constant start up time.
> (b) runs the benchmark in a loop. Probably (since the loop is long) > timing > each iteration independently.
It would be neither fair nor representative to rerun the Java test many times.
> Without those changes, I don't see that the benchmark can be saying > anything > very interesting. Of course, /with/ the changes it might still turn out > that > the server JVM was lagging badly behind C++ on this application. I > wouldn't be surprised myself, since the way it is written involves > creating infeasible large numbers of intermediate objects -- which would > obviously put Java at a > disadvantage. (Not that I'm suggesting that it's an invalid benchmark for > that reason.)
Indeed, there is no easy way around this and many (most?) numerically intensive programs use small vectors and matrices.
So the conclusion can only be that Java is not suitable for such applications.
> Chris Uppal wrote: > > Remon van Vliet wrote: > >> > I did and it makes no difference because it is the default. However, > >> > running with -client actually appears to be slightly faster (17.8s vs > >> > 18.3s). That might not be a significant difference though.
> >> That's a really odd result actually, i've never seen tight loops in java > >> code run faster on the -client than -server. Any explanation for this?
> Most of the time is spent in mutually recursive functions, rather than > "tight loops".
> > I'd guess that a small-ish part of it is that the server JVM has a longer > > startup time (especially with the fast-startup stuff in Java 1.5 which I > > assume > > (without checking) that only the client JVM bothers with). I can imagine > > that making a couple of seconds difference.
> The difference (of 0.5s) is well within random noise.
Startup time is added to the benchmark, and as such isnt measuring Java execution performance.
> > A possible explanation for the rest of the difference would be that the > > specific benchmark is structured so that the server JVM cannot optimise it > > without doing on-stack replacement of running methods. I don't know what > > the real story on that is, but my own experiments indicate that it is > > unlikely to optimise a benchmark that consists of a single set of nested > > loops.
> Then what can it optimise?
Well it should be obvious that nested loops cannot be optimised further by a VM than run-time compiling (JITing) them and possible unroll smaller loops. The server VM may and most likely will do this more aggressively.
> > The easy, and obvious, way to find out is to re-structure the benchmark so > > that it:
> > (a) times the relevant operation /itself/, rather than timing the > > execution of the whole program.
> If the "start up" time is proportional to the running time of the program > (e.g. because it is the time spent compiling to native code on-the-fly) > then it should be counted as it is. There is no evidence of a large > constant start up time.
There's no evidence for either point. Fact remains is that the VM does things on startup that a native compiler does at compilation time. If you're measuring raw execution performance neither are relevant.
> > (b) runs the benchmark in a loop. Probably (since the loop is long) > > timing > > each iteration independently.
> It would be neither fair nor representative to rerun the Java test many > times.
Untrue, if you're comparing execution speed then you should measure just that. Startup time is not related to the speed at which the actual computational code gets executed.
> > Without those changes, I don't see that the benchmark can be saying > > anything > > very interesting. Of course, /with/ the changes it might still turn out > > that > > the server JVM was lagging badly behind C++ on this application. I > > wouldn't be surprised myself, since the way it is written involves > > creating infeasible large numbers of intermediate objects -- which would > > obviously put Java at a > > disadvantage. (Not that I'm suggesting that it's an invalid benchmark for > > that reason.)
> Indeed, there is no easy way around this and many (most?) numerically > intensive programs use small vectors and matrices.
> So the conclusion can only be that Java is not suitable for such > applications.
That conclusion is severely flawed in my opinion. Computationally intensive programs do not by definition create a large number of intermediate (or any other kind of) objects. The fact that applications such as raytracing are computationally expensive are completely unrelated to object creation. In my experience matrices and vector objects can be reused and pooled to a point where object generation is minimal. Also, if there's one area where (JITted) Java code and C(++) code runs at comparable speeds it's arithmetic/logic/computation.
Finally, i did a test myself with the mentioned code (300) :
> Indeed, there is no easy way around this and many (most?) numerically > intensive programs use small vectors and matrices.
> So the conclusion can only be that Java is not suitable for such > applications.
Omitting the rest of the discussion, I would like to reply to that last statement only.
First. The languages are roughly divided into "compilative" and "interpretative". For the first ones more checks and optimizing work can be done at compile-time, while for the later they all fall to run-time. Interpretation loops of VM, start ups etc. take their time for work non-productive from the end-result point of view. Java, from one side, is implemented as a compilative language, i.e. with a JVM interpreter. You pay this price for being platform-independent. From another side, Java, in most of its features is a well-compilable language, alike C and C++. The solution that I have proposed giving my statistics, was to compile your application for a known target platform (if it is x86/Windows, it can be JET (http://www.excelsior-usa.com/jet.html), or it can be GCJ... The reason was to get rid of the interpreting engine - JVM and to benefit of the powerful optimizing techniques implemented in modern compilers.
Second. A theorem: For any language, any platform and any hardware there exists a problem that cannot be efficiently solved with them. Proof: consider any polynomic or NP-complete problem with high factors. Usually, is is not a tool to be blamed. Special methods to solve these "hard tasks" should be elaborated, standard algorythms should be optimized into special versions, etc. Consider the case that is now fundamental to programming with Windows: I mean "invalidation rectangles". I bet that initially the problem was that redrawing the full heap of windows on a screen when something changes in one of them was not acceptible (redrawing time being much more the eye wink time). Suppose that with modern speeds it is possible to recalculate the full screen picture at a time less than a screen update frequency (60Hz). Who would ever think on that problem? Hence the statement: a program that will need to be optimized should be optimized not only at the latest phase, when a VM or a compiler try to do their best, and not even (though in more extent) at the phase of coding the algorythm on a particular language, but yet on the phase of algorythm selection. As for the language, the skills of writing fast-working programs may be somewhat different from the skills of writing programs "as books and professors teach". It may be more preferrable to unroll loops, to precalculate often used data into huge static arrays, write in a mix of languages (well, after all, if 90% of time your program spents in a 5-lines loop, isn't it a reason to write this loop in assembler than to reject the language which is perfect for rest N-5 lines??), etc. etc...
So, the art of optimizing programs is an art indeed that should be learnt as we learn to program :)
Remon van Vliet wrote: > Startup time is added to the benchmark, and as such isnt measuring Java > execution performance.
The benchmark clearly is measuring Java execution performance.
> Well it should be obvious that nested loops cannot be optimised further by > a VM than run-time compiling (JITing) them and possible unroll smaller > loops. The server VM may and most likely will do this more aggressively.
Then I would expect to see a performance improvement, which I don't.
>> It would be neither fair nor representative to rerun the Java test many >> times.
> Untrue, if you're comparing execution speed then you should measure just > that. Startup time is not related to the speed at which the actual > computational code gets executed.
It is, of course. And you'd be able to see the startup time on my graphs were it not for the fact that it is negligible.
>> So the conclusion can only be that Java is not suitable for such >> applications.
> That conclusion is severely flawed in my opinion.
I have presented objective, quantitative evidence supporting it.
> Computationally > intensive programs do not by definition create a large number of > intermediate (or any other kind of) objects.
Like 3D vectors?
> The fact that applications > such as raytracing are computationally expensive are completely unrelated > to object creation. In my experience matrices and vector objects can be > reused and pooled to a point where object generation is minimal.
Then you're trying to work around a poor allocator and garbage collected. You don't have to do this in OCaml or SML. That is what I am trying to measure.
> Also, if > there's one area where (JITted) Java code and C(++) code runs at > comparable speeds it's arithmetic/logic/computation.
I'll try those next. What benchmark would show Java in the best possible light?
> Finally, i did a test myself with the mentioned code (300) :
> Remon van Vliet wrote: > > Startup time is added to the benchmark, and as such isnt measuring Java > > execution performance.
> The benchmark clearly is measuring Java execution performance.
you're measuring startup + execution
> > Well it should be obvious that nested loops cannot be optimised further by > > a VM than run-time compiling (JITing) them and possible unroll smaller > > loops. The server VM may and most likely will do this more aggressively.
> Then I would expect to see a performance improvement, which I don't.
Well i do, (see below)
> >> It would be neither fair nor representative to rerun the Java test many > >> times.
> > Untrue, if you're comparing execution speed then you should measure just > > that. Startup time is not related to the speed at which the actual > > computational code gets executed.
> It is, of course. And you'd be able to see the startup time on my graphs > were it not for the fact that it is negligible.
We'll agree to disagree
> >> So the conclusion can only be that Java is not suitable for such > >> applications.
> > That conclusion is severely flawed in my opinion.
> I have presented objective, quantitative evidence supporting it.
So far not even the results are reproducable on my system. And i maintain that if object creation is a problem for a language then just make sure you dont do it that often. You can call this a workaround but fact remains that creating thousands of objects on the fly continuously is bad practice in any language.
> > Computationally > > intensive programs do not by definition create a large number of > > intermediate (or any other kind of) objects.
> Like 3D vectors?
An enormous amount of 3D vectors are used in my real-time raytracer, but almost always are instances of such vectors reused in some way or form without breaking up code clarity. It's a simple matter of knowing what platform you're coding for.
> > The fact that applications > > such as raytracing are computationally expensive are completely unrelated > > to object creation. In my experience matrices and vector objects can be > > reused and pooled to a point where object generation is minimal.
> Then you're trying to work around a poor allocator and garbage collected. > You don't have to do this in OCaml or SML. That is what I am trying to > measure.
I thought you were measuring execution speed? If you use an approach suitable for OCaml that is highly inefficient to do in Java then obviously the benchmark results will reflect that. Note that i'm not argueing which language is better or anything, i'm sure the above languages have their benefits.
> > Also, if > > there's one area where (JITted) Java code and C(++) code runs at > > comparable speeds it's arithmetic/logic/computation.
> I'll try those next. What benchmark would show Java in the best possible > light?
A raytracer is fine actually, but alternatives can be any form of solving a computationally intensive problem is fine.
> > Finally, i did a test myself with the mentioned code (300) :
Remon van Vliet wrote: > "Jon Harrop" <use...@jdh30.plus.com> wrote in message > news:42b19429$0$41905$ed2619ec@ptn-nntp-reader03.plus.net... >> Remon van Vliet wrote: >> > Startup time is added to the benchmark, and as such isnt measuring Java >> > execution performance.
>> The benchmark clearly is measuring Java execution performance. > you're measuring startup + execution
For different execution times, yes.
>> Then I would expect to see a performance improvement, which I don't. > Well i do, (see below)
I had been using my own source. I just tried it with RayTracer.java and I still see little difference between -client and -server:
$ javac RayTracing.java $ time java RayTracing 256 >image.pgm
real 0m21.098s user 0m20.340s sys 0m0.230s $ time java -client RayTracing 256 >image.pgm
real 0m21.034s user 0m20.320s sys 0m0.180s $ time java -server RayTracing 256 >image.pgm
real 0m19.634s user 0m16.640s sys 0m0.280s
Compared to the most naive OCaml implementation on the same machine:
$ time ./ray 6 256 >image.pgm
real 0m6.374s user 0m5.860s sys 0m0.020s
or the optimised OCaml:
$ time ./ray 6256 >image.pgm
real 0m3.525s user 0m3.500s sys 0m0.010s
>> >> It would be neither fair nor representative to rerun the Java test >> >> many times.
>> > Untrue, if you're comparing execution speed then you should measure >> > just that. Startup time is not related to the speed at which the actual >> > computational code gets executed.
>> It is, of course. And you'd be able to see the startup time on my graphs >> were it not for the fact that it is negligible. > We'll agree to disagree
Can you see a significant startup time on my graphs?
>> I have presented objective, quantitative evidence supporting it. > So far not even the results are reproducable on my system. And i maintain > that if object creation is a problem for a language then just make sure > you dont do it that often. You can call this a workaround but fact remains > that creating thousands of objects on the fly continuously is bad practice > in any language.
Absolute nonsense. The C++ is doing exactly the same thing and is many times faster. I bet an OCaml version which abused objects in the same way would be similarly many times faster than Java.
>> > Computationally >> > intensive programs do not by definition create a large number of >> > intermediate (or any other kind of) objects.
>> Like 3D vectors? > An enormous amount of 3D vectors are used in my real-time raytracer, but > almost always are instances of such vectors reused in some way or form > without breaking up code clarity. It's a simple matter of knowing what > platform you're coding for.
You're just working around a poor allocator/GC implementation. OCaml and SML both allocate and GC exactly the same stuff as Java but run many times faster without the programmer having to understand the details of the compiler and platform.
>> Then you're trying to work around a poor allocator and garbage collected. >> You don't have to do this in OCaml or SML. That is what I am trying to >> measure. > I thought you were measuring execution speed? If you use an approach > suitable for OCaml that is highly inefficient to do in Java then obviously > the benchmark results will reflect that. Note that i'm not argueing which > language is better or anything, i'm sure the above languages have their > benefits.
So how would you implement 3D vectors efficiently in Java? In-line everything as you have to in C?
>> I'll try those next. What benchmark would show Java in the best possible >> light? > A raytracer is fine actually, but alternatives can be any form of solving > a computationally intensive problem is fine.
So I am showing Java in the best possible light?
>> > Finally, i did a test myself with the mentioned code (300) :
>> What's your setup, what values of n and level did you use? > I just ran RayTracing.java with args[0] = 300, it ran on a 1.5ghz Pentium > 3 DELL notebook with 1gb memory within Eclipse IDE.
Perhaps it is Intel specific. I don't have an Intel to try it on...
Jon Harrop <use...@jdh30.plus.com> wrote: > It is, of course. And you'd be able to see the startup time on my graphs > were it not for the fact that it is negligible.
Jon,
I don't know what you're seeing, but I can tell you that you are definitely reaching the wrong conclusion. It can be easily demonstrated that the Java virtual machine does have a considerable startup time before reaching application code, on the order of 0.5 to 1.5 seconds. The exact time depends mainly on the amount of the core API used by an application, and the amount of actual application code. Your benchmarks are extremely low on both accounts, so you will probably see a startup time of about half a second or thereabouts. It's also the case that the JVM requires "warm up" time before it reaches full speed, and that the first time a piece of code gets run will not be the fastest.
How you react to this depends on what you are trying to measure. If you're testing the ability of Java to execute one-shot batch or command line applications that run and terminate in a relatively short period of time, then you should benchmark startup and cold runs just as you're doing. The _problem_ is when people attempt to extract from these results to more common problem spaces, such as long-running and user- interactive applications, where those particular performance factors are irrelevant or near to it.
Chris Smith wrote: > How you react to this depends on what you are trying to measure. If > you're testing the ability of Java to execute one-shot batch or command > line applications that run and terminate in a relatively short period of > time, then you should benchmark startup and cold runs just as you're > doing. The _problem_ is when people attempt to extract from these > results to more common problem spaces, such as long-running and user- > interactive applications, where those particular performance factors are > irrelevant or near to it.
Yes. I'm seeing 0.2s startup for Java. The OCaml and MLton run-times have no startup and seem to be well warmed up after 30s.
I should plot a graph of the running time for the languages vs scene complexity. That should give a better indication of whether or not Java's performance is heading in the right direction for bigger problems.
|
