On Sat, Nov 22, 2003 at 03:12:48AM +1100, skaller wrote: > For example .c -cc--> .o -link--> exe, we can > use the cached .o instead of running cc on the > .c provided the .c is older.
Older is incorrect, as well. The program should run cc on the .c provided that the .c is different than the one used to produce the .o. Timestamps are useful as a cache, but a proper tool will need to use file hashes, or something to properly detect a file changing.
An unrelated annoyance I've found is that ocamlc doesn't allow for the objects to be stored in a separate directory from this source. This makes things, such as multiple configurations, multiple targets difficult. Since ocamlopt doesn't currently support cross compilation, anyway, this is less of an issue.
On Sat, Nov 22, 2003 at 03:12:48AM +1100, skaller wrote: > First observe that building a system > is a process where it is not always > possible to determine the full sequence > of commands in advance. Sometimes, > you have to perform some action before > you can decide what to do next.
Make doesn't try to determine everything in advance. Well, sorta :-) It does try to figure everything out in advance, and then sometimes looks to see if that needs to change. It makes it work in a few situations where it wouldn't before.
A while back, I build something like ocamake that let you specify a command like:
myomake -o target a b c
This would compile modules a, b, and c, and any that they depended on. They would be linked in an order that would link. If you need more control, you can specify additional modules on the command line. This kind of tool is useful for many projects.
Once I discovered not to try and figure everything out in advance, the tool not only became easier to write, but possible.
Once other stuff is brought in (C, literate programming), the specification needs to be more complex.
On Sun, 2003-11-23 at 04:08, David Brown wrote: > On Sat, Nov 22, 2003 at 03:12:48AM +1100, skaller wrote:
> > For example .c -cc--> .o -link--> exe, we can > > use the cached .o instead of running cc on the > > .c provided the .c is older.
> Older is incorrect, as well. The program should run cc on the .c > provided that the .c is different than the one used to produce the .o. > Timestamps are useful as a cache, but a proper tool will need to use > file hashes, or something to properly detect a file changing.
Yeah, that's a good point.
> An unrelated annoyance I've found is that ocamlc doesn't allow for the > objects to be stored in a separate directory from this source. This > makes things, such as multiple configurations, multiple targets > difficult. Since ocamlopt doesn't currently support cross compilation, > anyway, this is less of an issue.
It is still an issue if you have multiple Ocaml installations: I actually ought to have 3 of them:
(a) the standard one my Linux came with (b) the lastest release (c) the CVS version
I need (a) to run system scripts (/usr) I need (b) to check my code runs for clients (/usr/local), and I need (c) cause I love playing with Ocaml :-)))
In addition I have to build my software at least in both binary and bytecode form to check clients on non-Linux platforms have some chance of using my product if they can't get a native code compiler.
And then, there are versions of third party libraries ... ouch.
I have a starting concept here: I call it 'mount points'. And another called 'mount stacks'.
A mount point is a root, specified as an OS native directrory name. Files and directories down from the mount point are always specified using Unix convention (no matter what platform).
Then we have stacks. We start at the top mount point of the stack and look for a file, going down until we find it. However we always write on the bottom. Here is a stack:
The idea is you had hide *some* files with an updated file, but don't need to hide them all with a completely new distribution. [A variant would just use patch files]
Additionally, writes (of object files etc) are always most local and so don't interfere with administrator installed compiled versions.
Another key part of it is relative file lookups. The relativity is to a directory stack NOT a particular directory so eg C #include "foo.h" inside the file "bar.h" will follow the same stack search algorithm except it will look in the mount point of the stack + the subdirectory of the file "bar.h".
On Sun, Nov 23, 2003 at 01:34:22AM +1100, skaller wrote: > On Sat, 2003-11-22 at 06:04, sylvain.le-g...@polytechnique.org wrote:
> > I don't want to reproduce make bugs... I just want to do something > > simple where you can type :
> > let my_prog = { > > name : "my_prog"; > > compile_target : [Opt;Byte]; > > toplevel_files : [ "my_prog.ml" ; > > "something_ocamldep_dont_recognize_as_toplevel.ml" ]; > > package : "all_my_life" ; > > target_tag : Binary; ( or Documentation or Custom of string ) > > }
> > let _ = > > add_target my_prog
> > Ie, it is not makefile syntax.
> My guess is that the 'object oriented design > paradigm' has something to say here -- > do it bottom up. The first step is to define > things like
> compare_time_stamp(f1, f2)
> a module for 'dependency graph', and > other tools that may be useful.
Yeah, you are right, i was just showing you the final step... Ie what the user will have to write to compile...
I don't want to specify a full language schemes... Too much complicated. I just want to make it in ocaml in order to ease ocaml user to write ocaml script to compile ocaml ( i repeat a lot of time ocaml because i still think that it is enough expressive to describe a makefile process and i don't want people to learn 10 language to create : a configure.in, a Makefile.am... ).
> Then just write plain Ocaml code using these tools.
> This may be a bit long winded, so check out camlp4 > to sweeten it up.
> The advantage of this approach is that a make script > is an arbitrary caml script.
> Which means: it can do anything easily (since caml > is so expressive), it is also type checked (hey, > why shouldn't build scripts be type checked?)
> etc etc etc ...
> Finally after all that if you still think a language > translator is needed to express the build conditions > above and beyond caml with camlp4 ... then one > can be written, plugged into the front, invoked > by the caml build script, compiled and executed .. > all without requiring ME or anyone else > to use your language (you can just embed the > translator in your package).
> This is basically what interscript does only > its hooks Python and it's aimed at packaging > software components (meaning source codes and > documentation) in a more general fashion > than merely building them would require. > It actually has a language translator, > but it's a pretty brain dead one that basically > allows code, doco, and executable script to be > mixed together in a single source file.
Maybe i miss one step : where can i found interscript ( i am not an expert, i am sorry ).
> > For example .c -cc--> .o -link--> exe, we can > > use the cached .o instead of running cc on the > > .c provided the .c is older.
> Older is incorrect, as well. The program should run cc on the .c > provided that the .c is different than the one used to produce the .o. > Timestamps are useful as a cache, but a proper tool will need to use > file hashes, or something to properly detect a file changing.
That's true doing a hashing is nice for C, but ocaml compilation is fast enough so maybe the time difference between making the hash and compiling is not so big. Another issue is storing the hashing in a temp file, while you can compare directly time stamps of the object with the source. Actually the only case were hashing is useful compare to timestamps is when a file is only touched ( am I wrong ? ). This is not an average case, and when it's happening juste compiling only this file (without a bunch of .h) seems fast enough.
On Sun, Nov 23, 2003 at 12:25:13PM +0900, Nicolas Cannasse wrote: > > Older is incorrect, as well. The program should run cc on the .c
> That's true doing a hashing is nice for C, but ocaml compilation is fast > enough so maybe the time difference between making the hash and compiling is > not so big. Another issue is storing the hashing in a temp file, while you
Saying "older" is wrong not because it just compiles too often, but because it also doesn't compile things it should. Think of someone moving files around without changing timestamps, and then putting the old files back. Normal make won't recompile them.
It is better to comile when you shouldn't than it is to not compile when you need to.
And compiling ocaml isn't always fast. The code I'm working on doesn't compile the ocaml code any faster than the C code.
On Sun, 2003-11-23 at 14:25, Nicolas Cannasse wrote: > That's true doing a hashing is nice for C, but ocaml compilation is fast > enough so maybe the time difference between making the hash and compiling is > not so big. Another issue is storing the hashing in a temp file, while you > can compare directly time stamps of the object with the source. Actually the > only case were hashing is useful compare to timestamps is when a file is > only touched ( am I wrong ? ). This is not an average case, and when it's > happening juste compiling only this file (without a bunch of .h) seems fast > enough.
Perhaps we should create a Sourceforge project? Or add a new CVS module to ExtLib project?
BTW: one use of hashing is a language specific *content* hash. This is something that can ignore whitespaces and comments for example.
For C and C++, the compiler is slow and hashing could be worth it.
For Ocaml, the compiler is fast BUT there is a strict ordering requirement not present in C which means a file close to the "start" of a dependency graph is worth hashing to avoid recompiling not just that file .. but all the others after it: Ocaml separate compilation isn't quite as 'separate' as C's :-)
Perhaps you should take a look at Vesta (www.vestasys.org) It uses a functional language to describe how to build a system, uses hashes to determine file equality, and caches derived files for reuse. IT seems to be the result of some very bright people sitting down at a blank piece of paper and designing a build management system.
On Fri, Nov 21, 2003 at 09:05:22AM -0800, Jason Hickey wrote: > We have been using omake to build several large projects, primarily on > Linux and Windows. omake is written in OCaml, and provides a build > system with syntax similar to make, but project-wide dependency > analysis. Here are some features:
Is there a place to discuss omake?
Specifically, my question: is there something like the GNU Make 'shell' function? What I'm trying to do is use the output of 'ocamlc -where' in a variable. I need to run extract_crc as part of my build, and ocamlc -where is a nice portable way of finding this directory.
|
