Is there a convenient way to develop OCaml code, and be sure that
this code will be configurable, compilable, installable and
executable without changes, on any environment where OCaml is available?

I imagine a higher level "make", with most of its nicer properties, plus
many enhancements.
The following points seem important to me:

- target dependency system
- ease of use, nice syntax (for average humans with a simple text editor)
- loadable modules
- built-in programming language
- built-in fileutils (cp, mv, cd, rm, ...) and functions (filters, ...)
- universal paths for filenames inside the project tree
- natural highlighting of environment-dependent elements
- possibility of harmonious integration in a graphical development
  environment

and also a very flexible license that allows its free redistribution and
modification under any form.

Until right now I didn't find anything like this. And I was really scared
when I discovered Apache Ant.

Any suggestions?

Martin

In fact, my aim is to construct approximativelly what you are looking for
but for now, i have not yet the time to go over the whole things.

Ocaml-fileutils is the base path abstraction for files and fileutils.
Ocaml-preprocessor is a replacement for *.in -> * ( already developped,
very simple, it is something were you bind variable and function and
you pass it on files where there is for example @funct "abcd"@ and it
replace it with the result of the binding of funct "abcd"... ). It is
also not yet published, but it seems stable to me.

I want to develop the makefile in ML. Each module will register
variable, target, dependency and a big program will run to compute all
this and compile file, lib, exec...

For now, it is only at an early stage ( i have some lexer/parser to
autosort ocamldep generated file and try to figure how to compile only
one thing ).

If you are intersted, just drop me an email, to have more information,
or if you want to test ocaml-fileutils or ocaml-preprocessor.

Kind regard
Sylvain LE GALL

What is the difference with OCamake ?
http://tech.motion-twin.com/ocamake/

Nicolas Cannasse

There are several packages in the scons approach that seem like good
ideas, but they all make very heavy use of the scripting languages they
are written in.

I still haven't found anything nicer than gnatmake, the builder for Gnu
Ada.  A common compilation line:

  gnatmake mainmodule

Builds mainmodule.adb, as well as everything it depends on.  You can set
a path to search for parts.  This would work for ocaml, if a program had
a single module that brought in everything necessary via dependencies.

Dave

The first problem I see is that right now ocamldep is only giving you the
dependencies between files that are listed in the ocamldep command. That
means that you already need to know all the files before generating the
dependencies between them.

The other problem is that to much of the "special programs build" are
relying on OS tools that are sometimes not stricly compatible. The solution
to this is to rewrite all the tools in pure ocaml ( or with a small portable
C binding ) to give the "makefile" writter an API of portable commands. It
looks like this is the goal of Sylvain, and that would be really something
nice.

About OCamake : yes right know it only supports compilation of
ml/mli/mll/mly files in a standard way, but it is now mature and is actually
handling a lot of cases such as projects with multiple files paths and a
best effort dependency resolver (although based on ocamldep output). Adding
the auto building of C stubs could be done but this kind of tool need a lot
of time in order to fix correctly all the small issues.

Nicolas Cannasse

Dave

Well, i will maybe stole your code ;->

My aims is to make something more Makefile like ( ie you will have the
ability to add rules ). Off course, my first goal is to compile ml file.
So there will be less features than Makefile.

On the other hand the aim will be to have a self contained tools ( ie
when you write a makefile you use rm, cp, touch... here the aims will be
to have it in ocaml ). So you could build on every platform which
support ocaml.

Your makefile will be written in ocaml !

Regard
Sylvain LE GALL

Rich.

--
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
PTHRLIB is a library for writing small, efficient and fast servers in C.
HTTP, CGI, DBI, lightweight threads: http://www.annexia.org/freeware/pthrlib/

Martin

     - omake runs on Unix, Windows, MacOS, and presumably
       other architectures where OCaml is available.
     - dependency analysis is project-wide (like cons),
       based on MD5 digests
     - automated dependency analysis
     - there is builtin support for OCaml and C code,
       and it is easy to add support for other kinds
       of files (just like make).
     - the OMakefile syntax is similar to GNU make, but
         - omake has user-defined functions
         - OMakefile programs are functional
         - the .SUBDIRS target is used to define
           the project hierarchy
         - different parts of the project can have
           different configuration.

omake is available by anonynous CVS from cvs.metaprl.org.
    % cvs -d :pserver:anon...@cvs.metaprl.org:/cvsroot login
    The password is anoncvs.
    % cvs -d :pserver:anon...@cvs.metaprl.org:/cvsroot checkout omake

Alternatively, RPMs are available at rpm.nogin.org.

Here is a short description.  Every project must have an OMakeroot file
in the project root.  It is usually boilerplate; this is typical:

    # Include the standard configuration
    include $(STDROOT)

    # Include the OMakefile
    .SUBDIRS: .

The project commands are then placed in an OMakefile.  To build a
standalone OCaml program from files a.ml b.ml and c.ml, you just need
one line.  The OCamlProgram function is defined in the system OMakeroot.

    OCamlProgram(foo, a b c)

You can choose the byte-compiler, native-code compiler, or both.

    BYTE_ENABLED = true
    NATIVE_ENABLED = true
    OCamlProgram(foo, a b c)

Maybe you have some C files you need to include in your compile as well.
  Perhaps f.c is a generated file.

    f.c: f1.c f2.c
        cat $+ > $@
    StaticCLibrary(bar, d e f)
    LIBS = bar
    OCamlProgram(foo, a b c)

Perhaps you use the C-preprocessor on some .mlp files:

    %.ml: %.mlp
        $(CPP) $*.mlp > $@

The system sources contain more examples, and the MetaPRL system, also
available at cvs.metaprl.org, provides a very large, complex, example.

Jason

At best, you can cheat with system() function
but then all the dependency checking is lost.

Make systems are all conceptually wrong.
There a huge number of projects to replace
Make (and autoconf et al) and not a single
one I've seen recognizes that the fundamental
design is flawed .. instead they all try to
copy the make idea.

First observe that building a system
is a process where it is not always
possible to determine the full sequence
of commands in advance. Sometimes,
you have to perform some action before
you can decide what to do next.

Example: you cannot build interscript
packaged sources or a tarball directly,
you have to extract the contents first.

Example: the rules for building a package
may have to be generated or extracted
from the package.

Next observe that the linear dependency
checking is naive. First, what some
target depends on can be conditional:
obviously true for C :-)

But it is much worse. A target can
depend on itself. Interscript assumes that.
An example is a Latex build, which depends
on auxilliary files generated by the build
[Ocamldoc output can take 4 passes to fixate
for example].

This leads to the first novel idea. Fixpoints.
Interscript is based on that idea.

With fixpoints, you dont CARE about dependencies.
What you do is repeatedly execute some process
until the defined outputs are stable. The inputs
do not need to be known.

Now we backstep. We try to *predict* whether
the outputs will be the same this run as last.
If we can -- and we have to be *sure* -- we
can terminate one cycle earlier.

This is dependency checking with a new twist:
its optional. :-)

Well, I will jump ahead now. A build system
does NOT have targets. That's a bad idea too.

A build systems has processes and files.
Lets call the processes ARROWS and the files OBJECTS.

Oh. That's a category. We allow of course,
products of files (multiple outputs and inputs)
and sums of files (choose one of several).
[And of course we'd better make it cartesian
closed and allow processes to be files ..
called scripts .. :]

The control system allows you to 'click' on
an arrow to execute the process. This is manual
running. It also allows to to click on a object
to run all the processes required to produce
the object. By using a caching concept, some
process can be elided as an optimisation.

For example .c -cc--> .o -link--> exe, we can
use the cached .o instead of running cc on the
.c provided the .c is older.

BTW: I have *built* this system, including
a graphic interface which allowed construction
of the build category.

I needed this much power. Make is a total joke.
What I was doing was writing a book, literate
programmed of course, and needed to generate
a LOT of different documents -- I had to build
code, test it, capture the output, typeset
the book, build indices, contents ...

In a *real* build system, you do NOT want
to make everything from scratch everytime.
Sometime you're happy using old data,
at least for a while.

Anyhow .. a lot of new ideas for a build
system are here. Just copying the make concept
is not going to solve any problems .. its going
to make things much WORSE by having yet another
inadequate tool -- which of course the poor client
will have to build, making the tool chain even longer.
[Trying to replace Make destroyed Boost IMHO]

-Eric

It seems great to me...

Is there way to define camlp4 syntax ? ( for example XXX.ml needs camlp4
with cmo zoggy.cmo or something like that )

Is there a kind of configure in it ?

Can you use META files.

(... a lot of other question but i will take a look at it before asking
).

Kind regad
Sylvain LE GALL

I was thinking of dynlink for loadable module...

But i need to investigate on other tools ( omake ).

Regard
Sylvain LE GALL

I don't want to reproduce make bugs... I just want to do something
simple where you can type :

let my_prog = {
        name : "my_prog";
        compile_target : [Opt;Byte];
        toplevel_files : [ "my_prog.ml" ;
        "something_ocamldep_dont_recognize_as_toplevel.ml" ];
        package : "all_my_life" ;
        target_tag : Binary; ( or Documentation or Custom of string )
        }

let _ =
        add_target my_prog

Ie, it is not makefile syntax.

But, i don't really have begin my project... So i can't already said
that i will use this or that...

Kind regard
Sylvain LE GALL

ps : if you are interested, i will contact you when i will consider
enough mature to begin with this huge task ( in fact, i have made a kind
of start with the findlib branch of cameleon, but i am not finished ).

if true
    OCAMLFLAGS += -pp "camlp4 zoggy.cmo"
    XXX.cmx XXX.o:
    XXX.cmo:

The idea is the following:
- variables inside the "if" statements are locally scope (unless you
explicitly export the variable environment back to the parent scope), so
we use the "if true" as a synonym for "local". So, the above tells omake
to add a preprocessor flag to ocamlc/ocamlopt when compiling XXX.cmx,
XXX.o and XXX.cmo.

Alternatively, you could define a helper macro:

UseCamlp4(modules, files) =
    OCAMLFLAGS += -pp "camlp4 $(addsuffix .cmo, $(modules))"
    $(addsuffix .cmx, $(modules)) $(addsuffix .o, $(modules)):
    $(addsuffix .cmo, $(modules)):

UseCamlp4(zoggy, XXX)

Actually, it might be a good idea to add such a macro to the global
rules file included with omake - I filed
http://cvs.metaprl.org:12000/bugzilla/show_bug.cgi?id=119 on this.

.INCLUDE: foo
    bla bla > foo

.INCLUDE: foo: foo.in
    bla bla bla bla < foo.in > foo

The above tell omake that
- it should include foo and if the file does not exist, it should use
"bla bla" to generate it. Next, it should check whether foo is newer
than foo.in and generate foo again (using "bla bla blah blah" this time)
and then include it again.

You can also specify dependencies for the OMakefile itself:

if $(not $(equal $(OSTYPE), Win32))
     OMakefile: OMakefile.in config.status
    @echo "*** Build system is out-of-date, rebuilding ***"
    $(DOT)config.status
    @echo "*** OMakefile was rebuilt, session should restart ***"

On 21.11.2003 08:12, skaller wrote:

--
Aleksey Nogin

Home Page: http://nogin.org/
E-Mail: no...@cs.caltech.edu (office), alek...@nogin.org (personal)
Office: Jorgensen 70, tel: (626) 395-2907

Gerd Stolpmann write it ( www.ocaml-programming.de ). Sorry to the
author if i mistyped his name...

I take a quick glance to you OMakefile...

Well i think it is very powerfull and very interesting... But ( there is
always a but ), i am looking for something between your very complete
suite and ocamake which is light and permits to do fast job for ocaml
stuff.

I know, i will run into problem if not targeting all people... But, i
cannot choose all options. I want something enough powerfull to build
little to medium ocaml project -- with some C binding. I don't want to
rebuild make. The ideal project ( in size ) will be cameleon ( fully
written in ocaml ) or mldonkey ( mostly ocaml with some C ). I don't
really want to be able to compile C, python, Ada...

I want to use some kind of ocaml script ( let say makefile.ml which will
contain the instruction to build, explains the dependency etc ) to
enable user to be fully consistent with their project ( they learn ocaml
and not makefile syntax ). I think it should be necessary to have a
plugin system ( through dynlink or topfind, don't know yet ) to enable them
to develop their own set of rules ( if they want to compile specific
things ).

What i have read in you omakefile is a kind of full rewrite of make ( ie
you have construct of function, dependency etc ). I precisely don't want
to develop my own langage for this purpose, i want to have it written in
ocaml.

Just to give you an example, all the student i know have problem using
makefile because they don't want to spend two hours understanding this
big script... So they simply don't use it... How many times i have seen
one-file-project without makefile, or people using "ocamlc X.ml &&
ocamlc Y.ml && ocamlc Z.ml..."

I want ( just as i have posted before ) :

let my_prog = { name = "my_prog";...;toplevels = [ "X.ml" ] }
in
add_target my_prog

and that's all. And i want to be able to do it on linux or win without
being needed to do if Win32 then ... else ... ( every time you write if
... else ..., it is a possible infinite source of error ).

I will have a deeper look to your omakefile ( understanding how it is
working, if i can learn from the rules to target process ... ).

thanks for having helped me to find more project similar to mine.

Kind regard
Sylvain LE GALL

ps : i cannot find your project omakefile on humps
pps : if you have time you could take a look at cameleon CVS, branch
findlib : http://savannah.nongnu.org/projects/cameleon, there is a bunc
of ten lines makefile which are sufficient to define what to compile for
each directory... That is the mininum sums of information i want user to
give to have a fully working makefile system

[...]

Nicolas Cannasse

OcamlPack(dst, sources) =
    if $(and $(NATIVE_ENABLED) $(BYTECODE_ENABLED))
       $(dst).cmo $(dst).cmx $(dst).cmi $(dst).o: $(addsuffix .cmo,
$(sources)) $(addsuffix .cmx, $(sources)) $(addsuffix .o, $(sources))
$(addsuffix .cmi, $(sources))
          $(OCAMLOPT) $(OCAMLOPTFLAGS) -pack -o $(dst).cmx $(addsuffix
.cmx, $(sources))
          $(OCAMLC) $(OCAMLCFLAGS)-pack -o $(dst).cmo $(addsuffix .cmo,
$(sources))
    else
       $(dst).cmo $(dst).cmi : $(addsuffix .cmo, $(sources)) $(addsuffix
.cmi, $(sources))
          $(OCAMLC) $(OCAMLCFLAGS)-pack -o $(dst).cmo $(addsuffix .cmo,
$(sources))

       $(dst).cmx $(dst).cmi $(dst).o: $(addsuffix .cmx, $(sources))
$(addsuffix .o, $(sources)) $(addsuffix .cmi, $(sources))
          $(OCAMLOPT) $(OCAMLOPTFLAGS) -pack -o $(dst).cmx $(addsuffix
.cmx, $(sources))

And then just invoke it using

OcamlPack(foo, bar1 bar2 bar3 ... )

--
Aleksey Nogin

Home Page: http://nogin.org/
E-Mail: no...@cs.caltech.edu (office), alek...@nogin.org (personal)
Office: Jorgensen 70, tel: (626) 395-2907

Note that if one would want to use omake for simple projects, all that
[s]he would have to learn is couple of macros (which completely hide the
notions of "target", "dependency", etc) and couple of standard
variables. E.g, you'd need couple of line for the project:

OCAMLC = ocamlfind ocamlc -package "fileutils"
OCamlProgram(my_prog, X Y Z ...)

Note that the standard macros are not in any way built-in (the way make
has a big number of built-in rule), they just come from an include file
(that can be studied and even modified if a small simple project ends up
growing into a large and complicated one).

Note - if in the example above you do not want to rely on external
ocamlfind, you could add ocamlfind code to omake, changing the above to

OCAMLINCLUDES += $(ocamlfind fileutils)
OCamlProgram(my_prog, X Y Z ...)

--
Aleksey Nogin

Home Page: http://nogin.org/
E-Mail: no...@cs.caltech.edu (office), alek...@nogin.org (personal)
Office: Jorgensen 70, tel: (626) 395-2907

* as someone else pointed out, relying on convergence towards
   fixpoints may be too strong. some build processes may
   never converge, rather they evenually stabilise in some
   "open ball", so you might need some (user supplied) notion
   of "close enough". how do you do that?

* according to [1], a cartesian closed category with fixpoints
   and finite sums is equivalent to the category with one object
   and one arrow. how do you deal with this?

martin

[1] Hagen Huwig and Axel Poigne, A note on inconsistencies caused
by fixpoints in a cartesian dosed category, Theoretical Computer
Science 73 (1990), p. 101-112.

        compare_time_stamp(f1, f2)

a module for 'dependency graph', and
other tools that may be useful.

Then just write plain Ocaml code using these tools.

This may be a bit long winded, so check out camlp4
to sweeten it up.

The advantage of this approach is that a make script
is an arbitrary caml script.

Which means: it can do anything easily (since caml
is so expressive), it is also type checked (hey,
why shouldn't build scripts be type checked?)

etc etc etc ...

Finally after all that if you still think a language
translator is needed to express the build conditions
above and beyond caml with camlp4 ... then one
can be written, plugged into the front, invoked
by the caml build script, compiled and executed ..
all without requiring ME or anyone else
to use your language (you can just embed the
translator in your package).

This is basically what interscript does only
its hooks Python and it's aimed at packaging
software components (meaning source codes and
documentation) in a more general fashion
than merely building them would require.
It actually has a language translator,
but it's a pretty brain dead one that basically
allows code, doco, and executable script to be
mixed together in a single source file.

The idea of fixpoints is still novel and useful here.
As you comment, oscillations are possible. Divergence
is also possible. Interscript has a command line like

        iscr --passes=4 other options ...

which limits the number of passes. The default is one
pass, but it has a persistence mechanism (Python marshal),
so that executing the command 4 times is roughly equivalent
to a single invocation with --passes=4, except that with
the --passes=4 option 1 to 4 passes are executed.

Interscript is also Make friendly -- if it writes the
same as a files contents the timestamps are not changed.
[Which is how it detects convergence :-]

It *also* has dependency checking which can detect
that a pass would be the same as the last pass and
so it is possible that 0 passes are executed.

The design is not particularly efficient, however
it handles things like: you can read a file that
doesn't exist, and later write it. No problem.
The error on the first pass is ignored. The file
is there on the second pass .. although it might
not fix.

BTW: there is a fun caveat. The one thing that
you must be careful NOT to ever do in Interscript
is put the code generation time directly into the generated
code -- that guarrantees divergence :-)

Second .. well, Ocaml uses this model 'as inspiration'
as well, and it's authors know a good deal more theory
than me so they're more likely to be able to answer :-)

Yeah, both answers are a cop-out (meaning I don't know :-)