C POSIX compliant Makefile - Pattern Rules?

J

JAW

I'm trying to remove any implementation specific (GNU Make) details from my Makefiles but having a hard time fixing some issues. The structure of my projects is like so;

Code: 
myapp/
  Makefile
  src/
     foo/
        foo1.c
        foo2.c
     bar/
        bar1.c
  obj/

1) The mere presence of an obj directory at the root level of my project causes BSD make to chdir to it before doing anything else. I can avoid this by putting compiled objects into tmp instead, but it is annoying nonetheless! What is the rationale behind this unexpected behaviour?

2) I have a GNU make pattern rule (see below) to allow building an object file in the obj directory tree from the respective source file in the src directory tree. I understand that POSIX make doesn't support pattern rules such as this, and instead one must use suffix rules such as .c.o instead. However, using a suffix rule does not seem to cope with the directory tree structure which is handled by the GNU make pattern rule;

Code: 
$(OBJDIR)/%.o: $(SRCDIR)/%.c
        @$(MKDIR) $(@D)
        $(CC) $(CFLAGS) -c $< -o $@

Please could someone show an example of how this can be accomplished in a standards compliant way?

Thanks,
James
 
What are you trying to accomplish? Write a single Makefile, compatible with both -- FreeBSD make and GNU make? Not sure if that would be that easy. In my portable projects I create two makefiles: BSDmakefile and GNUmakefile, that contain platform-dependent parts and include common configuration, like list of source files, CFLAGS, LDFLAGS, etc.

Or I create just one (GNU) Makefile and a simple BSDmakefile containing just
Makefile: 
all clean:
    gmake $@
 
Bobi B. said:
What are you trying to accomplish? Write a single Makefile, compatible with both -- FreeBSD make and GNU make? Not sure if that would be that easy. In my portable projects I create two makefiles: BSDmakefile and GNUmakefile, that contain platform-dependent parts and include common configuration, like list of source files, CFLAGS, LDFLAGS, etc.
Click to expand...

Yep, I'm trying to create a single POSIX compliant Makefile that *should* work on anything. Apart from this issue everything else is cross platform, I was wondering if I had missed something obvious as it seems a fairly basic requirement (to be able to put sources into a src directory and the objects in an obj directory).
 
True, POSIX and BSD make do not support pattern rules. I don't think suffix rules can be used for directories, as you discovered. If you want portability by using POSIX make only, yet you insist on using simple rules and a directory for your object files, I think you're sort of sunk.

On the other hand, I understand the benefit of having a single makefile and using it for all platforms. Much less work, much less potential for errors. Fully agree that this is a worthy goal.

Here are a few options.
  • First, the disgusting one: Simply define gmake to be the standard, and use it on all platforms. It is available everywhere, with some effort. I've worked on projects where this was done, and it is easy and convenient. Personally, when I'm in control, I flat out refuse to do that, because it dumbs us down to the lowest common denominator.
  • Second, give up on putting your object files into a separate directory. Really, that is not necessary; it is perfectly fine to have the object files mixed into the source directories, as long as you don't try to use the same source tree for multiple deployment platforms.
  • Third: makefiles are so nasty to write, give up on hand-writing them at all. To a large extent, makefiles are already computer generated today: most people use "make depend" to create dependency file(s) somewhere, which is then used by make. Just go one step further: design a new meta-language in which you can describe a stripped-down make file, and write a simple script that generates the make file from your meta-makefile. When doing that, in addition to creating the dependencies, also create an explicit rule for each object file, which shows how that file is to be made. This will only take a few hours to design and build in a scripting language.
What I do for personal projects is to have three layers of makefiles. At the outermost layer, there are two makefiles, which explicitly depend on which flavor of make is used, and they are called "GNUmakefile" and "BSDmakefile". If you say "make foo", then gmake will read the first one, and BSD make will read the second one. Those two files determine the OS that is running, and immediately include files with names like "Makefile.Darwin", "Makefile.FreeBSD" and "Makefile.Linux". In the case of FreeBSD, I have to further distinguish between versions, so the "Makefile.FreeBSD" then calls "Makefile.FreeBSD.11" and so on. This is where OS-dependent settings are done, which includes compiler-specific settings (since Linux uses the Gnu compilers, while FreeBSD uses Clang/LLVM). And finally, they all include a file that's simply called Makefile, which then only contains the list of targets, and the dependencies, with explicit rules.

P.S. I'm a crazy person: I insist on hand-writing the dependencies. This is probably a bad idea, because it is a lot of extra work, and error-prone. I justify it by saying that I want to manually see all dependencies, because if they disagree with how I designed the software, there is something wrong in the implementation. But it is really hard to catch all dependencies by hand, so I occasionally have to run something like "make depend" and desk-check that my hand-written dependencies are actually correct.
 
  • Thanks
Reactions: JAW
JAW
I'm not here to solve your question. I only came to here for learn something.
What you're trying to do, exactly?
 
ralphbsz said:
True, POSIX and BSD make do not support pattern rules. I don't think suffix rules can be used for directories, as you discovered. If you want portability by using POSIX make only, yet you insist on using simple rules and a directory for your object files, I think you're sort of sunk.
Click to expand...

Ok thanks, good to hear it straight up! I just wanted to be sure I wasn't missing something obvious from the POSIX make dialect. :)

First, the disgusting one: Simply define gmake to be the standard, and use it on all platforms. It is available everywhere, with some effort. I've worked on projects where this was done, and it is easy and convenient. Personally, when I'm in control, I flat out refuse to do that, because it dumbs us down to the lowest common denominator.
Click to expand...

This is my fallback option, which sadly is becoming more than likely... :'‑(

Second, give up on putting your object files into a separate directory. Really, that is not necessary; it is perfectly fine to have the object files mixed into the source directories, as long as you don't try to use the same source tree for multiple deployment platforms.
Click to expand...

It's nice to have the option of simply deleting the obj directory to clean everything up (which also deletes any orphaned objects from sources that may have been renamed).

Third: makefiles are so nasty to write, give up on hand-writing them at all.
Click to expand...

I generally like doing things the hard way, to get an understanding of what is happening behind the scenes. o_O

Those two files determine the OS that is running, and immediately include files with names like "Makefile.Darwin", "Makefile.FreeBSD" and "Makefile.Linux". In the case of FreeBSD, I have to further distinguish between versions, so the "Makefile.FreeBSD"
Click to expand...

I much prefer this approach for OS specific stuff than littering a single file with conditions scattered all over the place. :)

P.S. I'm a crazy person: I insist on hand-writing the dependencies. This is probably a bad idea, because it is a lot of extra work, and error-prone. I justify it by saying that I want to manually see all dependencies, because if they disagree with how I designed the software, there is something wrong in the implementation. But it is really hard to catch all dependencies by hand, so I occasionally have to run something like "make depend" and desk-check that my hand-written dependencies are actually correct.
Click to expand...

But at least you understand what it's actually doing! :cool:

Thanks for replying, it's all good food for thought!
James
 
BSDAppentic3 said:
JAW
I'm not here to solve your question. I only came to here for learn something.
What you're trying to do, exactly?
Click to expand...

Hi, I'm trying to change an existing Makefile to be standards compliant so that I can use it across various platforms. Until recently I had been developing the project on Linux and Windows (with MinGW) and hadn't realised I was using a bunch of GNU Make specific extensions which do not work with BSD (or any other) Make. So I'm trying to figure out how to write a clean Makefile using only the POSIX standardised version of Make.

A specific problem that I've come across is GNU Make allows you to put your C sources in a different folder to the compiled object files using a "pattern rule", which POSIX Make does not support.

Hope that makes sense! :)
 
I applaud the OP for trying this. Many projects claim portability while imposing a specific tool chain ... which isn't really portable.

The short answer is include the folder prefix in your Makefile. find(1) and xargs(1) can help generate this.

CC = clang
LIBS = ...
INCLUDES = ...

obj/myapp : obj/foo1.o obj/foo2.o obj/bar1.o
$(CC) obj/foo1.o obj/foo2.o obj/bar1.o $(LIBS)

obj/foo1.o : src/foo/foo1.c
$(CC) -c src/foo/foo1.c $(INCLUDES)

obj/foo1.o : src/foo/foo2.c
$(CC) -c src/foo/foo2.c $(INCLUDES)

obj/foo1.o : src/bar/bar1.c
$(CC) -c src/bar/bar1.c $(INCLUDES)


The slightly harder part is figuring out which source files depend upon which header files. I sometimes write a bespoke utility that parses each source file to figure out which header files to include as dependents.

A makefile in this explicit format should work for bmake/pmake/gmake/nmake. If you have dozens of files, this can get out of hand.

A better alternative is to use cmake(1). Create a CMakeLists.txt file (start from a "hello world" example). cmake will generate the Makefile from that.

Another wicked option is to run juCi++ (pkg install juci). This C/C++ IDE allows me to browse to a folder with source files, select "recreate build" from the Projects folder. The IDE digs through the subfolders to generate the CMakeLists.txt file for me. I can compile (and debug) from the IDE or drop to a shell prompt and type 'make' or 'make clean'.
 
  • Thanks
Reactions: JAW
Crivens said:
Calling gmake from make is like typing google into bing.
Click to expand...
I code under editors/emacs, sometimes editing remote files thru an ssh tunnel, and I have a keyboard binding for compile command, that always runs make, no matter what local or remote OS is. Older projects, that started under Linux, use GNU makefiles, hence this... hack... suits me fine. I'm not saying it is pretty or elegant, tho.

unitrunker said:
The slightly harder part is figuring out which source files depend upon which header files. I sometimes write a bespoke utility that parses each source file to figure out which header files to include as dependents.
Click to expand...
-MM flag of gcc does that; clang supports it, as well.

FreeBSD includes a set of ready-made template Makefiles you can reference when coding; see ls(1)' Makefile, for example (/usr/src/bin/ls/Makefile). There are also templates for kernel modules and ports.
 
Most forms of "make depend" are implemented by calling gcc with the "-MM" flag.

Here is another alternative: Use a build system other than make. I know alternatives exist, build systems created from scratch that don't rely on make. I've used them in non-open commercial projects, and they can work much better than traditional make. But I don't know any FOSS versions, and even less have evaluated those.
 
Thanks for all the replies, it's interesting for me to see the various approaches. For now I've gone with simply adding sources with the find command, building the objects alongside the sources in the tree (rather than in a seperate obj directory), and cleaning them up with a find / rm command;

Code: 
EXE = myprog
RM = rm -fv
FIND = find
SRCDIR = src
SOURCES != $(FIND) -s $(SRCDIR) -type f -name "*.c"
OBJECTS := $(SOURCES:%.c=%.o)

...snip...

.PHONY: clean
clean:
        @echo Cleaning...
        @$(FIND) -s $(SRCDIR) -type f -name '*.o' -exec $(RM) {} \;
        @$(RM) $(EXE)

Need to also handle headers, might just rebuild everything if a header changes for now!
 
So, you are trying something big, as far as I can see...well, my level it's very low. And I don't have time to study everything that's related with this, ergo, still I haven't make something important.
My own project it's compilate a driver for my USB WiFi card.
Well, I wish you the best of the lucks.
 
JAW, I wouldn't accept constructs like SOURCES != $(FIND) -s $(SRCDIR) -type f -name "*.c" in projects I'm responsible for :-) (although it is handy, never knew this syntax var != shellcmd, but seems like it is not in POSIX make); better list the names of source files by hand. Compiling all sources automatically might surprise you one day, not in the pleasant sense.
 
  • Thanks
Reactions: JAW
That reminds me... Once I wrote a "lint for object files" that could dig trough a set of objects and libraries and tell you from the debug information if you screwed up something. Later the architects needed to explain why two sets of zlib were around with different build options and the three (almost) identical abstraction layers for it.

But that is where you are heading with finding all source files and building them.

Oh, and that #ifdef in class definitions in include files, yeah. What a brilliant idea.
 
Bobi B. said:
JAW, I wouldn't accept constructs like SOURCES != $(FIND) -s $(SRCDIR) -type f -name "*.c" in projects I'm responsible for :) (although it is handy, never knew this syntax var != shellcmd, but seems like it is not in POSIX make); better list the names of source files by hand. Compiling all sources automatically might surprise you one day, not in the pleasant sense.
Click to expand...

Damn it, you are right var != shellcmd is in both GNU make and BSD make, but not POSIX make. :'‑(
 
Crivens said:
[BGCOLOR=transparent]Oh, and that #ifdef in class definitions in include files, yeah. What a [/BGCOLOR]brilliant[BGCOLOR=transparent] idea.[/BGCOLOR]
Click to expand...
[BGCOLOR=transparent][/BGCOLOR]
[BGCOLOR=transparent]Serious?[/BGCOLOR]

[BGCOLOR=transparent]In the early days of C++, compilers and linkers were so broken, and we were so overworked with porting existing C code to C++ that we sometimes had to put that ifdef into include files: if one had to have a function implementation in the include file, make sure it only gets compiled once, otherwise early linkers used to blow up. This is the kind of garbage that gets cleaned up within a few months.[/BGCOLOR]

[BGCOLOR=transparent]Later on, we sometimes had young "genius" coders discover tricks like this, and try to use them. Fortunately, we also had code reviews and more experienced people, and the young geniuses were quickly taught common sense: while this kind of trick works, it is extremely brittle, and leads to unmaintainable code. We did have a case where a young programmer refused to learn this lesson, and checked the broken code into the production source control system. After the second time he did that, corporate security came and walked him to the parking lot, brought him his personal stuff (like coffee mug and picture of his wife) in a cardboard box, and his employment ended right there. This is not a joke: I've had a colleague fired on the spot for violating the "coding rules" document, and we all considered it to be a good thing.[/BGCOLOR]
 
Are you ladies & gentlemen seated? Yes? Very well then.

class foo {
#ifdef BAR
int something;
..
#endif

.... more decls ...
};

Doubleplusunfunny when BAR is not defined on all users of that file. I think we need some thread for stuff like this. I have seen some pretty horrible things. :(
 
It's too bad that this forum software only has a "thanks" button, not a "laugh" button. That code you posted is so horrible, it should not be cause for firing, but for requesting medical help for the person who wrote it: clearly, their brain has become disconnected from the fingers.

What I was referring to was more harmless: In the early days of C++ (say 1991 or so), every function had to be defined (implemented) exactly 1 time. Not 0 (obviously), but also not 2 times, that would lead to a link error. Unfortunately, some functions were in header files. So the idiom we had to use for a while was this:
Code: 
class foo {
    int bar(int x)
#ifdef ONLY_TRUE_IN_ONE_C_FILE
    { return x * elefant; }
#else
    ;
#endif
...
};
This works. And with extremely good discipline among coders, it can be kept working. The code is pretty unreadable, but that's par for the course in a broken engineering culture . But it is evil: one little slipup in the #defines (which might be coming from the makefile instead of the source, if the same source can be used for different projects), it will blow up. And finding the problem is hard, in particular for the inexperienced. So our department outlawed this usage.
 
Necro post!

I recently came across this issue and think I found a solution.

So we have this

Code: 
OBJ= \
    foo.o

all: $(OBJ)

.SUFFIXES: .c .o
.c.o:
    cc -c -o$@ $<

Lets say that your .c files are in a subdirectory called src. It will complain saying it cannot create foo.o because it cannot find foo.c
So we add the following:

Code: 
  foo.o: src/foo.c

And that should allow it to compile because it explicitly states where the .c file is in relation to the .o file. This then means that the inference rule will kick in and work but... Its a bit naff because we don't want to have to do that for every file. So we change it to the following:

Code: 
  $(OBJ): src/foo.c

And this will mostly work until we add bar.c to the OBJ list. Then it will fail again because it cannot find the corresponding src/bar.c file for the inference rule. So we change it.

Code: 
  $(OBJ): src/$(@:.o=.c)

Now what this will do is it will substitute the .o with a .c in the path name as it is using the rule. This will tell make for each .o file where the respective .c file is; ready for the .c.o inference rule to take over and work.

We don't need GNU pattern rules at all basically. A full example is here:

Code: 
OBJ= \
  foo.o \
  bar.o

BIN=prog

all: $(BIN)

clean:
        rm -f $(OBJ)
        rm -f $(BIN)

$(BIN): $(OBJ)
        cc -o$(BIN) $(OBJ)

$(OBJ): src/$(@:.o=.c)

.SUFFIXES: .c .o
.c.o:
        cc -c -o$@ $<

Edit:
Its actually much more powerful than demonstrated too; for example you could use general regex to substitute directories:
Code: 
$(OBJ): $(@:S/.o$/.c/:S/^obj\//src\//)
 
Thank you very much, I had dealings with this also.

Oh, and it's not necromancy. It is post-mortem communication!
 
You must log in or register to reply here.
Back
Top