Intro
This is a reply to a comment on doit in reddit. You should be familiar with doit to understand this article. The comment says:
"Much too verbose compared to make syntax." (nirs)
Well, that is actually true, but intentional. Here I will explain why it was designed this way. And show how easy it can be extended to be less (or more) verbose. The design of the syntax was based on the following principles:
- syntax must be easy to understand (even if a bit verbose).
- easy to extend. different domains require different syntaxes to keep it less verbose. Maybe I should say doit is a build-tool framework
doit != make
doit tasks can a bit more complex than a make’s target-dependencies-command (dependencies = prerequisites). doit supports multiple targets, a setup object… and it is extensible i.e. on next release it will support a "clean" parameter. Once you have a fixed syntax it is hard to expand it and add new features. But lets admit if all you want is target-dependencies-command doit is too verbose. right?
Lets say you want to specify it in a similar way to make… but writing straight to python code. You could define this into a text file and parse it.
I will take the example from GNU make manual . I removed part of it because this is a blog post not a book :D
version 0
make example:
edit : main.o kbd.o command.o display.o insert.o files.o
cc -o edit main.o kbd.o command.o display.o insert.o files.o
main.o : main.c defs.h
cc -c main.c
kbd.o : kbd.c defs.h command.h
cc -c kbd.c
command.o : command.c defs.h command.h
cc -c command.c
display.o : display.c defs.h buffer.h
cc -c display.c
insert.o : insert.c defs.h buffer.h
cc -c insert.c
files.o : files.c defs.h buffer.h command.h
cc -c files.c
clean :
rm edit main.o kbd.o command.o display.o \
insert.o files.o
A very naive translation to doit:
DEFAULT_TASKS = ['edit']
COMPILE = "cc -c %s"
def task_edit():
return {'action': ('cc -o edit main.o kbd.o command.o display.o '+
'insert.o files.o'),
'dependencies': ['main.o', 'kbd.o', 'command.o', 'display.o',
'insert.o', 'files.o'],
'targets': ['edit']
}
def task_main():
return {'action': COMPILE % 'main.c',
'dependencies': ['main.c', 'defs.h'],
'targets': ['main.o']
}
def task_kbd():
return {'action': COMPILE % 'kbd.c',
'dependencies': ['kbd.c', 'defs.h', 'command.h'],
'targets': ['kbd.o']
}
def task_command():
return {'action': COMPILE % 'command.c',
'dependencies': ['command.c', 'defs.h', 'command.h'],
'targets': ['command.o']
}
def task_display():
return {'action': COMPILE % 'display.c',
'dependencies': ['display.c', 'defs.h', 'buffer.h'],
'targets': ['display.o']
}
def task_insert():
return {'action': COMPILE % 'insert.c',
'dependencies': ['insert.c', 'defs.h', 'buffer.h'],
'targets': ['insert.o']
}
def task_files():
return {'action': COMPILE % 'files.c',
'dependencies': ['files.c', 'defs.h', 'buffer.h', 'command.h'],
'targets': ['files.o']
}
def task_clean():
return "rm edit main.o kbd.o command.o display.o insert.o files.o"
Sure. This is too verbose. But that’s not the way things should be done…
version 1
a more reasonable "translation".
doit make1:
DEFAULT_TASKS = ['make1:edit']
TASKS = {
'edit': ("main.o kbd.o command.o display.o insert.o files.o",
"cc -o edit main.o kbd.o command.o display.o insert.o files.o"),
'main.o': ("main.c defs.h",
"cc -c main.c"),
'kbd.o': ("kbd.c defs.h command.h",
"cc -c kbd.c"),
'command.o': ("command.c defs.h command.h",
"cc -c command.c"),
'display.o': ("display.c defs.h buffer.h",
"cc -c display.c"),
'insert.o': ("insert.c defs.h buffer.h",
"cc -c insert.c"),
'files.o': ("files.c defs.h buffer.h command.h",
"cc -c files.c"),
'clean': ("",
"rm edit main.o kbd.o command.o display.o insert.o files.o")
}
def task_make1():
for target, rule in TASKS.iteritems():
yield {'name': target,
'targets': [target],
'dependencies': rule[0].split(),
'action': rule[1]}
This version ain’t more verbose than the make version (except the "" and ()). the task method is all you need to "translate" from make syntax to doit syntax. but nobody writes makefiles like this… Lets follow the improvements from the make manual.
Version 2
Does anyone get impressed with variables?
make example 2:
objects = main.o kbd.o command.o display.o insert.o files.o
edit : $(objects)
cc -o edit $(objects)
main.o : main.c defs.h
cc -c main.c
kbd.o : kbd.c defs.h command.h
cc -c kbd.c
command.o : command.c defs.h command.h
cc -c command.c
display.o : display.c defs.h buffer.h
cc -c display.c
insert.o : insert.c defs.h buffer.h
cc -c insert.c
files.o : files.c defs.h buffer.h command.h
cc -c files.c
clean :
rm edit $(objects)
doit make2:
DEFAULT_TASKS = ['make2:edit']
objects = "main.o kbd.o command.o display.o insert.o files.o"
TASKS = {
'edit': (objects,
"cc -o edit " + objects),
'main.o': ("main.c defs.h",
"cc -c main.c"),
'kbd.o': ("kbd.c defs.h command.h",
"cc -c kbd.c"),
'command.o': ("command.c defs.h command.h",
"cc -c command.c"),
'display.o': ("display.c defs.h buffer.h",
"cc -c display.c"),
'insert.o': ("insert.c defs.h buffer.h",
"cc -c insert.c"),
'files.o': ("files.c defs.h buffer.h command.h",
"cc -c files.c"),
'clean': ("",
"rm edit " + objects)
}
def task_make2():
for target, rule in TASKS.iteritems():
yield {'name': target,
'targets': [target],
'dependencies': rule[0].split(),
'action': rule[1]}
Version 3
make has build in support for some kind of tasks like compiling a C object.
make example 3:
objects = main.o kbd.o command.o display.o insert.o files.o
edit : $(objects)
cc -o edit $(objects)
main.o : defs.h
kbd.o : defs.h command.h
command.o : defs.h command.h
display.o : defs.h buffer.h
insert.o : defs.h buffer.h
files.o : defs.h buffer.h command.h
.PHONY : clean
clean :
rm edit $(objects)
doit make3:
DEFAULT_TASKS = ['make3:edit']
objects = "main.o kbd.o command.o display.o insert.o files.o"
TASKS = {
'edit': [objects, "cc -o edit " + objects],
'main.o': ["defs.h"],
'kbd.o': ["defs.h command.h"],
'command.o': ["defs.h command.h"],
'display.o': ["defs.h buffer.h"],
'insert.o': ["defs.h buffer.h"],
'files.o': ["defs.h buffer.h command.h"],
'clean': ["", "rm edit " + objects]
}
def task_make3():
# letting doit deduce the commands
for target, rule in TASKS.iteritems():
if target.endswith('.o') and len(rule)==1:
c_file = "%s.c" % target[:-2]
rule[0] += " %s" % c_file
rule.append("cc -c %s" % c_file)
for target, rule in TASKS.iteritems():
yield {'name': target,
'targets': [target],
'dependencies': rule[0].split(),
'action': rule[1]}
When you have python do you really need built-in support for simple string manipulations?
Version 4
there are different styles to specify dependencies…
make example 4:
objects = main.o kbd.o command.o display.o insert.o files.o
edit : $(objects)
cc -o edit $(objects)
$(objects) : defs.h
kbd.o command.o files.o : command.h
display.o insert.o files.o : buffer.h
doit make4:
DEFAULT_TASKS = ['make4:edit']
objects = "main.o kbd.o command.o display.o insert.o files.o"
RULES = {
'edit': [objects, "cc -o edit " + objects],
objects: ["defs.h"],
'kbd.o command.o files.o': ["command.h"],
'display.o insert.o files.o': ["buffer.h"],
}
def task_make4():
# another style of doit
tasks = {}
for targets, rule in RULES.iteritems():
for target in targets.split():
if target not in tasks:
tasks[target] = rule[:]
else:
tasks[target][0] += " %s" % rule[0]
# letting doit deduce the commands
for target, rule in tasks.iteritems():
if target.endswith('.o') and len(rule)==1:
c_file = "%s.c" % target[:-2]
rule[0] += " %s" % c_file
rule.append("cc -c %s" % c_file)
for target, rule in tasks.iteritems():
yield {'name': target,
'targets': [target],
'dependencies': rule[0].split(),
'action': rule[1]}
just more a few lines of code and voila.
Conclusion
The provided syntax is the basic one. You can create your own syntax that is good for your domain on top of it. You can extend it even if there is no API, nothing to import, nothing to subclass :)
In the future I will probably include some helper methods for well-know domains like building C programs. Feel free to contribute ;) I guess doit could support the styles used on make, SCons, XML, JSON, you name it!
Caveats
To be fair there is a small problem. When using generators to create tasks. Tasks will be considered as "sub-tasks" and will have a compound name like "make4:edit". I will create a bug for that. It should be possible to generate tasks and have them not to be considered as "sub-tasks".
