Texmake: a makefile generate for latex projects

texmake is a makefile generator, much like cmake. In fact, it originally started with me looking for ways to make cmake work with latex documents. After a bit of work, I decided the system was too much designed around the c/c++ build process, so I started working on some tools to simplify my life.

Keeping it simple

texmake tries to eliminate as much work as possible from the document writer. It will try to figure out and resolve dependencies, including graphics files which require conversion from their source format. It also allows for out-of- source builds, so that the source directory doesn't get all cluttered with the intermediate files that latex generates.

Example

Consider this scenario: We have a very long complicated report to write (i.e. software manual, end-of-project report, thesis, how-to-book). We want to generate both a full version, and a simplified quick-start kind of version. The quick start version will contain a subset of the chapters of the full version. The quick start will be published in PDF and HTML (i.e. web- friendly). The long version will probably be a large document and we don't really want it to be browsed on-line, but it is likely to be printed, so we'll put it in pdf and dvi format, as well as epub for people who have e-readers.

[caption id="attachment_602" align="aligncenter" width="360" caption="Example Project Structure"][/caption]

In the process of making this document, we've generated many image files. Some of them are hand drawn SVGs. Some of them are generated tikz images. Some of them are diagrams drawn in DIA or plots in GNU Plot. Some of these figures are shown in multiple different chapters (because the author does not want to just refer the user back to a previous page, which is unnecessary in an electronic format, but may be more meaningful in a print format).

Furthermore, we have some things that need to only be included in each version. For instance each version should include some kind of header which tells the reader where he can find the other versions online, or where he can order them from.

Now, we can go maintain a makefile structure that manages all of this quite easily, but we will have to build it by hand. Every time we add a new chapter or image or output format, we have to go add a line to a makefile somewhere. Wouldn't it be nice if all of that work would just happen?

Features

here are a list of features that I have implemented or am working on

1. don't rebuild the document unless it needs it
2. multiple documents in the same project, potentially sharing various pieces of content
3. monitor documents dependency on package, style, and class files so that document is rebuilt if these are updated
4. monitor dependencies on all included files, so that if any included file is updated, the document is rebuilt
5. only run bibtex if needed
   1. one of the database files is updated
   2. there are unresolved citations in the document
6. rerun latex until it stabilizes
7. discover and automatically convert graphics source files (like svg) to the kind (pdf)latex(ml) understands (pdf,eps,png)
8. out of source builds so that sources can be in version control without complicated ignore rules
9. caches values of environment variables and binary locations so that initial environment does not have to be manually set-up each time the document is built
10. colorize output to make it clear where and when things get messed up

Usage

texmake relies on the presence of a texmakefile in the directory of the document you want to build. The texmakefile simply lists source files (.tex documents) and the output files that should be built from them. Multiple output files can be built from the same input file, so that the same latex source can be used to generate pdf, dvi, and xhtml versions (so-called single source builds).

texmake init is called from the build directory, accepting a single parameter being the directory that contains the root texmakefile. texmake resolves the absolute path to all the tools it needs (latex, pdflatex, bibtex, latexml, kpsewhich, …), and caches those locations. (TODO: also cache environment variables for latex and search paths for kpsewhich). It generates a makefile in that directory which contains rules for all of the output files registered, and creates directories, if they're missing, in which the output files will be located.

texmake relies on GNU make to determine when documents need to be rebuilt. This makefile includes several other makefiles (if they exist) containing detailed dependency lists for each of the output files. If the output files have never been built, these included files do not exist, but that's OK because it is already clear to make that it needs to build the output files.

The output files are built with texmake build. The texmake builder runs (pdf)latex(ml) with all the necessary switches to make sure that the source directory and output directory are searched for required files. The builder also scans the output of latex to build a list of all the files that latex loads (for the dependency graph) as well as all the files that are missing. If the missing files are graphics files, then it generates make rules for how to build those graphics files from similarly graphics files in the source directory (it is assumed the source file has the same name, but a different extension). This way, an svg source file can be used to generate pdf,eps,or png images depending on if the document being built is dvi,pdf, or xhtml (respectively). If the file in the source directory is already compatible with the latex that is being used, then no conversion is necessary. If the missing file is a bibliography file, then bibtex is run to generate the bibliography file. The builder also scans the output of bibtex to add bibliography database files (.bib files) to the dependency graph. Bibtex is also run if there are missing citations reported by latex (this is because bibtex will not include bibliography entries that are not cited in the latex source, so if a new citation is added, bibtex will need to be rerun). The texmake builder also scans the output of latex for rerun suggestions. It will rerun latex if it had to run bibtex, or if latex itself suggested that it be rerun (to get cross references straight, for example).

Major TODO items

1. cache environment variables in init
2. use diff to determine if TOC's get updated (they will get touched by latex so make will always think they're new) to determine of a rerun is necessary for TOCs
3. add image generation rules for tikz
4. consider a more modular design for making intermediates (i.e. run a program, like matlab, to generate intermediates)

Code

The texmake code is in my project tracker.