ENDNOTE.AWK and ENDNOTE.PL (version 1.42) A Practical Method for Handling Endnotes in Text Files by Eric Pement Academic writers often need note references in their files. In the editing process, it's nice to move, add, or delete paragraphs with notes without manually renumbering them all. Word processors like Microsoft Word permit users to edit note references, which are automatically renumbered when moved or deleted. Is something like this available to users with text editors like vi, vim, Emacs, Notepad++, or UltraEdit? Yes! With your preferred plain text editor, you can create a source file with embedded note references (unnumbered) and move them around, add them, or delete references without worry. At print time, the source document is run through a reformatting script which numbers the notes, gathers the note references, and moves them to the end of the document where they are printed in order. Endnotes can be separated by double spacing or single spacing, whichever you prefer. A. How to Format Your Document Within the paragraph, reserve a place for the note numbers by one of these placeholders: [#] or [##]. The placeholder is a left square bracket "[", one or two pound signs "#", and a right square bracket "]". The pound signs are converted to 1, 2, 3, etc. when the file is processed. The pound signs only are markers; they do not control how big the number can be. I call them numbering points or Note Points. [[ ##. Technically, you can use 1 to 4 pound signs. The number of pound signs you use has no bearing on how numbers will look when the document is formatted. Numbering begins with 1, unless an option switch is used. ##. Tip: If you expect to have less than 10 endnotes, use just one pound sign, [#]. If you expect to have less than 100, use two: [##]. ]] Below the paragraph with the ##-numbering points, add the references, fully spelled out. The notes should be placed in a Note Block delimited by '[[' and ']]', which may be in any of these formats: (a) WITHIN THE PARAGRAPH. This format works well if your reference is very short.[##] Confirm that the square brackets "[[" and "]]" will appear [[ #. Short, like one-line short. ]] as the first and last characters of the line. (b) AT THE END OF THE PARAGRAPH. This format works well if you like to keep the note documentation immediately below the paragraph which refers to it.[##] You do not need a blank line between the paragraph and the note body. The line below is an example. [[ ##. I use Note Point to refer to the placeholder string within the paragraph, and Note Body to refer to the full documentation associated with each Note Point. Note Blocks can contain multiple Note Bodies. ]] (c) AFTER THE PARAGRAPH. If you tried to reformat the preceding paragraph, the "[[" markers from the note block would very likely be pulled up at the end of the previous sentence. This is bad, because ENDNOTE expects the "[[" and "]]" to be the first and last characters on a line.[##] So to overcome the consequences of a "reformat paragraph" command, you can place the note documentation in a completely separate paragraph. ENDNOTE removes the extra blank line automatically. [[ ##. Technically, ENDNOTE allows spaces or tabs to precede the "[[" marker, and it permits trailing spaces or tabs to follow the "]]" marker, but no other characters may precede or follow these markers. ]] (d) W-A-A-Y AFTER THE PARAGRAPH. Though there is not much call for it, you are free to put the Note Block (wrapped in "[[" and "]]" markers) several paragraphs after the Note Point. This sort of defeats the main idea, which is to keep the Note Points and the Note Body close to each other while editing, but you are not forced to do it that way. ENDNOTE lets you edit however is easiest for you. It is helpful if the Note Block appears directly below the paragraph with the corresponding numbering points, but Note Block can occur almost anywhere (within reason). B. Details on Note Blocks Inside the Note Block---a text block wrapped in "[[" and "]]" markers---each Note Body begins with optional whitespace (spaces or tabs), followed by 1 to 4 pound signs, followed by a period. That period "." is important, because if it is not there, ENDNOTE will not consider this a new item for numbering. It is customary to begin each note reference flush left, but they can be indented.[##] [[ ##. As you can see here. (##) This item, because it has parenthesis mark between the beginning of the line and the first pound sign "#", will be considered part of the preceding note. It will not be interpreted as a separate Note Body for auto-numbering. ##---This item, because it has no period following the pound signs, will be considered part of the preceding note. It will not be interpreted as a separate Note Body for auto-numbering. ]] If you decide to use the in-line form of Note Blocks,[##] the first [[ ##. The in-line forms look like this. ]] space after the "[[" is optional, and your numbered references will STILL appear flush left in the endnote section. However, if there is more than one space after the "[[" marker, the additional space will be copied over into the endnote section. Something to remember, or experiment with. If you have multiple notes inside a Note Block, you can omit or keep blank lines between the notes. In other words, the following forms are treated identically: {{ ##. No blank lines between the notes. ##. None at all. }} {{ ##. Blank lines between the notes in the note block are ignored. ##. Whether you keep them or not, it doesn't matter in the output, }} When the Note Blocks are collected and formatted for printing, one blank line will be inserted between each separate endnote, even if there are no blank lines in the Note Block itself. Conversely, you can have blank lines between individual notes in the Note Block, but no blank lines in the formatted output file, by using the "ssnotes" option when running ENDNOTE. What if your notes or references contain several long paragraphs? Will these be stripped out or tampered with as well? Not to worry. Within each individual note (that is, from the first line to the last line of each note), internal blank lines are kept intact. Only blank lines *between* notes that are affected. Blank lines *within* a multi-paragraph note are not affected. Within a Note Block, ENDNOTE also supports nonprinting comment lines. If a line begins with ".." or "??" or "%" as the first character on the line, that line is not printed. This allows writers to add comments to themselves which do not appear in the output file. [[ .. ------------------------------------------------------- .. In fact, an entire Note Block can consist of nothing .. but nonprinting comments, which makes it handy if you .. want to remind yourself to look up a stray factoid, .. but don't want to see your comment in the output file. .. .. Nonprinting comments may not be placed outside of Note .. Blocks. Er, at least for right now. .. ------------------------------------------------------- ]] C. Details on Note Points Note Points may go directly after a word, like this[##], or there can be a space before the Note Point, like this. [##] [[ ##. Having no space before a Note Point keeps bracketed references from "wrapping" and floating to the left margin if a paragraph is reformatted. ##. Having one space before a Note Point sometimes looks better. But if you're not watching, it will sometimes "wrap" to the left margin of the following line. ]] ENDNOTE works by performing a single pass across the document, formatting and counting as it goes along. Therefore, it doesn't "know" until it reaches the end of the document whether the number of Note Points equals the number of Note Bodies for the "Endnotes" section. If there is a mismatch between the total number of Note Points and the total number of Note Bodies, ENDNOTE issues an explanatory error message and quits. (This action can changed by an option switch.) If there is no mismatch, ENDNOTE prints this section header: --------- ENDNOTES: followed by the collected series of Note Bodies, which have been automatically numbered. The "[[" and "]]" markers are discarded. D. Option switches The following options are available for ENDNOTE. The syntax differs depending on whether you are using awk or perl to run ENDNOTE. The switch names and functions, however, are the same. By default, Note Points in the text are indicated by 1 to 4 pound signs in square brackets. You may use something simpler, such as '*' (asterisk), by the switch `alt_np` (for "alternate note point string"). The characters will be interpreted as literal strings, not as regular expressions or metacharacters. By default, one blank line is automatically inserted after each Note Body (double-spacing between notes, which is not the same as double-spacing each note). If a switch is passed for `ssnotes` (single-spaced notes), the blank line is omitted. By default, note numbering always begins with 1. If you split your document into multiple part and want the notes for Part 2 to continue where Part 1 ended, ENDNOTE has a switch named `start` to begin numbering at any specified integer. By default, ENDNOTE halts if the number of Note Bodies does not correspond with the number of Note Points. A switch named `ignore_errors` causes ENDNOTE to ignore mismatched notes in the body and the endnote section, printing the notes "as is" without halting. This switch can be helpful if you need to print a working draft and you don't care about mismatched notes. E. Numbering simple lists Suppose you want to number items in a list, where the first character of each line is a hyphen. For example, suppose "mylist" contains: - milk - bread Either whole wheat or cracked rye - butter - eggs Extra-large size A standard Unix command like "nl" (number lines) or "cat -n" will not work, as this will number every line. But ENDNOTE can easily replace each hyphen with an incrementing number. Use the "alt_np" switch to change the Note Point pattern to (space, hyphen, space) to avoid matching hyphens in the middle of a word, and set the "ignore_errors" switch as well. In awk: (assuming that "endnote.awk" is in the $AWKLIB path) awk -v alt_np=" - " -v ignore_errors=1 -f endnote.awk mylist In perl: perl -s ../path/to/endnote.pl -alt_np=" - " -ignore_errors mylist If you need to, you can also add the "start" switch to begin numbering the list with a number other than 1. F. Running ENDNOTE with Awk You must install awk or perl to run ENDNOTE. If you are *only* interested in using ENDNOTE and if you have neither tool, Awk is far easier to install and understand than Perl. Awk is installed by putting one or a few files somewhere on the PATH where Windows can find it. ENDNOTE will run under either GNU awk (gawk.exe) or Michael Brennan's awk (mawk.exe).[##] If you have neither, GNU awk (gawk) is a better choice than mawk because gawk has better internal help, gives helpful error messages, and has ongoing development. [[ ##. Recommended sites for awk and mawk, precompiled for Windows: http://sourceforge.net/projects/ezwinports/files/ - latest GNU awk http://www.klabaster.com/freeware.htm - both gawk and mawk here http://www.delorie.com/djgpp/dl/ofc/dlfiles.cgi/current/v2gnu/ - gwk http://gnuwin32.sourceforge.net/packages.html - not up to date ]] ENDNOTE for awk is available from http://www.pement.org/awk.htm The syntax for the awk version of ENDNOTE is: awk [-options] -f endnote.awk source_file >output_file Switch placement. Note that if any options are used, they must all occur before the -f switch. Each option requires a separate -v switch. Options: -v alt_np='str' # use 'str' (literal) as an alternate note point -v ignore_errors=1 # ignore mismatched numbering in endnotes -v ssnotes=1 # omit blank line between notes (default: 1 line) -v start=n # start numbering at 'n' instead of 1 Example: to use the first two options, where the alternate Note Point string will be a single asterisk, use this syntax in Unix: awk -v alt_np='*' -v ignore_errors=1 -f endnote.awk infile >outfile In DOS/Windows, replace the single quotes with double quotes: awk -v alt_np="*" -v ignore_errors=1 -f endnote.awk infile >outfile G. Running ENDNOTE with Perl Perl is a high-powered programming and "glue" language, and if you are only interested in getting ENDNOTE to run, your best bet is GNU awk. However, if you have an obsolete version of Perl or if you want to install a fully running version of Perl, then it's easy to find.[##] If you're running a Unix or Linux system, then perl is almost certainly already installed. [[ ##. Recommended sites for perl: http://www.strawberryperl.com - for Windows 32 http://www.activestate.com/activeperl/ - free and commercial versions http://www.perl.org/get.html - for Unix/Linux, Mac OS X, Windows ]] ENDNOTE for perl is available from http://www.pement.org/perl.htm Additional documentation for ENDNOTE for perl is available by running the command "perldoc [/your/path/to/]endnote.pl" from a shell prompt. The syntax for the perl version of ENDNOTE is: perl [-s] endnote.pl [-options] source.txt >output.txt Switch placement. Note that if any options are used, the -s must also be present. The -s must come *before* the script name ("endnote.pl"), but the options (prefixed with a hyphen) must come *after* the script name. Options: -alt_np='str' # use 'str' (literal) as an alternate note point -ignore_errors # ignore mismatched numbering in endnotes -ssnotes # omit blank line between notes (default: 1 line) -start='n' # start numbering at 'n' instead of 1 H. Acknowledgements Finally, I must give credit where credit is due. Key ideas for ENDNOTE were adopted from "wsNOTE" by Eric Meyer, the author of VDE (a powerful DOS text editor, now freeware).[##] wsNOTE was a DOS utility for handling both footnotes and endnotes in WordStar files, at a time when WordStar supported neither.[##] wsNOTE generated WordStar-specific output; I have adopted his system to perform a similar task with plain ASCII files. [[ ##. VDE can be obtained from http://sites.google.com/site/vdeeditor/Home/vde-files ##. Documentation for wsNOTE is currently at http://sites.google.com/site/vdeeditor/Home/vde-files/wsnote-manual ]] -- The ENDNOTE perl and awk scripts were written by Eric Pement in 2002. This document was updated in November 2017.