Using MultiMarkdown with Scrivener

Introduction

This document assumes a basic familiarity with Scrivener, and with MultiMarkdown. It is designed to help you combine the two programs into a system to help you with the writing process, as well as the printing and publishing process.

It will guide you through the process of setting up a document that will be compatible with exporting to MultiMarkdown. The reasons for doing this could be numerous, but primarily:

• You want to have a clean, minimally distracting syntax for typing your document, without having to rely on RTF or WYSIWYG style conventions.

• You want to be able to produce professional-quality output from your document using LaTeX.

• You want flexibility in your document - you don’t want to be tied into a single document type. With MultiMarkdown you can convert your document to XHTML, RTF, LaTeX, or Word .doc, to name a few. With the proper knowledge, many other formats are possible.

MultiMarkdown will let you focus your efforts on writing, and let the computer focus on formatting. Not only does this remove some of the issues that distract you from writing, but it minimizes errors that provide inconsistent formatting when done by hand.

If this doesn’t entirely make sense, I suggest you check out the MultiMarkdown Sample Document. Be sure to download the zipfile and check out the pdf version, in addition to the actual MultiMarkdown source document.

Note: The sample document requires Scrivener version 1.05b or later.

How To Use This Document

First, you can read this document within the Scrivener application. This lets you experiment with the concepts it describes as you go.

More importantly, this document is an example of a MultiMarkdown formatted file in itself. So you can do the same things with it that you can do with any other MultiMarkdown file, including converting it into a web page, RTF file, LaTeX file, or PDF.

This document will focus on the implications of using MultiMarkdown within Scrivener, not on the specifics of the MultiMarkdown syntax itself. For details on the MultiMarkdown syntax, the MultiMarkdown program, or other MultiMarkdown utilities, I refer you to the MultiMarkdown web site.

What is MultiMarkdown?

MultiMarkdown is a variant of Markdown, which was written by John Gruber:

Markdown is a text-to-HTML conversion tool for web writers. Markdown allows you to write using an easy-to-read, easy-to-write plain text format, then convert it to structurally valid XHTML (or HTML). (Gruber)

Gruber’s philosophy regarding Markdown is fairly straightforward:

The overriding design goal for Markdown’s formatting syntax is to make it as readable as possible. The idea is that a Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions. While Markdown’s syntax has been influenced by several existing text-to-HTML filters, the single biggest source of inspiration for Markdown’s syntax is the format of plain text email.

Creating a MultiMarkdown Document in Scrivener

Preferences

There are certain settings you may find useful when working with MultiMarkdown documents in Scrivener:

• I recommend adjusting several settings under “Text Editing”, specifically using a monospace font and changing the formatting ruler to remove the leading indentation for new paragraphs. Leading spaces at the beginning of a line have meaning in MultiMarkdown, and this could create confusion when trying to track down “gremlins”.

• It’s up to you whether you use the built-in smart typography in Scrivener, or rely on SmartyPants, which is used by MultiMarkdown. This document uses SmartyPants, so that it is more likely to work as expected when the raw MultiMarkdown source copied to other applications or systems.

• Since paragraphs need to have an empty line between them to be recognized by the MultiMarkdown exporter, you might find it helpful to adjust Scrivener’s automatic separator settings to ensure that an empty line is placed between combined sections of your Draft. In the application preferences, under General, you’ll find a section indicating how separators are handled. At the very least, “Merged Documents” should have a double newline inserted. This way, when two documents are merged together, you needn’t worry about two paragraphs getting run together in MultiMarkdown.

Organizing your document

The structure that you use for your documents in the “Draft” section have more meaning than they do in a regular Scrivener document.

MultiMarkdown was originally built around XHTML, and uses the header levels to indicate a sort of “hierarchy” to the sections in a document. For instance, <h1> might represent a chapter, and <h2> might be a section within a chapter.

The hierarchy that you use in the Draft portion of the binder is translated into this structure. For instance, if your Draft looks like this document:

Draft
    Introduction
    How To Use This Document
    Creating a MultiMarkdown Document in Scrivener
        Preferences
        (etc.)

then this implies that you have several top level divisions (parts or chapters in a memoir book, depending on settings.) The “Creating a MultiMarkdown…” division has several sub-divisions (starting with “Preferences”.) This means that you can use Scrivener to easily rearrange your document as you write it. If you expand a section, and decide it should be it’s own chapter, no problem. Just drag it to the top level in the outline!

In order for MultiMarkdown to understand your hierarchy, you must enable exporting of Titles for both Documents and Groups within Scrivener. Scrivener will then add the title of each Group or Document in the appropriate MultiMarkdown format

As always, when exporting your draft, you can choose which sections actually get exported.

This said, there is nothing to stop you from manually using the MultiMarkdown syntax to enter headers within your documents themselves, such as:

#### This is a fourth level division ####

Of course, this will prevent you from using the outline view (and other Scrivener features) from reorganizing your document easily. If you use this approach, you should not enable the exporting of titles for Documents and Groups.

An import thing to note is that there is no such thing as a “correct” or “incorrect” structure in a MultiMarkdown document or an XHTML file. In other words, it is perfectly valid to have an “h5” section immediately following an “h1” section. Most of the time, however, you will likely use the outline view and have what is a more “typical” structure where each section is one level above it’s parent.

Metadata

An important part of using MultiMarkdown to its fullest extent is the use of metadata. In a raw MultiMarkdown document, the metadata appears at the top, as specified on the MultiMarkdown wiki page:

Title:              Scrivener and MultiMarkdown
Author:             Fletcher T. Penney
Keywords:           Scrivener, MultiMarkdown, XHTML, LaTeX, PDF, 
                    TextMate, writing, publishing, page layout, 
                    RTF, typography
Language:           English
LaTeX XSLT:          memoir.xslt
Base Header Level:  2

Scrivener uses a different place to specify metadata — under the File menu, select “MultiMarkdown Settings…”.

The MultiMarkdown page describes these keys in more detail, but the key things for Scrivener are:

• Title, Author, Subtitle, and Dateq are all self-explanatory. If there is no date, then today’s date is used.

• RTF and PDF files support keywords, which can be entered as a comma separated list in the Keywords metadata field.

• Language tells MultiMarkdown which language-specific version of SmartyPants to use. If you prefer, you can set this in the Typography section of Scrivener’s preferences. Otherwise, the default is English.

• LaTeX XSLT tells MultiMarkdown which XSLT to use for processing the document to LaTeX, should you choose that option. If not specified, the default is memoir.xslt

• Base Header Level is used to start numbering headers at the specified level. Memoir has the ability to specify “Parts”, and many times you want the top level of your document to be chapters, not parts. In that case, use a Base Header Level of 2 to indicate that your top level of organization is actually the second level as far as memoir is concerned. If you export your document using different XSLT files, you may have to change this field on a case by case basis.

Warning: Be sure that there are no blank lines in your metadata (or an extra return at the end of a metadata item). A blank line signals the end of metadata, so this could cause MultiMarkdown to stop processing your metadata as metadata, and instead treat it as the beginning of your content… If you are having trouble, export to a plain MultiMarkdown text file and ensure that you are getting the expected result.

Import Features

Importing MultiMarkdown Files

Scrivener has the ability to import MultiMarkdown formatted text files and attempts to retain the hierarchy contained in the file. In other words, your file will be imported as multiple Documents in the Draft outline.

There are a few limitations of the MultiMarkdown import process:

• Metadata within the document is not imported into the Scrivener MultiMarkdown Settings. This is necessary since you may import multiple MultiMarkdown documents into a single Scrivener project, and it could be inconvenient for your metadata settings to be overwritten each time a file was imported. The information is retained, however. You may notice a “Meta-Data” document at the top of your newly imported hierarchy that contains the metadata.

• Any text that occurs before the first heading will be included in the “Meta-Data” document described above. You can then choose what to do with this text.

• Scrivener is somewhat limited in its ability to import MultiMarkdown files with complicated hierarchy structures. Things should work fine if your “child” headers are one level higher than their “parent”. But if you have a level 4 header that is a direct descendent of a level 2 header, Scrivener will reinterpret that section as a level 3 header, and make it the direct child of the level 2 parent. If you desire, you can manually make changes to recreate your original structure.

• Scrivener cannot currently (as of 1.11) import a MultiMarkdown file that doesn’t contain any headers. Hopefully this will be fixed shortly.

Export Features

How to Export with MultiMarkdown

The key things to keep in mind when exporting to MultiMarkdown include:

• You probably want to enable the inclusion of titles and text for both Text and Groups, if you want the hierarchy of your Draft to be recreated. On the other hand, if you have manually entered headers for your document divisions, then you should disable titles for the various sections.

• If you enable the titles as above, remember that the MultiMarkdown document will be sensitive to the hierarchy level of your files and folders within Scrivener.

• Decide whether you wish to export Scrivener’s Footnotes and Annotations to your MultiMarkdown documents.

Scrivener Footnotes and Annotations

Scrivener can automatically convert footnotes from the Scrivener notations (i.e. the colored highlighted appearance) to the MultiMarkdown syntax for footnotes so that they can be processed in the usual manner.

If you are going to be making extensive use of footnotes, however, I might suggest manually using the MultiMarkdown syntax to keep things simple when debugging. But this is entirely up to you.

Similarly, Scrivener annotations can be converted to a new syntax that the MultiMarkdown to LaTeX XSLT files will then convert to colored text (it even preserves the color you select in Scrivener!) There is no real analogy to this in the MultiMarkdown syntax, so if you need to include annotations, this is your only option (or you could manually enter the syntax… Basically it consists of using <span style="color:#FFFFFF"> and </span> tags around your annotation, substituting the color of your choice.

WARNING: span level tags cannot include block level tags, so worlds may implode if you try to include multiple paragraphs within an annotation. Consider yourself warned. This limitation may or may not be improved significantly in the future. In short, if you have trouble with LaTeX, try disabling annotations.

Bold and Italics

Scrivener has some basic features to try and convert Bold and Italics to MultiMarkdown formatting, using the “Bold and Italics to MultiMarkdown Syntax” menu item. Things that are formatted as bold or italics in the RTF documents inside Scrivener will converted to the appropriate MultiMarkdown syntax when this command is chosen.

Consider yourself warned — this is another convenience feature, and will fail in certain situations. It is up to you to proof your final MultiMarkdown output to be sure that it accurately represents what you intended. I recommend that you stick to the usual MultiMarkdown syntax, rather than the RTF formatting. At the very least, I suggest manually verify that everything is converted properly before relying on this command for any critical purpose.

Another option is to periodically use the Text:Convert:Bold and Italics to MultiMarkdown Syntax menu command. This converts any bold and italics in your current document into the MultiMarkdown syntax for you, so that you can proof read the final results immediately. There is no command to perform the reverse operation, so this may or may not be a suitable solution for you.

MultiMarkdown (Text)

To export your draft as a plain text file using the MultiMarkdown syntax, simply choose the “MultiMarkdown” export format in the pop-up menu in the lower right corner of the Export Draft screen.

Choose this format when you wish to save a text copy of your document using the MultiMarkdown syntax. This can be useful for including in an email, pasting into a weblog or wiki (if it supports MultiMarkdown!), or anywhere else that plain text is needed. You can also use any other tool that processes MultiMarkdown to further process this file.

MultiMarkdown To XHTML

Use this option when you want to export a fully formatted web page, Scrivener includes some basic CSS formatting information, but you can override this with the XHTML Header and CSS metadata keys if you desire.

Note: If you are using the math syntax stuff, then you should use the “.xhtml” file extension to help certain browsers (e.g. Firefox) properly recognize the file. Remember, the math syntax uses MathML, which is not fully supported. But it does convert to LaTeX nicely. ;)

MultiMarkdown To RTF

This option creates an RTF from the MultiMarkdown formatted source. This is significantly different from Scrivener’s default RTF exporting feature. Scrivener’s default approach is to use the built in rulers and formatting information to create an RTF. The MultiMarkdown approach first creates an XHTML using standard XHTML tags, and then converts that file into RTF.

Basically, if you have been using the MultiMarkdown syntax to format your document, you will want to use the MultiMarkdown RTF Export option. Otherwise, you may prefer the default version.

MultiMarkdown To LaTeX

This is where MultiMarkdown really shines. You can take your plain text source, with minimal markup, and have the computer automatically format a LaTeX file for you that can be used by the LaTeX processing tool of your choice (TeXShop is quite nice).

By using MultiMarkdown to create your source document, you can create high-quality PDF output without ever having to learn the LaTeX language (which is not horribly complex, but MultiMarkdown is much easier!)

If you are looking to install LaTeX on your Mac, the MacTeX distribution is easy to install, includes TeXShop, and all the packages required by default for MultiMarkdown. There are other equally good packages out there, but this one is easy to install and does seem to have some features that make it easy to use Mac OS X fonts in your PDF output. But this is getting outside the scope of this document…

To use this, export your document to the LaTeX format, and then run pdflatex to create a pdf. (Note: you may need to run it multiple times to generate the table of contents, bibliography, etc. Again, this is outside the scope of this document.

Use the MultiMarkdown web site to learn more about PDF export, and how to use different XSLT files to format your output differently (8.5 x 11, 6 x 9, memoir, article, etc.)

Tip: If you end up with an empty file when you export to LaTeX, then try exporting to the MultiMarkdown text format, and using one of the “regular” MultiMarkdown approaches to see what the error is. Scrivener doesn’t offer any indication that there was an error.

Warning: I have had good luck using the MacTeX install, but will warn you that the memoir package there is a bit out of date. Try installing the newest version if you have trouble.

Advanced Features

Including Images

You can use your Scrivener Binder to store images that will be exported alongside the MultiMarkdown output.

To use this feature:

1. Add the desired images to your Scrivener Binder (in Research, or wherever)

2. When creating the image definition in your document in the MultiMarkdown syntax, use a “Scrivener Link” instead of the filename. This is created by the “Text:Scrivener Link” menu.

3. There is no step 3…

When you export your document, if Scrivener detects that you have used images, it will export your document to a folder, and include the appropriate images in that folder. It may rename some of the files under certain circumstances. For example, if you had multiple images named “image.png” they would be exported to unique filenames. If you use the Scrivener link to represent the filename in the image link, then it will be adjusted automatically.

Self-Publishing

Since many users of Scrivener will be writing towards the goal of publishing a book, and since many people never get their book published, it might be useful to be able to publish books on your own.

It’s quite easy to use the Scrivener, MultiMarkdown, LaTeX, PDF workflow to create a PDF suitable for uploading to one of the online, print-on-demand publishers. There is more information about this on the MultiMarkdown website.

I have successfully printed a hardcover book from Lulu.com. I used a customized version of the 6x9book XSLT file (set “XSLT File” to “6x9book.xslt” in metadata) to print a book with the hardcover (with dust jacket) binding. It turned out really well, though I did make a few adjustments to the margins on this XSLT file after getting the book back. And the pdf did just fine without any “post-processing”. I used latexmk.pl to process it, uploaded the pdf (and a separate pdf for the cover), and waited a couple of weeks. It was actually quite easy.

Note: I don’t have any connection with Lulu.com, other than having satisfactorily purchased a single book from them. I don’t necessarily recommend them over any other online publisher. Also, I make no guarantees as to the success rate that you will have with these steps. It worked for me, and it should work for you. The main thing is to double check everything before purchasing a book, and I also recommend purchasing a copy yourself before you make the book available to anyone else.

If you are interested in printing your own book, here are the steps I used:

1. Format your document using MultiMarkdown using the tool of your choice.

2. Choose an XSLT style that is appropriate to the format you want to print. I wanted a 6 x 9 inch hardcover book, and used a derivative of the 6x9book.xslt stylesheet that had a few customizations needed for my specific document.

3. Convert your document to LaTeX using the selected stylesheet.

4. Use the tool of your choice to convert from LaTeX to pdf - I like TeXShop, or the command line tool latexmk. Make sure to run the steps necessary for a bibliography, index, table of contents, etc, as appropriate (beyond the scope of this document…)

5. Carefully proofread your PDF — you don’t want to pay full price for a “rough draft” edition if you don’t have to. Pay attention to formatting as well as content. And it’s worth paying attention to the errors that pdflatex generates — it might help you pick up subtle errors you would have otherwise missed.

6. Be careful about customizing fonts - if you create an XSLT stylesheet that uses other fonts, you need to verify that the fonts are included within the pdf. (Beyond the scope of this document)

7. Upload your pdf file to your publisher of choice, fill in the data they require, and upload a cover image (also beyond the scope of this document.)

TextMate Integration

Some people find that certain shortcuts and “auto-formatting” is useful in their text editor. For example, Scrivener has the Screenplay mode to provide some automatic formatting.

If you’re familiar with TextMate, I’ll take this opportunity to remind you that TextMate can be configured to work along with Cocoa programs as an external text editor. See the TextMate web site for more details, but basically you can hit a keystroke to take the text from the document you are working on, and edit it in TextMate. Combined with my MultiMarkdown Bundle for TextMate, this can be an easy way to speed up the creation of your documents within Scrivener.

This isn’t for everybody, but for those who like the power and flexibility of TextMate, it can be quite useful. In particular:

• the MultiMarkdown Bundle for TextMate makes it very easy to create well laid-out MultiMarkdown tables, lists, and other specially formatted sections

• it also includes support for auto-completing links, images, etc. This might not help if your link definitions are included in a document other than the one you are working on, but it can still help

Several cautions, however:

• Using this feature seems to screw up Scrivener’s undo stack. Hopefully this will be fixed in future versions of TextMate, but I make no promises. Save snapshots of your work in Scrivener and make backups regularly.

• Additionally, converting to TextMate changes RTF to plain text, causing you to lose any bold, italic, footnote, or annotations that you have created using Scriveners RTF based tools. The same formatting, when created using MultiMarkdown syntax, will convert to and from TextMate without a problem. Use with caution, though you can use the “Bold and Italics to MultiMarkdown Syntax” command prior to editing in TextMate. This would also make it easier to spot errors.

BibDesk Integration

I previously modified the BibDesk Input Manager to allow for auto-completing BibTeX citations within a MultiMarkdown document.

Basically, you start to type a citation in the usual MultiMarkdown format:

[#Smith:2000][]

Once you have the [# typed, you can use the OS X autocomplete feature (the Esc key) to find matching citations in your current BibDesk file. See the BibDesk site for more information (and for those who are interested, BibDesk is included in the MacTeX download.)

Of note, there is no way to store your BibDesk file within your Scrivener project, but you can include a link to it in a document in your Research section. Then you can simply double-click this link to open your BibDesk file whenever you need it.

Note: As of Leopard, the Input Manager support for BibDesk has been discontinued. An alternative approach is to use BibDesk’s Template feature to allow you to drag and drop your citations into your MultiMarkdown document. These will be uploaded to the XSLT Files page when they’re available, or you can create your own.

Updating MultiMarkdown

Scrivener comes with a built-in copy of MultiMarkdown. By the time you download Scrivener, this may or may not be the newest version. In addition to this built-in copy, Scrivener will look for a “Common” MultiMarkdown installation in ~/Library/Application Support/MultiMarkdown. You can download an installer that will put the latest version of MultiMarkdown in this location, and you can add additional XSLT files to this folder in order to increase your reformatting options.

Basically, this makes it possible to upgrade your MultiMarkdown installation separately from your Scrivener application. And if you download an updated version of Scrivener, you won’t overwrite your customizations to your MultiMarkdown installation.

Scrivener and the XHTML Header Metadata

By default, Scrivener adds some style information to the metadata that is exported. Specifically, it adds XHTML Header metadata that replicates some of the RTF styling of your document when converted to XHTML.

Some people don’t like this, and prefer their own styling, or none at all. To older MMD users, it can even be confusing.

In any event, it can be disabled. Simply included an empty XHTML Header metadata value in your document. When Scrivener exports the document as XHTML, this empty metadata will over-ride whatever Scrivener adds, and you’ll never know that it happened.

FAQ

• When I try to export as LaTeX, I get an empty document, or a document that won’t process. What’s wrong?

  Almost every time this occurs, there is a problem the document, not with Scrivener or MultiMarkdown. The trick is to troubleshoot. The first step is to export your document as MultiMarkdown text, not as LaTeX, XHTML, etc. This will allow you to open the text file in a text editor, such as TextMate, or even TextEdit, and look for problems manually. You can also use Terminal to manually process the file by running multimarkdown2XHTML.pl, which is located in the bin directory inside the MultiMarkdown folder.

  The first places to look are:

  1. Check that your metadata doesn’t have extra spaces at the end of the lines.

  2. Check that there is only one section of metadata - remember that a blank line ends metadata and starts your document.

  3. Look for “illegal” characters that may have been pasted accidentally. These can be tough to find. Often, the best way is to divide your document in half, and process each half separately. Find the half that doesn’t work, and then divide it in half. Repeat this process until you get to a manageable chunk of text, and then scan it very carefully for illegal characters. Using a text editor with the ability to show invisible characters will be quite helpful here.

• When I export to XHTML or RTF, everything shows up in Courier.

  Scrivener adds some XHTML metadata by default. There is not a setting to disable it, but you can override it by including your own empty metadata. See Scrivener and the XHTML Header Metadata for details.

• When I try to import a MultiMarkdown text file into Scrivener, I get a message that it’s not a valid MultiMarkdown file.

  Scrivener’s current definition of what constitutes a valid MMD file is actually a bit narrow (as of version 1.11). It requires at least one header (not necessarily level 1), and anything before the header is treated as metadata. Keith and I are working to fix this, but it might take a little while. So for now, if you are importing a MultiMarkdown document, be sure that the body of the document starts with a header, and check the imported Meta-Data section carefully to ensure that no text was inadvertently snipped.

• How do I export to MultiMarkdown, and then reimport my file so that I can…?

  There are a variety of reasons to want to “round-trip” data from Scrivener to plain text, and back again. It allows you to share with users who don’t have Scrivener, or who are working on PC’s or *nix machines. You can use the plain text files more readily with version control systems. Or whatever.

  In order to export properly, follow the instructions in Export Features.

  When you reimport, things won’t look quite the same. There will be a new File called Meta-Data. This contains the metadata from the MultiMarkdown document. If it has not been changed, you can delete this file (but not it’s children!) There is no automatic way to synchronize the metadata. You’ll have to do this by hand.

  You’ll want to rearrange the imported hierarchy so that it’s back to being a top-level child of Drafts.