fletcherpenney.net

MMD-CMS has support for “make”

2010-02-26T19:55:36-04:00

MMD-CMS has support for “make”

02/26/2010 19:55:36

I’ve been quite happy with using MultiMarkdown-CMS to run my web site. The main frustration has been when I make a change that requires rebuilding the entire site. I had to use a bunch of commands to run MultiMarkdown over various directories and subdirectories in order to rebuild the entire site, and then remember to rebuild the vector map index so that the similar pages feature works.

Inspired by the Makefile support in Dr. Drang’s personal wiki project, I decided it was time to learn a little make myself…. After a little fiddling around, I created a Makefile that allows me to rebuild my entire site with one simple command:

make

If there are any text files that do not have a corresponding html file, make runs the proper mmd2web.pl command to generate the missing page(s). Then it runs the map_my_site command to rebuild the vector map. The nice thing is that if there are no new pages, the vector map is left alone.

This means that I can run the make command without fear of unnecessarily rebuilding anything. If any files have changed, they get rebuilt. If anything is missing, it gets rebuilt. Otherwise, leave it alone.

A bonus feature is that I can remove all of the html files with a simple:

make clean

Not sure why I would want to do this, but it’s neat that I can.

To use the make command, you will need to change the path to mmd2web.pl in Makefile to match your system. But that should be pretty straightforward.

New OPML Features added to MultiMarkdown

2010-02-17T10:22:44-04:00

New OPML Features added to MultiMarkdown

02/17/2010 10:22:44

For anyone using MultiMarkdown with OmniOutliner or other OPML applications, I have added additional OPML support to MultiMarkdown.

Specifically it is now possible to “round trip” a document from a plain text MMD file, to an OPML file containing MMD markup, and back to a plain text file without corrupting the structure, or the content.

mmd2OPML.pl is used to convert a plain text file into an OPML file
OPML2mmd.pl is used to convert an OPML file into a plain text file

The layout of the OPML file is consistent with the structure expected by my OmniOutliner export plugin. This means that the OPML file can be imported into OmniOutliner, and then exported back as an OPML, or exported directly within OmniOutliner into a MultiMarkdown text file (with or without attachments). Which brings up the point that the attachment support is only for OmniOutliner (at least for now).

I’m not sure what other applications are out there that would be useful for editing MultiMarkdown in an OPML format, but these new tools make it possible. Alternatively, it may be helpful to temporarily convert a MultiMarkdown text file into OPML if only to help reorganize the document quickly.

One limitation of this approach is that if you are using a “non-standard” numbering scheme for your header levels you may find that the structure of the resulting documents are not what you expect. For example, the “children” of an h2 level section, will by definition be h3. With plain text files, you can specify whatever level you like — in other words the level of the header (h2, h3) had no intrinsic meaning. In an OPML document, they correlate directly to the levels of the outline, which obviously cannot skip a “generation”. In my mind, however, this is a feature, not a bug, and will probably not be changed.

Additionally, I just discovered that not every OPML tool supports a “_notes” field. OmniOutliner allows you to have a “title” for each piece of the outline, and a separate notes field that holds the content. It appears that there is no standard approach to this. So some OPML apps will simply import the hierarchy of section titles, but no content…. If anyone else uses these features, let me know if there is a workaround for the app you use… Unfortunately, it appears that these features may only be useful for OmniOutliner users…

MultiMarkdown CMS now incorporates “LESS”

2010-02-13T10:34:37-04:00

MultiMarkdown CMS now incorporates “LESS”

02/13/2010 10:34:37

I recently re-engineered the CSS and div structure of my layout to work better. Specifically, I used some ideas from Matthew James Taylor to allow the content to be described before the sidebar, and for the sidebar to be switched from right to left with a single change. This required adding one more div to my overall structure, and reworking the math on the CSS to achieve the proper proportions.

It occurred to me that because of the complicated setup, it would require a calculator and lots of debugging any time I wanted to change the width of a column, or the spacing between them.

Which then brought up the idea of using LESS. LESS is a way of creating CSS that allows a bit more flexibility and shortcuts. It also allows the use of variables and “math”. So I could set up a list of variables to specify the desired width of the main content, the sidebar, and the margins for each. LESS then calculates the required settings, and generates the CSS. So a few quick changes can then reengineer the entire layout, and I don’t have to break out a calculator. It took some time debugging the changes to get everything worked out, but I think it works properly now.

If you don’t want to use LESS, you can look at the LESS source, do the calculations yourself, and then manually update the css. Or you can stick with the default proportions. But if you use CSS much, it’s worth at least taking a look at LESS.

Note: The only problem I have encountered with LESS so far is that it doesn’t handle the @media command, so I had to use a separate css file for print media.

Several Updates to MultiMarkdown CMS

2010-02-05T13:50:01-04:00

Several Updates to MultiMarkdown CMS

02/05/2010 13:50:01

There have been many changes to MMD CMS throughout the week. I rewrote much of the code to be more uniform, fixed a fair number of bugs, and streamlined a few things.

The biggest change is in the requirements for how it is hosted. There are now two branches - the regular branch, and the advanced branch. The differences in purpose and configuration are detailed on the project page, and in the README.

How to use MMD as your CMS

2010-01-31T21:26:19-04:00

How to use MMD as your CMS

01/31/2010 21:26:19

A while back I mentioned that I had started using MultiMarkdown to power my web site, without using a separate CMS. I have now released the software I’ve been using for anyone who’s interested.

MultiMarkdown-CMS

How to delete all notes from Simplenote

2010-01-31T13:25:07-04:00

How to delete all notes from Simplenote

01/31/2010 13:25:07

With the recent addition of Simplenote synchronization to Notational Velocity, I wanted to experiment with the new features and check out my Simplenote account.

However, I discovered that I had a bunch of notes, duplicates, and other cruft in my Simplenote account, and I seemed to be unable to delete them. They kept reappearing.

After trial and error, I was able to figure out how to finally kill them for good:

Delete the simplenote application from your iPhone
Run the “nuclear-option.pl” script that I added to SimplenoteSync.pl — this script systematically deletes all notes from your Simplenote account.
In order to prevent the notes from reappearing, you have to start with a fresh copy of the iPhone app. You can now reinstall it, and connect it to your Simplenote account using the previous login and password.
After synchronizing you should have a fresh install. You can now migrate any old notes over using your method of choice (Notational Velocity, SimplenoteSync.pl, etc)

Status update on SimplenoteSync

2010-01-31T10:32:52-04:00

Status update on SimplenoteSync

01/31/2010 10:32:52

A few months ago, I wrote the SimplenoteSync.pl script to scratch a particular itch. I wanted to use Notational Velocity on my mac, and Simplenote on my iPhone, and I wanted the same data on both.

So I wrote the perl script, got it (mostly) working, and released it for others to try out. Thanks to everyone’s feedback, I was able to fix some bugs, and improve the script. I even received an email from NV’s author informing me that there would eventually be built-in support for Simplenote synchronization.

It looks like that has finally arrived. I have not reviewed the code, but I imagine that for users who just want to synchronize their Notational Velocity database with Simplenote, the built-in approach will definitely be easier, and likely more reliable than using SimplenoteSync. For these users, it would appear that SimplenoteSync is no longer necessary.

There are definitely other mac users of SimplenoteSync who don’t use Notational Velocity (why not?!??!?), and users of other operating systems who can’t. For these people, SimplenoteSync is still an option.

SimplenoteSync.pl is still available. It’s open source, so anyone can improve it. I’ll still answer emails, and of course the discussion list is still around. But I haven’t been using SimplenoteSync myself for quite a while (I switched primarily to WriteRoom, and will now use Notational Velocity to directly sync with Simplenote for the occasional use). I’m not actively developing it, though I will certainly add any bug fixes that I become aware of.

To summarize, my recommendations are:

If you use a mac, you really should try out Notational Velocity, and its support for synchronizing to Simplenote
Otherwise, you can still use SimplenoteSync and it will continue to work as well as it has so far
Please contact me via the discussion list if you have questions about using SimplenoteSync.pl. If you have difficulty or questions about how to use the command line, perl, shell scripts, etc then please use google to find more appropriate resources.

Thank you to everyone who has used and supported SimplenoteSync.pl, and I’ll continue to support ongoing users as much as I can!

Additional OPML Support for MMD

2010-01-30T14:58:36-04:00

Additional OPML Support for MMD

01/30/2010 14:58:36

When I first started working with MultiMarkdown, I was using OmniOutliner. I created a plug-in for OmniOutliner that would export an outline into a Markdown document. The idea was that it made working with complex documents easy - you could easily move sections around in outline form, and converting the outline to chapters, sections, etc. was trivial.

Then I began using Scrivener, and let my OO work fall by the wayside.

Then I got back into using TextMate, and stopped using Scrivener.

This week’s announcement of the iPad, and Omni’s subsequent announcement that they plan a version of OmniOutliner for the iPad at some point got me thinking about this again.

I wrote a quick perl script that converts a MultiMarkdown plain text file into an OPML file that maintains the overall structure. It moves the metadata (if present) to the end of the file, following the convention I established for my OmniOutliner Markdown export plugin.

So basically, it is now possible to round-trip a document from plain text to OPML (which is easily imported into OmniOutliner or other outliners) and back again.

I have no idea when a version of OmniOutliner for the iPad might become available, but it would be nice to have this ready when it is. And, in any event, it opens up additional options for using MMD with any OPML capable outliner.

Check out more information on MultiMarkdown and OmniOutliner.

Revised Movable Type instructions for MMD

2010-01-28T17:24:06-04:00

Revised Movable Type instructions for MMD

01/28/2010 17:24:06

I have rewritten the page describing how to configure MultiMarkdown when using Movable Type.

Automatic postings to MMD discussion list

2010-01-26T13:03:33-04:00

Automatic postings to MMD discussion list

01/26/2010 13:03:33

Whenever I post a new entry to this site, I want to make sure that it is properly announced. This used to require manually sending out an email to the discussion list. Then it involved using posterous to update my twitter account. Then I switched to a home built CMS, and I redesigned the work flow.

I use a cgi script to serve up an Atom feed for the site, so whenever it is called, it automatically provides the latest entries.

I also use twitterfeed to automatically tweet whenever a new entry is added to the site. This happens automatically within about 30 minutes from when the entry is added.

Until today, however, I still had to manually send an email to the MMD Mailing List. This involved copying the text, and remembering to send an email.

I looked into RSS to email solutions, but the free options I found didn’t suit my needs, and it wasn’t worth paying for when a simple script should do it for me.

So I added a new script that looks for new entries, and when found sends an email of the content to the specified address, appending a link to the entry at the end. It also sends the raw MMD as the plain/text version, and the XHTML as the “pretty” version. The script is called by git in the post-update hook.

We’ll see how well it works, but it should make it much easier to keep everything in sync!

Update to SimplenoteSync

2010-01-25T15:02:25-04:00

Update to SimplenoteSync

01/25/2010 15:02:25

I tweaked the SimplenoteSync script at the suggestion of a user, and it should work on *nix systems (more or less). The date synchronization may not work, and I am open to suggestions on how to improve it.

Please let me know if it works for you.

http://github.com/fletcher/SimplenoteSync

Sample MultiMarkdown Document for Scrivener “fixed”

2010-01-10T16:35:56-04:00

Sample MultiMarkdown Document for Scrivener “fixed”

01/10/2010 16:35:56

As a few people have noticed, the sample MMD document for Scrivener was saved using a test version of Scrivener, which made the document incompatible with older versions.

I created a new version of the document that is compatible with 1.52, the latest public version as of the time of this writing.

Enjoy!

http://github.com/fletcher/MultiMarkdown-Gallery

Interesting discussion about the lack of Markdown development

2009-12-30T22:02:33-04:00

Interesting discussion about the lack of Markdown development

12/30/2009 22:02:33

http://www.red-sweater.com/blog/1078/open-source-obligations

I’m even mentioned…. ;)

Perhaps I should polish up some of my proposals for a future of Markdown development and put them out for others to ponder.

Happy Holidays!

2009-12-25T11:18:22-04:00

Happy Holidays!

12/25/2009 11:18:22

Wishing everyone Happy Holidays, and a Happy New Year!!

Return of the tag map

2009-12-15T10:56:39-04:00

Return of the tag map

12/15/2009 10:56:39

I had previously written a script to help make sense of all the tags applied to posts on my site when it was run using OddMuse. I have recently enabled tags on my home built CMS, and have modified the TagCategorizer script to help try to make sense of the tags here.

You can see the results by viewing the Tag Map.

Freeshell has moved

2009-12-07T18:02:07-04:00

Freeshell has moved

12/07/2009 18:02:07

My old site at http://fletcher.freeshell.org/ has finally been shut down. I moved to this new domain 2 1/2 years ago, and moved most of the “important” content over within a short period of time. I left the old site up partly out of laziness, and partly because there was still some content on there that may have been of interest to some people.

I have now moved that vast majority of the content from my old wikis — both at freeshell, and the old wiki here at fletcherpenney.net. I have not moved the comments, but I plan to do so at some point. We’ll see.

If you attempted to go to a page at the old site, and were redirected here then please feel free to look around. It may take Google a few days to index the new pages here, so the search function may be limited. But the “similar pages” feature is working, and tags are working for the pages that have them. You can also browse the archives.

What is an OpenID?

2009-12-07T16:57:27-04:00

What is an OpenID?

12/07/2009 16:57:27

OpenID is a mechanism for authenticating individuals using a decentralized approach. Please review the link for more information, as I would almost certainly not give a clear explanation.

For this site, however, it means that I don’t actually store any database of users to authenticate against for submitting comments. You can log in using any (as far as I know) valid OpenID that follows the standard. You may already have an OpenID, since various services are already configured to serve as an OpenID provider.

Old web entries migrating to this site

2009-12-07T11:19:20-04:00

Old web entries migrating to this site

12/07/2009 11:19:20

My web sites have previously used blosxom, then OddMuse, then Movable Type, and now finally my own custom solution relying primarily on MultiMarkdown and a few cgi scripts. Between the changes, I have also used two different domain names.

When I started this site, I moved much of my “main” content over, but left behind several hundred pages, some of which contained some marginally interesting/useful information. It was stored in an OddMuse wiki, and I didn’t take much time to think about how to easily move all of the content over, or what to do about forwarding any requests to the old site.

Over the weekend, I wrote a script to convert the old OddMuse page files into MultiMarkdown documents suitable for posting to my new site. I was able to salvage the original creation date (or at least the last modification date) for the posts.

These posts will appear in the archives filed under their original date. I have tagged some of them using my new tag support, but they are all indexed for the similar pages feature. I suspect that will be how any visitors may stumble across them.

Rather than trying to do anything fancy with my old site, however, I will probably pull the plug and put a URL redirect that directs visitors to a landing page here. I don’t think it gets much traffic, and it certainly hasn’t seen any maintenance from me in a long time….

Over time, I hope to continue to refine my content analysis tools and will probably further tag and classify this old content to make it more accessible to others (and myself).

New tag feature enabled

2009-12-04T12:22:30-04:00

New tag feature enabled

12/04/2009 12:22:30

After rebuilding my site to basically run off of static html files, with a healthy dose of server side includes, I have been experimenting with how to add other feature. For example, I used Google’s Custom Search Engine to handle searches, I added a feature to suggest similar pages, custom atom feeds, and more.

I was also interested in re-enabling tags, but wasn’t sure of the best way to do it. Ideally, the system would be intelligent — suggesting tags for new content, providing a mechanism to analyze the tag framework as it grows to find redundancy, and ensuring that pages are tagged appropriately.

I’ve started working on a system. To back up for a moment, though, I want to describe the similar pages feature. Basically, it creates a “vector” for each page in the site that corresponds to the number of occurrences of each word in the document. The words are stemmed, certain common words are removed, and everything is lower-cased. For example, a document consisting of “The Cow jumped over the Moon” would become:

cow     0.577350269189626
moon    0.577350269189626
jump    0.577350269189626

Because of the way the math works, you don’t need to store a “0” for each word that is not in a given document, but occurs in other documents. This makes things much easier.

To compare two documents, one simply calculates the cosine of the angle between the two respective vectors. The closer the vectors align with each other, the closer the cosine is to 1 (a perfect match).

The way I suggest similar pages is to compare each document with every other document and calculate their similarities. When a page is shown, this list is scanned, and matches are found that exceed a certain threshold. I also cap the list at the top 5 choices so a visitor isn’t overwhelmed with 12 similar pages.

Back to tags…. I have a routine I can run locally that compares pages to the overall vector for the pages contained within each tag. In otherwords, I can find pages that look similar to pages already in a tag with the idea that perhaps this new page belongs there as well. This can also suggest tags for a new document that has yet to be posted (thanks, TextMate!).

I still would like to create a tool to visualize the relationships between pages based on similarities and tags to more formally look for a structure to the pages and the relationships between them. This might help suggest new tags that aren’t being used — for example, if there is a cluster of closely related pages about “ice cream”, that would likely be a tag that should be added.

Automatic content sharing

2009-11-22T14:02:50-04:00

Automatic content sharing

11/22/2009 14:02:50

When I redesigned my web site to run primarily as a static site, with a few cgi scripts to add interactivity, I still wanted to incorporate some of the aspects of social sites I was using previously.

For example, I liked that posterous would automatically update my twitter feed when new content was posted to my old news site. Movable Type already provided atom and RSS access to my content, so I also needed to add that.

So basically, I wrote a script that would serve my static content as an atom feed — easy to do since my content was stored in MMD text and minimally marked up XHTML files. Easy enough.

Then I used twitterfeed to monitor that feed, and when it detects new articles, it adds a comment to my twitter account announcing the title and a shortened URL.

Using these methods, I simply add a new article to my site. Automatically, the Atom feed is updated, and within half an hour, an update is posted to my twitter account (@MultiMarkdown).

I am currently testing a similar setup for comments on my site (a separate Atom feed for comments has been around for a while).

WARNING - Problems with SimplenoteSync

2009-11-22T10:00:49-04:00

WARNING - Problems with SimplenoteSync

11/22/2009 10:00:49

Over the last few weeks (since some changes were made on the Simplenote server?) I have noticed files going missing from my local machine.

I have emailed the support team, but didn’t get anything helpful back.

I haven’t changed anything locally.

The problem didn’t start until around the time they announced changes on the server backend.

My personal solution was to no longer leave Notational Velocity running all the time. This way, when it launches, it seems to do a better job of noticing the missing files and asking if I want to restore them (and I say YES).

I assumed it was something I had done wrong (though I’m not sure what), and after restoring my files everything seemed fine.

This morning I launched Notational Velocity and was warned of deleted files. It appears that approximately 260 of my 340 files were deleted. I promptly restored them thanks to Notational Velocity, and then I disabled the cron job I use to automatically run SimplenoteSync.pl every 15 minutes.

I don’t know what the problem is. It’s possible that there is a bug in my script, but the timing seems rather coincidental. In any case, until it is resolved, or the direct integration between Simplenote and Notational Velocity occurs, I suggest:

Disable any scripts that automatically run SimplenoteSync.pl
Make (and maintain) up to date backups of your data (you’re doing this already, right?!?)
Quit Notational Velocity before running SimplenoteSync.pl
run the SimplenoteSync.pl script manually, and verify the count of items in the folder appears correct before and after syncing
re-run Notational Velocity after running SimplenoteSync.pl and look for warnings of any missing files that seem incorrect

I’ll notify you if I learn anything new, but I wanted to try to warn everyone about this potential problem…

New “Similar Pages” Feature On My Site

2009-10-18T17:23:00-04:00

New “Similar Pages” Feature On My Site

10/18/2009 17:23:00

For a while I have been interested in using computers to analyze the content of text documents, and how programs are able to find similar documents. I found a web site that had a “Similar entries” feature that would list a couple of posts that were similar to the one you were viewing. That’s clearly a useful feature to help visitors locate content on other pages they may be interested in.

After much digging around the internet, I was able to find some descriptions and examples of algorithms to implement such a feature. Basically, I count all the instances of each word in a given document. These are used to generate a normalized vector which represents the overall content of the document in terms of which words are used more frequently. These vectors are calculated for each document in the site.

Then, the vectors for any given pair of documents on the site are compared (technically, by using the cosine of the angle between them). The function generates a number between 0 and 1 that indicates how “parallel” the vectors are, which correlates to an estimate of how similar the documents are.

I then wrote a quick script to query this database and show similar pages that match a threshold of similarity, and also limit the number of matches shown so as not to be overwhelming.

I am really interested in generalizing this to look into how to automatically tag certain documents, and find new tags I should be using… We’ll see!

SimplenoteSync Update

2009-10-15T16:51:20-04:00

SimplenoteSync Update

10/15/2009 16:51:20

I released an update to SimplenoteSync yesterday that fixes an issue where the script would download a note just to “verify” it was flagged as deleted, when it should know this from downloading the index.

Basically, this fix should make the script run faster and hit the server less often.

It also adds a flag to log network activity - if it seems to be running slow for you, enabling this flag will let you know each time it hits the Simplenote web server. They were seeing some increased activity, most of which was probably from improper user configuration (e.g. running the script too often), but this will certainly help some.

Please update, and as always — BACKUP YOUR DATA!

New Search Features on My Site

2009-10-15T16:25:38-04:00

New Search Features on My Site

10/15/2009 16:25:38

For a few weeks I have been using Google to search my site, instead of running a local search script. Today, I modified it to also search the contents of the MultiMarkdown and (newly formed) SimplenoteSync discussion groups. In this way, by running one search query, you can find information related to my software on any of the “official” places.

New discussion group for SimplenoteSync

2009-10-15T16:20:40-04:00

New discussion group for SimplenoteSync

10/15/2009 16:20:40

There is now a discussion list for SimplenoteSync on Google groups. You can access on the web or by email.

http://groups.google.com/group/simplenotesync

simplenotesync@googlegroups.com

fletcherpenney.net

MMD-CMS has support for &#8220;make&#8221;

MMD-CMS has support for &#8220;make&#8221;

New OPML Features added to MultiMarkdown

New OPML Features added to MultiMarkdown

MultiMarkdown CMS now incorporates &#8220;LESS&#8221;

MultiMarkdown CMS now incorporates &#8220;LESS&#8221;

Several Updates to MultiMarkdown CMS

Several Updates to MultiMarkdown CMS

How to use MMD as your CMS

How to use MMD as your CMS

How to delete all notes from Simplenote

How to delete all notes from Simplenote

Status update on SimplenoteSync

Status update on SimplenoteSync

Additional OPML Support for MMD

Additional OPML Support for MMD

Revised Movable Type instructions for MMD

Revised Movable Type instructions for MMD

Automatic postings to MMD discussion list

Automatic postings to MMD discussion list

Update to SimplenoteSync

Update to SimplenoteSync

Sample MultiMarkdown Document for Scrivener &#8220;fixed&#8221;

Sample MultiMarkdown Document for Scrivener &#8220;fixed&#8221;

Interesting discussion about the lack of Markdown development

Interesting discussion about the lack of Markdown development

Happy Holidays!

Happy Holidays!

Return of the tag map

Return of the tag map

Freeshell has moved

Freeshell has moved

What is an OpenID?

What is an OpenID?

Old web entries migrating to this site

Old web entries migrating to this site

New tag feature enabled

New tag feature enabled

Automatic content sharing

Automatic content sharing

WARNING - Problems with SimplenoteSync

WARNING - Problems with SimplenoteSync

New &#8220;Similar Pages&#8221; Feature On My Site

New &#8220;Similar Pages&#8221; Feature On My Site

SimplenoteSync Update

SimplenoteSync Update

New Search Features on My Site

New Search Features on My Site

New discussion group for SimplenoteSync

New discussion group for SimplenoteSync

MMD-CMS has support for “make”

MMD-CMS has support for “make”

MultiMarkdown CMS now incorporates “LESS”

MultiMarkdown CMS now incorporates “LESS”

Sample MultiMarkdown Document for Scrivener “fixed”

Sample MultiMarkdown Document for Scrivener “fixed”

New “Similar Pages” Feature On My Site

New “Similar Pages” Feature On My Site