CommonMarkMarkdown done right

Colin O’Dell@colinodell

COLIN O’DELL

Creator & Maintainer of league/commonmarkLead Web Developer at Unleashed TechnologiesAuthor of PHP 7 Migration Guide e-book

@colinodell

TOPICS

Origins of MarkdownCommonMark Specleague/commonmark OverviewCustomizing league/commonmark

ORIGINS OF MARKDOWN

Created in March 2004 by John GruberInformal plain-text formatting languageConverts readable text to valid (X)HTMLPrimary goal - readability

HISTORY OF MARKDOWNHello Nomad PHP!----------------

Markdown is **awesome**!

1. Foo2. Bar3. Baz

Wikipedia entry: <https://en.wikipedia.org/wiki/Markdown>

WHY IS IT SUCCESSFUL?

1.Syntax is visually-similar to the resulting markup

2.Non-strict, forgiving parsing3.Easily adaptable for different uses

67+ DIFFERENT FLAVORS

Source: https://github.com/markdown/markdown.github.com/wiki/Implementations

ActuariusBlackfridayBlueClothBlueFeathercebe/markdownCocoaMarkdownDiscountffi-sundownGHMarkdownParserGoskirtHoedownHoepKnockoffkramdownLaikalibpandocLowdownlua-discount

Lunamarkmarkdownmarkdown-cljmarkdown-jsmarkdown-oo-phpmarkdown.bashmarkdown.luamarkdown.plmarkdown4jMarkdownDeepMarkdownJMarkdownPapersMarkdownSharpmarkedMarukumd2html.awkMisakaMistune

MMMarkdownMoonShineMultiMarkdownnode-discountnode-markdownnode-multimarkdownOMDPandocParsedownParsedown Extrapeg-markdownpeg-multimarkdown & forkpegdownPHP MarkdownPHP Markdown ExtraPHP-SundownPython-Discountpython-hoedown

Python-MarkdownPython-Markdown2RDiscountRedcarpetRoboSkirtShowdownSundownSundown HSSundown.nettext-markdowntexts.jsTxtmarkupskirt.go

https://xkcd.com/927/

COMMONMARK IS…A strongly defined, highly compatible specification of Markdown.

Written by John MacFarlane(in collaboration with people from Github, StackOverflow, Reddit, and others)

Spec includes: Strict rules (precedence, parsing order, handling edge cases) Specific definitions (ex: “whitespace”, “punctuation”) 616 examples

WHY IS IT NEEDED?

*I love Markdown*<p><em>I love Markdown</em></p>

WHY IS IT NEEDED?

*I *love* Markdown*

WHY IS IT NEEDED? Source: http://johnmacfarlane.net/babelmark2/

30%

WHY IS IT NEEDED?

*I *love* Markdown*<p><em>I <em>love</em> Markdown</em></p>

*I *love* Markdown*<p><em>I </em>love<em> Markdown</em></p>

*I *love* Markdown*<p><em>I *love</em> Markdown*</p>

15%

33%

Source: http://johnmacfarlane.net/babelmark2/

WHY IS IT NEEDED?

1. > Hello World! ------

Source: http://johnmacfarlane.net/babelmark2/

WHY IS IT NEEDED?

1. > Hello World! ------

Source: http://johnmacfarlane.net/babelmark2/

WHY IS IT NEEDED?

1. > Hello World! ------

Source: http://johnmacfarlane.net/babelmark2/

WHY IS IT NEEDED?

1. > Hello World! ------

Source: http://johnmacfarlane.net/babelmark2/

WHY IS IT NEEDED?

1. > Hello World! ------

Source: http://johnmacfarlane.net/babelmark2/

LEAGUE/COMMONMARK

A well-written, super-configurable Markdown parser for PHP based on the CommonMark spec.

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

ADDING LEAGUE/COMMONMARK$ composer require league/commonmark:^0.14

<?php$converter = new CommonMarkConverter();echo $converter->convertToHtml('Hello **Nomad PHP!**');

INTEGRATIONS

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

CONVERSION PROCESS

<https://nomadphp.com>

<a href="https://nomadphp.com">https://nomadphp.com

</a>

CONVERSION PROCESS

<https://nomadphp.com>

Markdown Parse

<document> <paragraph> <link destination="https://nomadphp.com"> <text>https://nomadphp.com</text> </link> </paragraph></document>

CONVERSION PROCESS

Markdown AST RenderParse

CONVERSION PROCESS

<a href="https://nomadphp.com">https://nomadphp.com

</a>

Markdown AST HTMLRenderParse

CONVERSION PROCESS

Markdown AST HTMLRenderParse

Add your own custom parser, processor, or renderer

EXAMPLE 1: CUSTOM PARSER<https://nomadphp.com>

<a href="https://nomadphp.com">https://nomadphp.com

</a>

<@colinodell>

<a href="https://twitter.com/colinodell">

@colinodell</a>

class TwitterUsernameAutolinkParser extends AbstractInlineParser { public function getCharacters() { return ['<']; }

public function parse(InlineParserContext $inlineContext) { $cursor = $inlineContext->getCursor(); }}

CURSOR

Learning CommonMark with <@colinodell>!

getCharacter()getFirstNonSpaceCharacter()getRemainder()getLine()getPosition()

advance()advanceBy($count)advanceBySpaceOrTab()advanceWhileMatches($char)advanceToFirstNonSpace()

isIndented() isAtEnd() match($regex) peek($count)

CURSOR::ADVANCE()

Learning CommonMark with <@colinodell>!

getCharacter()getFirstNonSpaceCharacter()getRemainder()getLine()getPosition()

advance()advanceBy($count)advanceBySpaceOrTab()advanceWhileMatches($char)advanceToFirstNonSpace()

isIndented() isAtEnd() match($regex) peek($count)

CURSOR::ADVANCEBY(INT)

Learning CommonMark with <@colinodell>!

getCharacter()getFirstNonSpaceCharacter()getRemainder()getLine()getPosition()

advance()advanceBy($count)advanceBySpaceOrTab()advanceWhileMatches($char)advanceToFirstNonSpace()

isIndented() isAtEnd() match($regex) peek($count)

CURSOR::PEEK(INT)

Learning CommonMark with <@colinodell>!

getCharacter()getFirstNonSpaceCharacter()getRemainder()getLine()getPosition()

advance()advanceBy($count)advanceBySpaceOrTab()advanceWhileMatches($char)advanceToFirstNonSpace()

isIndented() isAtEnd() match($regex) peek($count)

CURSOR

Learning CommonMark with <@colinodell>!

public function getCharacters() { return ['<'];}

CURSOR

Learning CommonMark with <@colinodell>!

getCharacter()getFirstNonSpaceCharacter()getRemainder()getLine()getPosition()

advance()advanceBy($count)advanceBySpaceOrTab()advanceWhileMatches($char)advanceToFirstNonSpace()

isIndented() isAtEnd() match($regex) peek($count)

CURSOR

Learning CommonMark with <@colinodell>!

getCharacter()getFirstNonSpaceCharacter()getRemainder()getLine()getPosition()

advance()advanceBy($count)advanceBySpaceOrTab()advanceWhileMatches($char)advanceToFirstNonSpace()

isIndented() isAtEnd() match($regex) peek($count)

/^<@[A-Za-z0-9_]+>/

CUSTOMIZING LEAGUE/COMMONMARK

class TwitterUsernameAutolinkParser extends AbstractInlineParser { public function getCharacters() { return ['<']; }

public function parse(InlineParserContext $inlineContext) { $cursor = $inlineContext->getCursor(); if ($match = $cursor->match('/^<@[A-Za-z0-9_]+>/')) { // Remove the starting '<@' and ending '>' that were matched $username = substr($match, 2, -1);

$profileUrl = 'https://twitter.com/' . $username; $link = new Link($profileUrl, '@'.$username); $inlineContext->getContainer()->appendChild($link);

return true; }

return false; }}

$environment = Environment::createCommonMarkEnvironment();$environment->addInlineParser( new TwitterHandleParser());

$converter = new CommonMarkConverter($environment);

$html = $converter->convertToHtml( "Follow <@colinodell> on Twitter!");

EXAMPLE 2: CUSTOM AST PROCESSOR<document> <paragraph> <link destination="https://nomadphp.com"> <text>https://nomadphp.com</text> </link> </paragraph></document>

<document> <paragraph> <link destination="https://bit.ly/foo"> <text>https://nomadphp.com</text> </link> </paragraph></document>

class ShortenLinkProcessor implements DocumentProcessorInterface { public function processDocument(Document $document) { $walker = $document->walker(); while ($event = $walker->next()) { if ($event->isEntering() && $event->getNode() instanceof Link) { /** @var Link $linkNode */ $linkNode = $event->getNode(); $originalUrl = $linkNode->getUrl(); $shortUrl = $this->bitly->shorten($originalUrl); $linkNode->setUrl($shortUrl); } } }}

$environment = Environment::createCommonMarkEnvironment();$environment->addDocumentProcessor( new ShortenLinkProcessor());

$converter = new CommonMarkConverter($environment);

$html = $converter->convertToHtml( "Meetings: <https://nomadphp.com/upcoming/>");

EXAMPLE 2: CUSTOM AST PROCESSOR

EXAMPLE 3: CUSTOM RENDERER<document> <paragraph> <text>Hello World!</text> </paragraph> <thematic_break /></document>

<p>Hello World!</p><hr />

<p>Hello World!</p><img src="hr.png" />

EXAMPLE 3: CUSTOM RENDERERclass ImageHorizontalRuleRenderer implements BlockRendererInterface { public function render(...) { return new HtmlElement('img', ['src' => 'hr.png']); }}

$environment = Environment::createCommonMarkEnvironment();$environment->addBlockRenderer( League\CommonMark\Block\Element\ThematicBreak::class, new ImageHorizontalRuleRenderer());

$converter = new CommonMarkConverter($environment);

$html = $converter->convertToHtml("Hello World!\n\n-----");

EXAMPLE 3: CUSTOM RENDERER

BUNDLING INTO AN EXTENSIONclass MyCustomExtension extends Extension { public function getInlineParsers() { return [new TwitterUsernameAutolinkParser()]; }

public function getDocumentProcessors() { return [new ShortenLinkProcessor()]; }

public function getBlockRenderers() { return [new ImageHorizontalRuleRenderer()]; }}

BUNDLING INTO AN EXTENSION

$environment = Environment::createCommonMarkEnvironment();$environment->addExtension(new MyCustomExtension());

$converter = new CommonMarkConverter($environment);

$html = $converter->convertToHtml("...");

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

WELL-TESTED

94% code coverage Functional tests

All 616 spec examples Library of regression tests

Unit tests Cursor Environment Utility classes

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

PERFORMANCE

Parsedown

cebe/markdown

PHP Markdown Extra

league/commonmark

0 10 20 30 40 50 60 70 80 90

league/commonmark is ~35-40ms slower

PHP 5.6 PHP 7.0

Time (ms)

Tips: • Use PHP 7 (50-80% boost)• Choose library based on your

needs• Cache rendered HTML (100%

boost)• Optimize custom functionality

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

STABILITY

Current version: 0.15.0 Conforms to CommonMark spec 0.26 1.0.0 will be released once CommonMark spec is 1.0

No major stability issues Backwards Compatibility Promise:

No BC breaks to CommonMarkConverter class in 0.x Other BC breaks will be documented

FEATURES100% compliance with the CommonMark specEasy to implementEasy to customizeWell-testedDecent performance (Relatively) stable

Installation & Documentation: http://github.com/thephpleague/commonmark

Learn More About CommonMark: http://commonmark.org

Slides / Feedback: https://joind.in/talk/22293

@colinodell