1 Introduction

Jeremy and Rachel, esteemed founders of fastai, quote Blogging is like Resume, only better in this post. Blogging really is one of the best medium to learn while doing. It helps to communicate and formulate ideas, and share knowledge with your peers and colleagues. Blogging for me addresses provides three main motivations:

  1. It helps in improving communication and writing skills.
  2. It provide medium to showcase technical expertise in programming, data science and other domains.
  3. It is a great medium for self learning and teaching.

With nbdev, fastai library and video courses; Fastai team has contributed some of the best material and tools for learning data science specificaaly and programming in general. The top down learning style , with focus on learning the big picture by execution and programming, is an amazingly useful and productive technique especially for someone who is working full time.

Fastpages is the recent addition in the same tradition of tools made available by FastAI team to new and practicing data scientist to facilitate there learning journey. It capitalizes on nbdev codebases, a python literate programming tool and github actions to enable blogging by jupyter notebook and markdown files.

My objective in this post is to showcase some of the minor changes which I have made which enables me to use entire toolchain of fastpages; to blog with emacs org mode along with notebook, markdown or word format. I also intend to demonstrate how powerful org mode really is ; and why it belongs in a Data Scientist toolbox alongwith the list of tools mentioned above.

2 EMACS ORG-MODE: WHAT? WHY? HOW?

BEFORE CAME JUPYTER, BEFORE CAME INTERNET AND EVEN BEFORE THERE WERE PERSONAL COMPUTERS.THERE WAS SOMETHING CALLED EMACS

2.1 The What?

So what is org-mode? At the very basic level org mode or org format is a text markup defined for structured text editing. It is similar to markdown as it is defined in plaintext, priortizes readability and has an human centric approach. Combination with emacs elisp system makes it insanely powerful. Instead of yet another markup syntax; it becomes a system of text editing which is highly extensible and infinitely programmable. With various open source plugins' it can be used like a literate programming tool like notebooks or frontend for latex report creation or even online presentations.

2.2 The Why?

So why should you be interested in Org Mode .Here is a brief list of things that you can do with it.

2.2.1 Blogging

You can write a blog post like this. Since Emacs is insanely powerful for text editing you can do really fast typing.

2.2.2 FastPages Functionality

Lot of bells and whistle related to boxes and link are enabled using org mode link functionality

  • Links are prepended by a directive
  • When doing html export link get converted into jekyll include statement already defined in the code
  • When exporting to other backends it follows the same structure as normal link for that structure
  1. Youtube Video Link

    You can embed individual video or even playlist 

    [[yt:https://youtu.be/5haX95nk02E][New Link Feature Org Mode]]


    
[[yt:https://www.youtube.com/playlist?list=PLxc79l2wpbJYTI5rv2os7OoKQMqxReZpr][The Playlist for Org Mode]]

  2. Twitter Card Link
    [[twitter:https://twitter.com/jakevdp/status/1204765621767901185?s=20][Altair v4 release]]

  3. Remote Image Link
    [[img:https://www.fast.ai/images/fastai_paper/show_batch.png][Credits: https://www.fast.ai/2020/02/13/fastai-A-Layered-API-for-Deep-Learning/]]

    Credits: https://www.fast.ai/2020/02/13/fastai-A-Layered-API-for-Deep-Learning/
    Credits: https://www.fast.ai/2020/02/13/fastai-A-Layered-API-for-Deep-Learning/

    [[img:https://upload.wikimedia.org/wikipedia/commons/7/71/ChessPawnSpecialMoves.gif]]

    Image

  4. Local Images
    [[img:/images/Emacs.png][Emacs is the king]]

    Emacs is the king
    Emacs is the king

    This has been awesome so far.

  5. Boxes and Stuff
    [[alert:This is an alert box]]

[[alert:box][Danger ahead]]

[[info:This is info box]]

[[info:box][This is for your information]]

[[warning:box][You have been warned]]

[[important:box][This is important]]

[[tip:box][You are going to get lucky with emacs]]

[[note:box][This is a note]]

*I am currently in process of fixing doclink functionality inside note box*
[[note:box][A doc link to [an example website: fast.ai](https://www.fast.ai/) don't work yet.]]

    This is an alert box

    Danger ahead

    This is info box

    This is for your information

    Warning: You have been warned

    Important: This is important

    Tip: You are going to get lucky with emacs

    Note: This is a note

    I am currently in process of fixing doclink functionality inside note box

    Note: A doc link to [an example website: fast.ai](https://www.fast.ai/) don't work yet.
  6. IFrames and Revealjs Presentation
    [[iframe:https://revealjs.com][Reveal JS Presentation]]

    You can embed websites as iframes. This is particularily useful for embedding revealjs presentation like this.

    Infact it is also possible to create revealjs presentation directly from org mode. I will be working on integrating the same in next few days.

    You can also embed powerpoint presentation

2.2.3 Document Embedding

For a longish article, sometimes you would like to split it in different files and combine them together. This workflow is usually better for writing as well; as each document can be focussed on a single aspect of entire topic you want to present in your article.

In org-mode it is trivial to do this using `#+INCLUDE` functionality. As an example above section on Fastpages functionality is actually part of a different blog published in here. I have just imported relevant portions for this post. Code/ configuration files in the bottom section are embedded using the same technique. 

#+INCLUDE: "2020-04-29-Awesome-Org-Mode-Links.org::FastPages link" :only-contents t

2.2.4 Literate Programming

You can do literate programming just like jupyter notebooks, create technical reports, even write full fledged books. Org mode also supports various export backends which means same document can be converted to a html blog, a latex report, beamer or revealjs presentations. Here are a few good links to explore

  • A Multi-Language Computing Environment for Literate programming and Reproducible Research here
  • Org Mode Recipes

  • Literate programming in python with org-mode and noweb here

2.2.5 GTD, Task Manager, Todo

You can use it as task manager, do project management, create to do list. Follow the recommendations of GTD managing your life in a text edit. And many many more things as described here

2.2.6 Are you excited?

What I have done is just touch upon the surface of what is possible with org mode, and by extension blogging on fastpages with org mode. There are many more options and functionality available in org mode including beamer presentation, spreadsheets , agenda views etc… which are awesome but too broad to cover in this introductory blog post. However over time I will continue to post articles on some tidbits' here and there which can greatly enhance your blogging workflow. Now I hope I have motivated you enough, so time to figure out to do it yourself.

2.3 The How?

I have already shown earlier what org mode can do and how it can enhance your writing workflow. Now I will describe how you can do it yourself. Before I go into specifics let me share a bit on general design of fastpages and describe how it can be extended.

2.3.1 Design of fastpages

So here is brief synopsis. Fastpages is based on Jekyll. It already provides you functionality to write a blog in markdown, notebook or word document. This is how it works

  1. Markdown Blogging
    • Markdown is the native format for Jekyll Blogging Engine. Jekyll uses an enhanced version of Markdown called Kramdown which comes with addtional functionality for source code coloring links etc…
    • Layout of Jekyll (Structure of html) is written in `Liquid` template engine. You have liquid templates in two places
      • _layouts folder - used for defining page/post structure.
      • _includes folder - used to define general html blocks or additional functionality which may be included in an article
    • YAML Front Matter - Each article needs to have a yaml frontmatter which defines specifics like title, author, layout to choose, comments to include or exclude , categories , tags etc… which are either generic cross cutting functionality or indicator to select some specific configuration for blogging engine. It needs to come at very top of page
  2. Notebook Blogging
    • Notebook blogging is really what makes fastpages special for data scientist.I should rather specify and say python notebook blogging is what is possible natively in fastpages. Notebook blogging is enabled by a python package called "nbdev" another tool from FastAI.
    • The way it works is as follows
      • You need to specify front matter in first cell of notebook
      • Import / Export/ Hide / Show/ Collapsible hide etc… are directive implemented as comments which need to appear on top of cell to indicate inclusion or exclution from html export
      • In reality notebooks from _notebooks folder are converted to html[not markdown] and exported to _posts folder with a fake (.md) extension as an initial preprocessing step
      • From _post folder Jekyll machinery takes over to convert them into rendered site alongwith markkdown posts
      • Fastpages provides various shims(utlities/methods) to convert some notebook specific directives for links, youtube or boxes to liquid include templates. These are then converted by Jekyll to final version of site.
  3. Word Blogging
    • Word blogging structure is a similar
      • It uses pandoc for "docx" to "html" conversion. Which is copied to _posts folder with (.md) extension
      • Some shim methods are provided by FastPages to handle jekyll _includes and _layout nicely
      • For Words there is only one place to specify "Front Matter". It is _actionfiles/wordfrontmatter.txt file
  4. Automation via Github Actions
    • All processing automation are handled by Github Actions
      • Checkout code to some machine
      • Cleaning up old _site or html
      • Convert files in _notebooks and _word to _posts
      • Building the site using Jekyll into _site
      • Copying / Commiting the _site folder to ghpages etc..
    • Everything in Github Actions are handled by services/ different docker containers running different scripts
    • For local viewing a parallel system is provided using docker-compose.yml and Makefile.

2.3.2 General Idea for extension

So based on the intuition above we basically need to figure out 3 pieces

  • HTML/Markdown Converter :Way to convert custom format to html( with fake .md extension) or actual kramdown style markdown. It needs to have a way to insert/ send YAML FrontMatter to exported files
  • Shims for Jekyll: Some customization scripts to handled Jekyll _include template nicely to manage look and feel
  • Automation Code
    • Github Action [ Include an addtional conversion step in ci.yml or integrate with existing action files possibly in _actionfiles/actionentrypoint.sh ]
    • Update Docker Compose and Makefile

2.3.3 Org Mode Blogging

So with the ideas above , I managed to create some extensions and tricks to enable org mode blogging. It has following parts

  1. Org File Customization

    A small customization is required to export frontmatter from org file. 

    
#+OPTIONS: toc:nil 
#+BEGIN_EXPORT html
---
layout: post
categories: [orgmode]
title: Blogging with Org mode and FastPages
description: Tutorial to setup Fastpages with org mode
comments: true
use_math: false
---
#+END_EXPORT


# COMMENT ------------------OPTIONAL FROM HERE------------------------------


#+TOC: headline 3
    • I use org mode specific publishing framework which comes alongwith Emacs.
    • In my export configuration, which I will talk about later, I have some settings which only export body of converted html document.
    • Here first line is for disabling toc specfic to org mode. Default org mode TOC is added on top of body before content which interferes with YAML Frontmatter
    • You can't include #+TITLE directive in org mode , as it is rendered before everything , and thus interfere's with Front Matter.
    • Then comes section for YAML front matter. Wrapping it in html export block allows this to be the first section on the file.
    • Things below OPTIONAL FROM HERE line need not be included. In fact I am still looking forward to customize css so that we can colorize source code similar to notebooks and markdown

    • With these simple rules, you can easily write a blog post with all the functionality of org mode.
  2. FastPages Customization

    I have done minimal non invasive customization to enable automation for org mode in fast pages.

    Note: All the code described below is available in github repo: https://github.com/Rahuketu86/FastPagesDev
    1. Folder Structure

      We need to create a _org folder . Inside _org folder , it should mirror structure of top level directories as follows

      Structure of _org directory in fastpages
      Structure of _org directory in fastpages

      Note: For local images always assume root to be _org [The structure of _org directory should be replica of blog folder structure]. Images will always start with `/images` or `/assets`
    2. Github Action Customization
      • I have created some custom actions, which I save in _customactionsfiles directory at top level. It has got 3 files

        • publish.el - Emacs script to control publishing from _org/posts to _posts folder and converting org to html. 

          (require 'package)
(package-initialize)
(unless package-archive-contents
  (add-to-list 'package-archives '("org" . "https://orgmode.org/elpa/") t)
  (add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
  (package-refresh-contents))
(dolist (pkg '(org-plus-contrib htmlize))
  (unless (package-installed-p pkg)
    (package-install pkg)))

(require 'org)
(require 'ox-rss)
(require 'ox-publish)
(require 'ox-html)
(require 'fpemacs)

(setq org-html-htmlize-output-type nil)

(setq org-publish-project-alist
	'(

    ("org-post-fastpages"
	    ;; Path to your org files.
	    :base-directory "./_org/_posts"
	    :base-extension "org"

	    ;; Path to your Jekyll project.
	    :publishing-directory "./_posts/"
	    :recursive t
	    :publishing-function org-fp-publish-to-html
	    :html-extension "md"
	    :body-only t ;; Only export section between <body> </body>
      )


      ("org-static-fastpages"
	    :base-directory "./_org/assets"
	    :base-extension "css\\|js\\|png\\|jpg\\|gif\\|pdf\\|mp3\\|ogg\\|swf\\|php"
	    :publishing-directory "./assets/"
	    :recursive t
	    :publishing-function org-publish-attachment
	    )

      ("org-images-fastpages"
	    :base-directory "./_org/images"
	    :base-extension "css\\|js\\|png\\|jpg\\|gif\\|pdf\\|mp3\\|ogg\\|swf\\|php"
	    :publishing-directory "./images/"
	    :recursive t
	    :publishing-function org-publish-attachment
	    )
      ("fastpages" :components ("org-post-fastpages" "org-static-fastpages" "org-images-fastpages"))

  ))

(defun fastpages-publish-all ()
  "Publish the blog to HTML."
  (interactive)
  (org-publish-all))


(provide 'publish)

        • fpemacs.el - Helper functions to enable links functionality translating to html/ jekyll includes. This is really the fastpages shim version of emacs. We can even call it fastpages-emacs library. All the changes should go here

          Warning: This files contains include template in code and can't be rendered fully. Some lines are skipped. 

          (defun jekyll-include-local-img (url caption)
  (let ((n_url (s-lex-format "${url}")))
    (if caption
	  (s-lex-format "<figure>
    <img src=\"${n_url}\"
	   alt=\"${caption}\">
    <figcaption>${caption}</figcaption>
</figure>")
	(s-lex-format "<figure>
    <img src=\"${n_url}\" >
</figure>"))))


(defun embed-img (url caption)
  (cond ((s-starts-with? "/images" url) (jekyll-include-local-img url caption))
	  ((s-starts-with? "/assets" url) (jekyll-include-local-img url caption))
	  (t (jekyll-include-remote-img url caption))))

;;(jekyll-include-img "/images/Emacs.png" "Emacs")

(defun embed-iframe (url)
  (s-lex-format " <div style=\"text-align: center;\">
	<iframe width=\"560\" height=\"315\" src=\"${url}\" frameborder=\"0\" allow=\"autoplay; encrypted-media\" allowfullscreen></iframe>
   </div>"))


(defun get-yt-code (url)
  (car (s-split "&list=" (s-chop-prefixes '("https://www.youtube.com/watch?v=" "https://www.youtube.com/playlist?list=" "https://youtu.be/") url))))

;;(get-yt-code "https://www.youtube.com/watch?v=SmH3BPpl0TI")
;;(get-yt-code "https://www.youtube.com/playlist?list=PLxc79l2wpbJYTI5rv2os7OoKQMqxReZpr")
;;(get-yt-code "https://www.youtube.com/watch?v=SzA2YODtgK4&list=PLxc79l2wpbJYTI5rv2os7OoKQMqxReZpr")
;;(get-yt-code "https://youtu.be/VawlmG9tsXI")


(defun embed-yt(url)
  (if (s-starts-with? "https://www.youtube.com/playlist?list=" url)
	(let ((code (get-yt-code url))
	      (embed-base "https://www.youtube.com/embed/videoseries?list="))
	  (embed-iframe (concat embed-base code)))
    (jekyll-include "youtube.html" (concat "https://youtu.be/" (get-yt-code url)))))

;;(embed-yt "https://www.youtube.com/watch?v=SmH3BPpl0TI")
;;(embed-yt "https://www.youtube.com/playlist?list=PLxc79l2wpbJYTI5rv2os7OoKQMqxReZpr")
;;(embed-yt "https://youtu.be/VawlmG9tsXI")

(org-link-set-parameters
 "yt"
 :export (lambda (path desc backend)
	     (cond
	      ((eq 'html backend)
	       (embed-yt path ))))
 :help-echo "This links helps in exporting link to jekyll youtube liquid template")

(org-link-set-parameters
 "twitter"
 :export (lambda (path desc backend)
	     (cond
	      ((eq 'html backend)
	       (jekyll-include "twitter.html" path ))))
 :help-echo "This links helps in exporting link to jekyll liquid twitter template")

(org-link-set-parameters
 "img"
 :export (lambda (path desc backend)
	     (cond
	      ((eq 'html backend)
	       (embed-img  path desc))))
 :help-echo "This links helps in exporting link to jekyll liquid image template")

(org-link-set-parameters
 "alert"
 :face '(:foreground "red" :underline t)
 :export (lambda (path desc backend)
	     (cond
	      ((eq 'html backend)
	       (jekyll-include-box "alert.html" "text" (or desc path)))))
 :help-echo "This links helps in exporting link to jekyll alert template")


(org-link-set-parameters
 "info"
 :face '(:foreground "blue" :underline t)
 :export (lambda (path desc backend)
	     (cond
	      ((eq 'html backend)
	       (jekyll-include-box "info.html" "text" (or desc path)))))
 :help-echo "This links helps in exporting link to jekyll info template")


(org-link-set-parameters
 "warning"
 :face '(:foreground "pink")
 :export (lambda (path desc backend)
	     (cond
	      ((eq 'html backend)
	       (jekyll-include-box "warning.html" "content" (or desc path)))))
 :help-echo "This links helps in exporting link to jekyll warning template")


(org-link-set-parameters
 "important"
 :face '(:foreground "yellow")
 :export (lambda (path desc backend)
	     (cond
	      ((eq 'html backend)
	       (jekyll-include-box "important.html" "content" (or desc path)))))
 :help-echo "This links helps in exporting link to jekyll important template")

(org-link-set-parameters
 "tip"
 :face '(:foreground "green")
 :export (lambda (path desc backend)
	     (cond
	      ((eq 'html backend)
	       (jekyll-include-box "tip.html" "content" (or desc path)))))
 :help-echo "This links helps in exporting link to jekyll tip template")


(org-link-set-parameters
 "note"
 :face '(:foreground "light blue")
 :export (lambda (path desc backend)
	     (cond
	      ((eq 'html backend)
	       (jekyll-include-box "note.html" "content" (or desc path)))))
 :help-echo "This links helps in exporting link to jekyll note template")

  (org-link-set-parameters
  "iframe"
  :export (lambda (path desc backend)
      (cond
	  ((eq 'html backend)
	  (embed-iframe path ))))
  :help-echo "This links help in embedding iframe and revealjs presentation")


(setq org-html-htmlize-output-type nil)

;; Define A new backend for fastpages html export

(defun org-fp-code-folding (block)
  (s-lex-format "<div class=\"cell border-box-sizing code_cell rendered\">
    <details class=\"description\">
      <summary class=\"btn btn-sm\" data-open=\"Hide Code\" data-close=\"Show Code\"></summary>
	<p>
	   <div class=\"input\">
		${block}
	  </div>
	</p>
    </details>
</div>"))

(defun org-fp-html-src-block (src-block contents info)
  "Transcode a SRC-BLOCK element from Org to HTML.
CONTENTS holds the contents of the item.  INFO is a plist holding
contextual information."
  (if (org-export-read-attribute :attr_html src-block :textarea)
	(org-html--textarea-block src-block)
    (let ((lang (org-element-property :language src-block))
	    (caption (org-export-get-caption src-block))
	    (code (org-html-format-code src-block info))
	    (label (let ((lbl (and (org-element-property :name src-block)
				   (org-export-get-reference src-block info))))
		     (if lbl (format " id=\"%s\"" lbl) ""))))
	(org-fp-code-folding
	(if (not lang) (format "<pre class=\"example\"%s>\n%s</pre>" label code)
	  (format
	   "<div class=\"org-src-container\">\n%s%s\n</div>"
	   (if (not caption) ""
	     (format "<label class=\"org-src-name\">%s</label>"
		     (org-export-data caption info)))
	   (jekyll-highlight lang code)))))))

;;	 (format "\n<pre class=\"src src-%s\"%s>%s</pre>" lang label code))))))

(defun org-fp-inline-src-block (inline-src-block _contents info)
  "Transcode an INLINE-SRC-BLOCK element from Org to HTML.
  CONTENTS holds the contents of the item.  INFO is a plist holding
  contextual information."
  (let* ((lang (org-element-property :language inline-src-block))
	   (code (org-html-fontify-code
		  (org-element-property :value inline-src-block)
		  lang))
	   (label
	    (let ((lbl (and (org-element-property :name inline-src-block)
			    (org-export-get-reference inline-src-block info))))
	      (if (not lbl) "" (format " id=\"%s\"" lbl)))))
    (jekyll-highlight lang code)))



(org-export-define-derived-backend 'fastpages 'html
  :menu-entry
  '(?f "FastPages Export Backend"
	 ((?A "As HTML Buffer (Fastpages)" org-fp-export-as-html)
	  (?a "As HTML file (Fastpages)" org-fp-export-to-html)))
  :translate-alist '((inline-src-block . org-fp-inline-src-block)
		       (src-block . org-fp-html-src-block)))



;;;###autoload
(defun org-fp-export-as-html
  (&optional async subtreep visible-only body-only ext-plist)

  (interactive)
  (org-export-to-buffer 'fastpages "*Org FP HTML Export*"
    async subtreep visible-only body-only ext-plist
    (lambda () (set-auto-mode t))))

;;;###autoload
(defun org-fp-convert-region-to-html ()

  (interactive)
  (org-export-replace-region-by 'fastpages))

;;;###autoload
(defun org-fp-export-to-html
  (&optional async subtreep visible-only body-only ext-plist)


  (interactive)
  (let* ((extension (concat
		       (when (> (length org-html-extension) 0) ".")
		       (or (plist-get ext-plist :html-extension)
			   org-html-extension
			   "html")))
	   (file (org-export-output-file-name extension subtreep))
	   (org-export-coding-system org-html-coding-system))
    (org-export-to-file 'fastpages file
	async subtreep visible-only body-only ext-plist)))

;;;###autoload
(defun org-fp-publish-to-html (plist filename pub-dir)

  (org-publish-org-to 'fastpages filename
			(concat (when (> (length org-html-extension) 0) ".")
				(or (plist-get plist :html-extension)
				    org-html-extension
				    "html"))
			plist pub-dir))

(provide 'fpemacs)

        • custom.sh - Used to run emacs in batch mode with above files 

          #!/bin/sh

echo "Inside custom converter code"
# emacs --batch --no-init-file --load ./_custom_action_files/publish.el --funcall toggle-debug-on-error --funcall fastpages-publish-all
rm -rf /root/.org-timestamps
emacs --batch --no-init-file --load ./_custom_action_files/fpemacs.el --load ./_custom_action_files/publish.el --funcall toggle-debug-on-error --funcall fastpages-publish-all

      • Additionally I have defined a public docker image encapsulating emacs and other dependencies for add on which is available from dockerhub as rahuketu86/fastpages-emacs. I add an addtional step to convert org files in my ci.yaml to enable this conversion automatically on git push. My modified ci now looks as follows 

        name: CI
on:
  push:
    branches:
      - master # need to filter here so we only deploy when there is a push to master
  # no filters on pull requests, so intentionally left blank
  pull_request:
    branches:
      - '!cms**'

jobs:     
  build-site:
    if: ( github.event.commits[0].message != 'Initial commit' ) || github.run_number > 1
    runs-on: ubuntu-latest
    steps:

    - name: Copy Repository Contents
      uses: actions/checkout@master
      with:
	persist-credentials: false

    - name: setup directories for Jekyll build
      run: |
	rm -rf _site
	sudo chmod -R 777 .

    - name: convert org files
      uses: docker://rahuketu86/fastpages-emacs
      if: github.event.deleted == false
      with:
	args: ./_custom_action_files/custom.sh

    - name: convert notebooks and word docs to posts
      uses: ./_action_files

    - name: Jekyll build
      uses: docker://hamelsmu/fastpages-jekyll
      with:
	args: bash -c "gem install bundler && jekyll build -V --strict_front_matter --trace"
      env:
	JEKYLL_ENV: 'production'

    - name: copy CNAME file into _site if CNAME exists
      run: |
	sudo chmod -R 777 _site/
	cp CNAME _site/ 2>/dev/null || :

    - name: Deploy
      if: github.event_name	== 'push'
      uses: peaceiris/actions-gh-pages@v3
      with:
	deploy_key: $
	publish_dir: ./_site
    3. Finally for local viewing

      • I had to update docker-compose.yml to include an orgconverter service to use my rahuketu86/fastpages-emacs 

        version: "3"
services:
  fastpages: &fastpages
    working_dir: /data
    environment:
	- INPUT_BOOL_SAVE_MARKDOWN=false
    build:
      context: ./_action_files
      dockerfile: ./Dockerfile
    image: fastpages-dev
    logging:
      driver: json-file
      options:
	max-size: 50m
    stdin_open: true
    tty: true
    volumes:
      - .:/data/

  orgconverter:
    working_dir: /data
    image: rahuketu86/fastpages-emacs
    volumes:
      -  .:/data/
    command: _custom_action_files/custom.sh


  converter:
    <<: *fastpages
    command: /fastpages/action_entrypoint.sh

  watcher:
    <<: *fastpages
    command: watchmedo shell-command --command /fastpages/action_entrypoint.sh --pattern *.ipynb --recursive --drop

  jekyll:
    working_dir: /data
    image: hamelsmu/fastpages-jekyll
    restart: unless-stopped
    ports:
      - "4000:4000"
    volumes:
      - .:/data/
    command: >
     bash -c "gem install bundler
     && jekyll serve --trace --strict_front_matter"

      • Update Makefile to add a single line in convert [# convert word & nb without Jekyll services] to run orgconverter alongwith nbdev converter 

        help:
	cat Makefile

# start (or restart) the services
server: .FORCE
	docker-compose down --remove-orphans || true;
	docker-compose up

# start (or restart) the services in detached mode
server-detached: .FORCE
	docker-compose down || true;
	docker-compose up -d

# build or rebuild the services WITHOUT cache
build: .FORCE
	chmod 777 Gemfile.lock
	docker-compose stop || true; docker-compose rm || true;
	docker build --no-cache -t hamelsmu/fastpages-nbdev -f _action_files/fastpages-nbdev.Dockerfile .
	docker build --no-cache -t hamelsmu/fastpages-jekyll -f _action_files/fastpages-jekyll.Dockerfile .
	docker-compose build --force-rm --no-cache

# rebuild the services WITH cache
quick-build: .FORCE
	docker-compose stop || true;
	docker build -t hamelsmu/fastpages-nbdev -f _action_files/fastpages-nbdev.Dockerfile .
	docker build -t hamelsmu/fastpages-jekyll -f _action_files/fastpages-jekyll.Dockerfile .
	docker-compose build 

# convert word & nb without Jekyll services
convert: .FORCE
	docker-compose up orgconverter
	docker-compose up converter

# stop all containers
stop: .FORCE
	docker-compose stop
	docker ps | grep fastpages | awk '{print $1}' | xargs docker stop

# remove all containers
remove: .FORCE
	docker-compose stop  || true; docker-compose rm || true;

# get shell inside the notebook converter service (Must already be running)
bash-nb: .FORCE
	docker-compose exec watcher /bin/bash

# get shell inside jekyll service (Must already be running)
bash-jekyll: .FORCE
	docker-compose exec jekyll /bin/bash

# restart just the Jekyll server
restart-jekyll: .FORCE
	docker-compose restart jekyll

.FORCE:

2.3.4 Trying it yourself

Tip: If you quickly want to try above setup yourself. Please follow following instruction
  • Create a Repository by using fastpages template repository as documented here. Please click on this link to start generating your repository from template.
  • After you finish the setup process by generating and storing ssh keys, clone your blog repository to local computer.
  • Clone or copy FastPagesDev Repository to a different folder.
  • Copy _org folder from FastPagesDev repository to your blog repository
  • Copy _customactions folder from FastPagesDev repository to your blog repository
  • Copy Makefile, docker-compose.yml and .github/ci.yml to your blog repository. This will overwrite the files in your blog repository.
  • Now you are ready. You can either build your repository locally or commit your code and see it working on published gh-pages

3 Conclusion and way forward

3.1 Org Mode Bridge and Other systems

Org mode on emacs is a really powerful tool for text processing, writing and literate programming. With all the development done on this repository; it is now fairly easy to get started with org mode on fastpages. In future similar techniques can be adopted to create exporter for Rmarkdown or literate-julia or any other target. In view of author, it is important to use the language native to programming system to build these extensions. So for Rmarkdown , it would be useful to leverage existing infrastructure in "knitr" and "bookdown/ blogdown" packages rather than building things from scratch or extending nbdev. This is less burdensome to maintain and enables a writing flow where we don't have to go back and forth from existing system.

Now even though lot of functionality is enabled in orgmode bridge to fastpages, which mirrors or sometimes extends notebook version there are a few pending issues

3.2 Outstanding Tasks

  1. TODO Fix links inside boxes in org mode
  2. TODO Provide more controls on code folding , currently implemented on every block.
  3. TODO Define a specific layout for org files which can include some common header files.
  4. DONE Define a mechanism for code highlighting
    • It can be done using custom css like org.css. But thing approach need some customization to match with default styles
    • Another approach is to use highlight.js and minor customization to read code blocks. This may be more reliable way to approach this

    Note: Best Solution was to define a derived export backend and wrap source block in highlighting liquid template. It is now implemented in fpemacs.el
  5. TODO Handle internal links and html export more robustly
  6. TODO Enable org-reveal functionality to directly generate and intergrate reveal-js presentation in blog post
  7. TODO Create a demo for exporting to beamer and intergrating with blog post

    Tip: Some custom function to streamline above might need to be included in fpemacs.el

    I hope you find this blog post informative to get started with org mode.Please share your suggestions below which can help in improving or extending this setup.