Rendering djot markup to HTML

support
1

I found this discussion, but it veered off on a detail and didn’t answer the original question.

The author of pandoc created a markup language called djot. It’s very like Markdown, but well-formed and based on a proper BNF description. It differs in some of the syntax, and has a number of features missing from Markdown. One of the tools for working with it is a pandoc plugin, which means pandoc can do the djot → anything pandoc can do conversion.

I’d like to have Hugo be able to process my .dj files. I assumed the pandoc reader used pandoc for processing, but the configuration options for the command are sparse.

Is there any way to configure hugo to tell it how to convert a file with a given extension to … something else that it can already work with? I assume Hugo has some internal document model so that it can run templates on them; I would think if I could configure Hugo to run pandoc with the djot plugin and generate whichever the most rich representation Hugo understands, that would be all that’s needed.

I’m currently preconverting down to markdown, but am losing features and it’s an extra step; I’d like my content source to be stored in djot, not intermediate markdown.

Is this at all possible with Hugo?

2

Which features of djot do you need that are not present in pandoc’s enhanced Markdown syntax?

I’m not asking about differences between the two; I’m asking about the features you need.

You can test your .dj files by adding this to your site configuration:

[mediaTypes.'text/pandoc']
suffixes = ['pandoc', 'pdc', 'dj']

That configuration tells Hugo to process .dj files as if they were using pandoc’s enhanced Markdown syntax.

3

It’s not a question of features alone; it’s a question of familiarity. djot semantics differ from markdown; a single asterisk denotes strong text; underscore denote underline. I use djot everywhere else, and it’s a question of cognitive load of having to remember which syntax I’m writing in, or being surprised at the rendering. Definition lists are an entirely different, and incompatible format; you can’t write djot definition lists and have them understood by markdown or Pandoc’s markdown – djot is the opposite syntax of markdown:

: Life

   Ibi nullus timet mortem sed pro baccho mittunt sortem

: Duty

   Dulce et decorum est pro patri mori

Markdown is the other way. djot supports comments, which can be done in markdown by resorting to HTML, but is more clumsy and fails if you try to generate anything other than HTML (hugo may not be the only destination for a document). Inline document comments are something I do use quite a bit.

My request isn’t based on feature set; you can do just about anything in markdown, if you resort to embedding HTML. I’m asking about supporting a transitive document format – transitive in that it’s what I’ve become used to, believe is superior, and happens to have features defined that translate to more than just HTML. Much of Markdown’s features are tightly coupled to HTML output, and pandoc’s extensions inherit this.

4

I asked this question because many of the features in djot are present in pandoc’s enhanced Markdown syntax.

5

Of course; entirely reasonable. djot shares much with pandoc markdown, as much as pandoc markdown shares with commonmark, and commonmark shares with asciidoc, and the dozen other plain text markup languages. It is, however, substantially different both in semantic meaning of some markup, feature set, formal correctness, and features. It’s not surprising that the author of pandoc markdown would produce a very similar markdown when he eventually decided to create a formal BNF-based markup language.

Most significantly, djot does not support embeded HTML, whereas pandoc markdown and markdown both rely heavily on it for CSS and any features missing from the spec. djot was designed to be able to be converted to other outputs, and to more closely align with the old email ASCII defacto standards we used to use before HTML.

My objective is to be able to use djot, the markup language I use elsewhere, for Hugo. pandoc markdown may have similar syntax and more feature overlap than plain markdown, but it’s not the same language.

6

This isn’t a comment about the merits of supporting djot… just some notes for future reference.

  1. In July 2024 John MacFarlane stated:

    I still think of it [djot] as a new experimental thing. I think there are still a number of issues that I’d want to resolve before declaring any kind of stability.

    https://github.com/jgm/djot/discussions/310#discussioncomment-10182767

  2. The only Go implementation that I can find is sivukhin/godjot. It doesn’t appear to be used in many projects yet, and the author includes this disclaimer:

    This implementation passes all examples provided in the spec but can diverge from original javascript implementation in some cases.

    Additionally, a performance comparison of the available implementations suggests that this package has not been optimized (it’s comparatively slow).

  3. Although we can shell out to v3.1.12 or later of the pandoc executable, the performance won’t be great, presumably similar to shelling out for .pandoc files.

  4. I’ve searched through this forum and our GitHub issues, and could not find any other references to the djot format. That doesn’t mean that this is a bad idea; it just means that no one else has asked for it.

7

These are all fair comments and justifiable reasons why to wait. There are many markup formats, but github made git dominant and it made markdown dominant and anyone using anything else is an edge case. It’s chicken/egg thing; if Hugo, github, Sourcehut – if the publishing platforms supported djot, more people might start using it. Since nothing does, fewer people do.

While John says it’s experimental, the website says that

Djot isn’t completely stable yet, minor future changes to the syntax are expected. The official JavaScript implementation is complete and ready for use.

which sounds a lot more “done” than the other verbiage; I’d argue that markdown was widely used while still being in flux for years, and it didn’t stop people from supporting it. That’s probably something John’s leery of, because it caused a lot of fragmentation in markdown – however, the djot website itself makes the spec sound more complete.

Yeah, that’s the only Go implementation. It’s something I think is worth digging into – not for you folks, but for the djot community. Being the person he is, and the goals he has for djot, if there are implementations which diverge but both follow the spec, it’s something that needs to be addressed in the spec. Are you worried that Hugo would get bug reports? Does the Hugo project have a category of “experimental” features, whereby users could be redirected to the godjot project if the issue is a divergence?

The speed issue is… an issue. I don’t have an answer for that, although if there’s a chance to get Hugo to support djot, I’d allocate optimization time to the godjot project.

Again, it comes down to chicken/egg, which is your point #4. If there is more software supporting the format, the format gets more exposure and becomes an option for more users and the ecosystem improves. I guess a part of my request is advocacy, but it’s sincerely born out of a desire for convenience.

Shelling out to pandoc was an X:Y question, not a recommendation. I was asking whether it was possible. I’d much rather see Hugo integrate godjot and render natively – maybe as a compile-time option, or an experimental feature flag, or whatever mechanism the Hugo project uses.

If PRs would be accepted, I was willing to dig into it. I know there’s an ongoing maintenance concern, and if there’s a way to sumbit a PR for experimental features, I’d happily go that route. What I didn’t want is to spend all that time and effort only to get a “wontfix” rejection; maintaining parallel forks is, IME, a royal PITA, eventually. So if it’s something I can maintain via PRs that ultimately get merged, but have to be enabled at compile time or via flag, I’ll put that in my stack of projects to maintain.

Is Hugo open to this as an experimental feature? Does Hugo have a process for that?

8

FYI,

All I can say is that over the past 2-3 years, there has been about the same amount of change in djot as in commonmark, and less change than in GFM. Neither has a 1.0 release with an implied guarantee of stability. Both are at a “pretty stable” stage, where further changes are likely to be minor. That said, I want to leave open the possibility of further changes to djot, because I want to get it right.

@jgm on the djot github discussion