Built-in `highlight` function adds whitespace (new lines)

Jura · June 5, 2019, 1:44pm

I normally include code with ~~~ in my Markdown files, but sometimes use the highlight function for the line numbers. But I noticed that on Hugo 0.55.6 (Extended, Windows 64-bit) the output differs.

Behaviour of `~`

Example:

~~~tv
//@version=3
study(title="Pivot indicator", overlay=true)

// Input options
pivOffsetA = input(title="Pivot Offset A", type=integer, defval=7)
pivOffsetB = input(title="Pivot Offset B", type=integer, defval=15)
~~~

Generates this HTML:

<div class="highlight"><pre class="chroma"><code class="language-tv" data-lang="tv"><span class="c1">//@version=3</span>
<span class="nf">study</span>(title<span class="kp">=</span><span class="s">&#34;Pivot indicator&#34;</span><span class="p">,</span> overlay<span class="kp">=</span><span class="kc">true</span>)

<span class="c1">// Input options</span>
pivOffsetA <span class="kp">=</span> <span class="nf">input</span>(title<span class="kp">=</span><span class="s">&#34;Pivot Offset A&#34;</span><span class="p">,</span> type<span class="kp">=</span><span class="nv">integer</span><span class="p">,</span> defval<span class="kp">=</span><span class="m">7</span>)
pivOffsetB <span class="kp">=</span> <span class="nf">input</span>(title<span class="kp">=</span><span class="s">&#34;Pivot Offset B&#34;</span><span class="p">,</span> type<span class="kp">=</span><span class="nv">integer</span><span class="p">,</span> defval<span class="kp">=</span><span class="m">15</span>)
</code></pre></div>

Notice how the // Input options comment is rendered as a single line.

Here’s how it looks on the webpage:

2019-06-05_15-41-30

Totally good looking .

Behaviour with `highlight`

{{<highlight tv>}}
//@version=3
study(title="Pivot indicator", overlay=true)

// Input options
pivOffsetA = input(title="Pivot Offset A", type=integer, defval=7)
pivOffsetB = input(title="Pivot Offset B", type=integer, defval=15)
{{</highlight>}}

Generates this HTML:

<div class="highlight"><pre class="chroma"><code class="language-tv" data-lang="tv"><span class="c1">//@version=3
</span>
<span class="nf">study</span>(title<span class="kp">=</span><span class="s">&#34;Pivot indicator&#34;</span><span class="p">,</span> overlay<span class="kp">=</span><span class="kc">true</span>)

<span class="c1">// Input options
</span>
pivOffsetA <span class="kp">=</span> <span class="nf">input</span>(title<span class="kp">=</span><span class="s">&#34;Pivot Offset A&#34;</span><span class="p">,</span> type<span class="kp">=</span><span class="nv">integer</span><span class="p">,</span> defval<span class="kp">=</span><span class="m">7</span>)
pivOffsetB <span class="kp">=</span> <span class="nf">input</span>(title<span class="kp">=</span><span class="s">&#34;Pivot Offset B&#34;</span><span class="p">,</span> type<span class="kp">=</span><span class="nv">integer</span><span class="p">,</span> defval<span class="kp">=</span><span class="m">15</span>)
</code></pre></div>

Notice how the comments now have an implicit newline character after them.

As a result, the output on the webpage is also incorrect:

2019-06-05_15-41-41

Question: How do I change/fix this behaviour?

(I looked at the docs and seem to be using highlight correctly.)

zwbetz · June 5, 2019, 1:47pm

If you inspect that element in DevTools, what extra CSS do you see applied to it? Am wondering if the highlight shortcode adds more CSS.

Jura · June 5, 2019, 1:55pm

Good point!

The problem also shows when I remove the CSS files:

2019-06-05_15-55-21

zwbetz · June 5, 2019, 2:15pm

So I created a sample project to test this and cannot reproduce what you’re getting.

My layouts/index.html

{{ .Content }}

My content/_index.md

---
title: Home
---

## With highlight shortcode

{{<highlight tv>}}
//@version=3
study(title="Pivot indicator", overlay=true)

// Input options
pivOffsetA = input(title="Pivot Offset A", type=integer, defval=7)
pivOffsetB = input(title="Pivot Offset B", type=integer, defval=15)
{{</highlight>}}

## With markdown code fences

```tv
//@version=3
study(title="Pivot indicator", overlay=true)

// Input options
pivOffsetA = input(title="Pivot Offset A", type=integer, defval=7)
pivOffsetB = input(title="Pivot Offset B", type=integer, defval=15)
```

My config.toml

baseURL = "http://example.org/"
languageCode = "en-us"
title = "My New Hugo Site"

pygmentsCodefences = true
pygmentsStyle = "pygments"

Then running hugo server on version 0.55.6 gives me

Jura · June 6, 2019, 1:14pm

Thanks for taking the time to test this on your end. Really appreciate it!

Unfortunately, I cannot repeat your outcome. When I make a clean website project, I use these config settings:

baseURL = "http://example.org/"
languageCode = "en-us"
title = "My New Hugo Site"

theme = "theme2019"

pygmentsCodefences = true
pygmentsStyle = "pygments"

Then I put this in example.md:

With `~~~`:

~~~tv
//@version=3
study(title="Pivot indicator", overlay=true)

// Input options
pivOffsetA = input(title="Pivot Offset A", type=integer, defval=7)
pivOffsetB = input(title="Pivot Offset B", type=integer, defval=15)
~~~

With the highlight function:

{{<highlight tv>}}
//@version=3
study(title="Pivot indicator", overlay=true)

// Input options
pivOffsetA = input(title="Pivot Offset A", type=integer, defval=7)
pivOffsetB = input(title="Pivot Offset B", type=integer, defval=15)
{{</highlight>}}

The result that I get is:

2019-06-06_15-04-59

So there’s still unwanted whitespace after comments when I use highlight. It happens on Firefox and with Chrome.

I’ve uploaded the test website that showed the above problem here: https://github.com/Jos512/hugo-issue-highlight

(Btw, I’m on the same Hugo version as you.)

Hugo Static Site Generator v0.55.6/extended windows/amd64 BuildDate: unknown
GOOS="windows"
GOARCH="amd64"
GOVERSION="go1.12.2"

zwbetz · June 6, 2019, 3:51pm

Interesting. It still looks fine for me when running your repo.

17%20AM

Here’s my hugo env.

$ hugo env
Hugo Static Site Generator v0.55.6-A5D4C82D2/extended darwin/amd64 BuildDate: 2019-05-18T08:06:24Z
GOOS="darwin"
GOARCH="amd64"
GOVERSION="go1.12.2"

Also, I’m using Chrome.

Jura · June 6, 2019, 4:42pm

Very odd. But I don’t think it’s a browser issue, because Chrome on my side has the same problem:

2019-06-06_18-36-25

For what it’s worth, when I switch from extended to ‘regular’ Hugo the same problem appears.

I tested that with this version:

Hugo Static Site Generator v0.55.6-A5D4C82D windows/amd64 BuildDate: 2019-05-18T07:57:00Z
GOOS="windows"
GOARCH="amd64"
GOVERSION="go1.12.2"

This one seems identical as yours (although A5D4C82D and A5D4C82D2 do differ one letter), and you have a operating system it seems. But that shouldn’t matter right with Hugo?

zwbetz · June 6, 2019, 4:44pm

Hmm. My current hunch is that there is a difference in Windows vs Mac for this particular thing.

zwbetz · June 8, 2019, 2:27pm

Thinking about this some more, see if any of these make a difference:

try in both Chrome and Firefox
if you’re using browser extensions, try opening the browser in incognito mode (or the equivalent of that)
try cloning your repo on another machine with a different OS, installing a linux VM, or installing Windows Subsystem for Linux

Jura · June 11, 2019, 11:38am

Thanks for the reply.

I tried point 1 and 2 and found no effect. Point 3 is unfortunately outside my technical skills and resources too.

So I’m hoping on another Windows user to see this thread and with some free time on his or her hands.

jkboxomine · June 12, 2019, 6:10am

The same issue occurs on my Windows PC with the following env:

Hugo Static Site Generator v0.54.0/extended windows/amd64 BuildDate: unknown
GOOS="windows"
GOARCH="amd64"
GOVERSION="go1.11.5"

I’m not sure if this persisted over time or was brought in with some change.

Jura · June 12, 2019, 1:09pm

Thanks for confirming jkboxomine!

I’ve made an issue here on GitHub to track the problem.

Thanks everyone again for their help.

jkboxomine · June 13, 2019, 1:38pm

@Jura,

it’s regrettable that I did not find this earlier, but the issue is probably not related to Hugo or Chroma at all.

Check the newline characters in the MD file, and if it’s CRLF, try changing it to LF only. (simple task with tools like Visual Studio Code) This solved the issue in my environment.

Jura · June 17, 2019, 2:37pm

That’s an amazing find! That solves the problem on my end too. Thank you!

zwbetz · June 17, 2019, 2:43pm

Agreed, nice find @jkboxomine

jkboxomine · June 18, 2019, 6:04am

No problem. Glad it helped.

I am still not sure which of Hugo or Chroma is susceptible to CRLF processing; I’ve tried to set up Pygments instead of Chroma in an attempt to narrow down the cause, but timeout occurred during build on my code-heavy MD files, thus to no avail.

Jura · January 9, 2020, 1:13pm

Is it perhaps possible to make Hugo render Chroma-based code in a platform agnostic manner?

It seems that with Goldmark now the default standard, the previous issues I had with the highlight shortcode affect regular code fences too.

This is confusing and extra work for me and the people I work with, also because it doesn’t affect all languages.

JavaScript syntax, for instance, seems to run fine under CRLF. But Python not so that requires LF. But Go language is highlighted properly either way. But then again C# is not.

Topic		Replies	Views
Using {{- .Inner -}} still adds newlines support	11	6490	April 27, 2018
Pygments howto support	6	2927	December 3, 2014
Syntax highlighting example code - please check [local problem] support	3	978	June 3, 2015
Empty lines in C++ syntax highlight blocks support	4	836	February 11, 2020
None code with bold or italic in syntax highlighter support	2	701	January 17, 2021

Built-in `highlight` function adds whitespace (new lines)

Behaviour of ~

Behaviour with highlight

Related topics

Behaviour of `~`

Behaviour with `highlight`