Content adapters: examples and performance

I can confirm the performance boost on Windows 11 from 26 to 8 Seconds.
looks like the Defender does not care about running stuff in WSL2, so here it’s 7 seconds

thx for the fast addition to the docs

I would recommend to use the absolute path to instead of only “hugo.exe” (like “c:\bin\hugo.exe”)

1 Like

That’s … remarkable …

I understand the intent, but our user base has a wide range of experience and expertise. A user may have no idea where the executable resides, particularly if installed with Scoop, Chocolately, or winget.

And explaining how to use where.exe isn’t going to happen.

See the Microsoft Defender Antivirus exclusion documentation.

1 Like

Please let me explain my concerns:

  • For the ones lets say “just get it run”.
  • You know how we all read documentation :wink:
  • non native speakers

I have a proposal at the end. Maybe you consider that valuable for the Docs. I’m not a native speaker… hope it’s understandable

Let’s go to the referenced site from Microsoft:

at the top a prominent statement on the general impact:

Caution: Adding an exclusion to Windows Security means that Microsoft Defender Antivirus will no longer check those types of files for threats, which could leave your device and data vulnerable. Make sure you really want to do this before you proceed.

and at the process exclusion another one

Tip: It’s recommended that you use the full path and file name to exclude a specific process. This makes it less likely that malware could use the same filename as a trusted and excluded process and evade detection.

The Hugo section on that:

Virus scanners are an essential component of system protection, but the performance…

And later:

Before building a site, your virus scanner has already evaluated the files in your project directory. Scanning them again while building the site is superfluous. To improve performance, add Hugo’s executable to your virus scanner’s process exclusion list.

and a brief “How to for Defender”

This could be easily taken as:

Just do it - It’s safe

skipping the discussion for why and what could be insecure…

So I would rather add something like

Virus scanning

Be vigliant

Virus scanners are an essential component of system protection. Fiddling around with these settings may harm your overall system security.
So only do this if you are aware of the impact and are willing to take the risk.

Applications like Hugo that frequently read and write to disk may be heavily affected by the permanent scanning of the accessed files. For example, with Microsoft Defender Antivirus, build times for some sites may increase by 400% or more.

Before building a site, your virus scanner has already evaluated the files in your project directory. Scanning them again while building the site is superfluous.

Most Virus scanners provide options to skip dedicated folders or applications while scanning. You could for example create an exclusion for the hugo binary executable.

  • the example for Defender here

We recommend to use the full path to the executable to lower the security impact.

Other virus scanners my have similar exclusion mechanisms. For details please refer to your Virus Scanner documentation.

A user may have no idea where the executable resides, particularly if installed with Scoop, Chocolately, or winget.
And explaining how to use where.exe isn’t going to happen.

imho especially for those it is a good idea to exaggerate a little.

If you feel strongly about it create a PR in the docs repository and we’ll consider it.

Having said that, I’d prefer to point users to the MS documentation. For example, add a single admonition:

To understand the implications of this change, see the HYPERLINK documentation.

Or something similar.

I do :slight_smile:

On a 14 year old desktop, 16GB RAM, Ubuntu 22.04 and a usually speedy fiber connection:

Start building sites … 
hugo v0.126.1-3d40aba512931031921463dafc172c0d124437b8+extended linux/amd64 BuildDate=2024-05-15T10:42:34Z VendorInfo=gohugoio


                   |  EN    
-------------------+--------
  Pages            | 20021  
  Paginator pages  |  1999  
  Non-page files   |     4  
  Static files     |     1  
  Processed images |     4  
  Aliases          |     7  
  Cleaned          |     0  

Built in 45377 ms

That dropped to 26397 ms on second rebuild.

That’s not bad at all.

2 Likes

Awesome feature, thank you! :innocent:
Is it possible to sort dynamic pages by weight also?
I’m trying to pass on weight as a parameter, but it doesn’t seem to affect the order:

  {{ $content := dict "mediaType" "text/markdown" "value" $json_page.content }}
  {{ $params := dict "weight" $json_page.weight }}
  {{ $page := dict
    "content" $content
    "kind" "page"
    "params" $params
    "path" $release
    "title" $release
  }}

You need to set it on the top level

{{ $page := dict
    "content" $content
    "kind" "page"
    "weight" 42
    "params" $params
    "path" $release
    "title" $release
  }}

We have a strict(er) separation of user defined params (=> params) and Hugo keywords (top level) compared to front matter.

1 Like

This works perfectly. Many thanks Bjørn! :slight_smile:

2 Likes

6 posts were split to a new topic: Customizing the Relearn theme

anyone else have some cool “content adapters” examples? i would love to see live sites using this, for inspiration.

i am really excited to find out about this…my mind is immediately going to ideas for using Hugo as a data plotting/visualization tool.

i was actually just doing something where i took a CSV of 100 entries, then made a python template script to fill out each page’s .md. if only i had reviewed the latest and greatest first!

3 Likes

I also used to build md pages from csv files in Python and this new feature looks amazing. I am curious if can be used with csv files.

Yes you can, see transform.Unmarshal | Hugo

1 Like

I’ve tried to change the path to the pages using this concatenation of variables, but the path is still getting the values just from the .title

There is something wrong with my code ?

{{/* Add page. */}}
  {{ $content := dict "mediaType" "text/markdown" "value" .description }}
  {{ $dates := dict "created_at" (time.AsTime .created_at) }}
  {{ $params := dict  "id" .id }}
  {{ $custom_path := printf "%s-%s" .id .title }}
  {{ $page := dict
    "content" $content
    "dates" $dates
    "kind" "page"
    "params" $params
    "path"  $custom_path
    "title" .title
  }}
  {{ $.AddPage $page }}

Your code looks correct, but are you sure .id has a value?

yes, they have numbers inside the json where the data is pulled from

"id": 624,

If your .id is a number, you might want to use a corresponding format in the printf.

Is strange that even if I hard code it with a random value, the path is still the same {{ $custom_path := printf "%s-%s" "123123" .title }}

This works great:

{{ $custom_path := printf "%v-%s" .id .title }}

Try it:

git clone --single-branch -b hugo-forum-topic-49830 https://github.com/jmooring/hugo-testing hugo-forum-topic-49830
cd hugo-forum-topic-49830
rm -rf public && hugo && tree public
2 Likes