Hello,
Does this command hugo --buildDrafts rebuild the whole site? meaning all markdown files under “Content” folder will be generated as html files in public folder.
Is there any way to build the new content and the modified contents only?
Hugo does not yet do incremental builds. While it is something we are exploring for the future, for virtually all of Hugo’s users, the speed at which Hugo can build entire sites is fast enough that you won’t notice.
Thanks! @spf13
It took 7 sec to build 584 pages on my machine ((processor: Intel® Xeon® CPU E5-2637 v3 @ 3.50GHz (8 CPUs), ~3.5GHz, Memory: 32768MB RAM (32 GB))). I can feel the wait.
Is there any rough estimated date on when we can get the incremental builds? Thanks!
λ hugo --buildDrafts
2 of 2 drafts rendered
0 future content
584 pages created
0 paginator pages created
6 tags created
42 categories created
in 7513 ms (7.5 sec)
Run the following command and see what you get:
hugo --buildDrafts --stepAnalysis
Can you describe what your site contains? Is it public? Do you have a lot of media or syntax highlighting?
I am trying to move my wordpress blog to hugo.
Here http://michaelsync.net is my blog URL.
I have the generated markdown files exported using wordpress-to-hugo exporter. (I am currently updating that plugin to support the static comments generation as well but that’s a side story. )
Here is the output of analysis.
D:\Lab\hugo\testsite
λ hugo --buildDrafts --stepAnalysis
initialize & template prep:
10.001ms (12.0012ms) 0.25 MB 4207 Allocs
load data:
0 (12.0012ms) 0.01 MB 62 Allocs
import pages:
4.810481s (4.8224822s) 248.66 MB 610315 Allocs
build taxonomies:
2.0002ms (4.8254825s) 0.31 MB 9692 Allocs
render and write aliases:
0 (4.8254825s) 0.00 MB 0 Allocs
render and write taxonomies:
188.0188ms (5.0145014s) 25.21 MB 81542 Allocs
render & write taxonomy lists:
0 (5.0145014s) 0.01 MB 356 Allocs
render and write lists:
24.0024ms (5.0385038s) 5.04 MB 12856 Allocs
render and write pages:
2.4042404s (7.4437443s) 456.87 MB 170120 Allocs
render and write homepage:
4.0004ms (7.4477447s) 0.96 MB 3805 Allocs
render and write Sitemap:
20.002ms (7.4687468s) 4.17 MB 78372 Allocs
2 of 2 drafts rendered
0 future content
584 pages created
0 paginator pages created
6 tags created
42 categories created
in 7463 ms
D:\Lab\hugo\testsite
Screenshots
@michaelsync That’s very helpful. There are some odd things happening. We would typically expect the import pages: step to be a lot LOT faster.
For comparison here is my stepAnalysis on my blog rendered from my 4 year old macbook pro. While you have more content, we have about the same number of rendered pages. Mine renders in about ~400ms.
initialize & template prep:
45.166956ms (45.773928ms) 1.36 MB 14710 Allocs
load data:
32.049µs (45.897804ms) 0.00 MB 44 Allocs
import pages:
49.476296ms (95.466022ms) 31.38 MB 192815 Allocs
build taxonomies:
11.603739ms (107.14171ms) 0.86 MB 32825 Allocs
render and write aliases:
4.429µs (107.208732ms) 0.00 MB 0 Allocs
render and write taxonomies:
201.160422ms (308.437016ms) 110.91 MB 936612 Allocs
render & write taxonomy lists:
9.471792ms (317.992803ms) 1.09 MB 28742 Allocs
render and write lists:
22.535408ms (340.603391ms) 6.70 MB 49069 Allocs
render and write pages:
78.426078ms (419.106219ms) 29.59 MB 481939 Allocs
render and write homepage:
8.381462ms (427.550826ms) 3.49 MB 11553 Allocs
render and write Sitemap:
10.04333ms (437.66197ms) 1.28 MB 30911 Allocs
0 of 12 drafts rendered
0 future content
150 pages created
0 paginator pages created
272 tags created
22 topics created
in 398 ms
What is most telling about this is that operations that are taking so long (10x longer than expected) are all IO heavy operations.
What disk is this running on? What if you just copied the files directly? What if you run hugo server with the 0.15 release which while it will still read from disk, it will write to memory? That would be very telling if the write operations decreased significantly.
Do you have a lot (or large) of media files in /content – there is an issue in that department that is fixed in Hugo 0.15 (released today!)
I think I am using ATA HDD. The benchmark for HDD is as below.
I don’t understand. What I did is that I exported my wordpress by using exporter. It gave me the zip file that contains all media and converted markdown file for all posts and pages. I copied all markdown files to “Content” folder of hugo. All of the images are hosted on Azure CDN so I changed all images path pointing to CDN as well. All zip files (sample projects) are hosted on my github repo.
What files do you want me to copy directly?
I tested with hugo 1.5 but its a bit slower than 0.14. Please check the screenshot below.
I have a link to zip file (sample project) and some images in each posts. but All images are hosted on Azure CDN and my post are pointing to that images even those the original images are still in my wordpress.
So you have ~500 zip files in /content?
1 Like
oh … you are right. I deleted it now and its a lot faster now. Thanks @bep
but still new version is a bit slower than old version 0.14.
hugo_0.15_windows_amd64
hugo_0.14_windows_amd64
BTW, back to my first question. Is there any est: timeline to support the incremental build?
No…
Thanks!