NOTE: My github repo is private, but I’ll someone as collaborator if they’re willing to help.
Hi, my blog has 9,700 pages. It builds fairly quickly, but it takes about 30 minutes to upload and process on Netlify. And I haven’t even migrated most of my images because I’m not sure the deploy would ever end.
How can I speed it up?
It builds super fast locally (20 seconds), and the build time on Netlify isn’t bad. It’s the deploy/processing that takes 20+ minutes.
Here’s my metrics:
cumulative average maximum
duration duration duration count template
---------- -------- -------- ----- --------
5.426369398s 2.102429ms 45.225654ms 2581 _default/list.html
1.817594382s 1.570954ms 27.310659ms 1157 _default/single.html
1.810800586s 469.484µs 23.576344ms 3857 partials/footer.html
1.644528197s 426.374µs 16.803403ms 3857 partials/header.html
1.286260322s 161.834µs 6.493141ms 7948 _default/li.html
1.283920136s 11.068277ms 39.983145ms 116 index.html
1.186835996s 307.709µs 15.551626ms 3857 partials/head.html
1.156802423s 299.922µs 23.45745ms 3857 partials/link.html
853.079161ms 370.742µs 16.22766ms 2301 _internal/_default/rss.xml
573.752401ms 148.871µs 41.873077ms 3854 partials/siteinfo.html
282.682481ms 73.29µs 6.575491ms 3857 partials/menu.html
251.659965ms 217.51µs 7.643296ms 1157 partials/related.html
167.908962ms 43.533µs 9.368193ms 3857 partials/head_custom.html
143.643834ms 143.643834ms 143.643834ms 1 _internal/_default/sitemap.xml
112.865371ms 56.432685ms 97.25894ms 2 _default/terms.html
105.534114ms 39.13µs 5.241093ms 2697 partials/pagination.html
69.960544ms 18.138µs 3.328558ms 3857 partials/search.html
20.915103ms 5.422µs 4.943453ms 3857 partials/svgpack-sprite.html
10.92832ms 2.833µs 1.993562ms 3857 partials/footer_custom.html
1.753666ms 292.277µs 1.344981ms 6 _internal/shortcodes/youtube.html
624.361µs 624.361µs 624.361µs 1 404.html
| EN
+------------------+------+
Pages | 5760
Paginator pages | 398
Non-page files | 47
Static files | 5376
Processed images | 0
Aliases | 2299
Sitemaps | 1
Cleaned | 0
Total in 5909 ms
Thanks!
zwbetz
July 21, 2019, 1:49am
2
Hi, explain more about this. And give Netlify logs as well.
It usually takes only about 2-3 minutes to begin “Starting deploy” Then it crawls. Here’s the Netlify log:
5:17:54 PM: Build ready to start
5:17:56 PM: build-image version: 9e0f207a27642d0115b1ca97cd5e8cebbe492f63
5:17:56 PM: build-image tag: v3.3.2
5:17:56 PM: buildbot version: ef8d0929ed0baabafd8bbb7d0b021e1fc24180c0
5:17:56 PM: Fetching cached dependencies
5:17:56 PM: Starting to download cache of 1.3GB
5:18:06 PM: Finished downloading cache in 9.933509435s
5:18:06 PM: Starting to extract cache
5:18:21 PM: Finished extracting cache in 14.971249202s
5:18:22 PM: Finished fetching cache in 25.89568038s
5:18:22 PM: Starting to prepare the repo for build
5:18:23 PM: Preparing Git Reference refs/heads/master
5:18:52 PM: Found netlify.toml. Overriding site configuration
5:18:52 PM: Starting build script
5:18:52 PM: Installing dependencies
5:18:52 PM: Started restoring cached node version
5:18:55 PM: Finished restoring cached node version
5:18:56 PM: v10.16.0 is already installed.
5:18:57 PM: Now using node v10.16.0 (npm v6.9.0)
5:18:57 PM: Attempting ruby version 2.6.2, read from environment
5:18:58 PM: Using ruby version 2.6.2
5:18:59 PM: Using PHP version 5.6
5:18:59 PM: Installing Hugo 0.55.6
5:18:59 PM: Hugo Static Site Generator v0.55.6-A5D4C82D2/extended linux/amd64 BuildDate: 2019-05-18T08:08:34Z
5:18:59 PM: Started restoring cached go cache
5:18:59 PM: Finished restoring cached go cache
5:18:59 PM: unset GOOS;
5:18:59 PM: unset GOARCH;
5:18:59 PM: export GOROOT='/opt/buildhome/.gimme/versions/go1.12.linux.amd64';
5:18:59 PM: export PATH="/opt/buildhome/.gimme/versions/go1.12.linux.amd64/bin:${PATH}";
5:18:59 PM: go version >&2;
5:18:59 PM: export GIMME_ENV='/opt/buildhome/.gimme/env/go1.12.linux.amd64.env';
5:18:59 PM: go version go1.12 linux/amd64
5:18:59 PM: Installing missing commands
5:18:59 PM: Verify run directory
5:18:59 PM: Executing user command: hugo --gc --minify
5:18:59 PM: Building sites …
5:19:28 PM: | EN
5:19:28 PM: +------------------+------+
5:19:28 PM: Pages | 9683
5:19:28 PM: Paginator pages | 1045
5:19:28 PM: Non-page files | 47
5:19:28 PM: Static files | 5376
5:19:28 PM: Processed images | 0
5:19:28 PM: Aliases | 3316
5:19:28 PM: Sitemaps | 1
5:19:28 PM: Cleaned
5:19:28 PM: | 0
5:19:28 PM: Total in 28485 ms
5:19:28 PM: Skipping functions preparation step: no functions directory set
5:19:28 PM: Caching artifacts
5:19:28 PM: Started saving pip cache
5:19:28 PM: Finished saving pip cache
5:19:28 PM: Started saving emacs cask dependencies
5:19:28 PM: Finished saving emacs cask dependencies
5:19:28 PM: Started saving maven dependencies
5:19:28 PM: Finished saving maven dependencies
5:19:28 PM: Started saving boot dependencies
5:19:28 PM: Finished saving boot dependencies
5:19:28 PM: Started saving go dependencies
5:19:28 PM: Finished saving go dependencies
5:19:28 PM: Build script success
5:19:28 PM: Starting to deploy site from 'public'
5:19:31 PM: Creating deploy tree asynchronously
5:20:31 PM: 7411 new files to upload
5:20:31 PM: 0 new functions to upload
5:26:41 PM: Starting post processing
5:26:54 PM: Starting post processing
5:28:44 PM: Finished processing build request in 10m48.041340993s
5:41:44 PM: Post processing done
5:41:44 PM: Post processing done
5:41:44 PM: Site is live
5:41:44 PM: Site is live
zwbetz
July 21, 2019, 2:33am
4
What netlify post processing options do you have configured?
None.
But, when deploying from netlify deploy using their cli, it looks like a lot of time is spent on CDN diffing and then loading 14,000 pages of tags.
Whoops. That was the first deploy to dummy project. It timed out waiting for site to go live. I made two small changes, rebuilt, and started netlify deploy again. This time, it got through CDN diff and upload in a few seconds, but it’s been churning on Waiting to deploy site for a while again.
zwbetz
July 21, 2019, 3:21am
7
I would recommend contacting Netlify Support. Closing this topic.
2 Likes