Issue with generating site - missing posts

Kaicastledine · May 25, 2018, 11:04pm

Alright so weird one here…

I had a hugo site setup locally which both the source and public are hosted on github.
Not added any content since end of last year so wanted to add a few pages.

I’ve started “Hugo server” locally and loads of pages are not being rendered… only about 5 out fo 30.

They’ve not being shown in any of the source code in the index.html either.

So thinking that my local copy of the site has issues I went and git cloned the hugo source code from my live working site which in the public folder you can see the correct pages being generated in the index.html.

I then ran hugo server in that site and same issue appears and when you run “hugo” to generate the new site … It overwrites the correctly generated one with the problematic one.

I’ve then gone back through various versions/commits on github of my site which all have worked previously (can be shown not only on the live site but the commits/versions of the Public folder repo which is hosted)

I then thought this was some local hugo issue on my new machine … repeated the process on a linux VM… same issue.

I have no idea how what was working source code for a site which is even working hosted now … Does not work despite whatever commit I roll back to.

The source code for the site is here and you can see the commits
https://github.com/KCSEC/IVWSource

And the Public folder linked by submodule here which is the live hosting

https://github.com/KCSEC/IVOIDWARRANTIES.github.io

I thought this was some baseurl issue when locally viewing it with hugo server … but when generated it doesn’t show the right content either.

Any ideas ?

Kaicastledine · May 25, 2018, 11:06pm

You can see in the hosted version the index.html shows as follows,
https://github.com/KCSEC/IVOIDWARRANTIES.github.io/blob/master/index.html

but if you git clone the source code
https://github.com/KCSEC/IVWSource

most of the content does not generate at all yet this was and still is the unchanged code that generated the currently live and shown site in the IVOIDWARRANTIES.github.io

Kaicastledine · May 25, 2018, 11:25pm

This may actually be not related … but it seems to only show posts dated within the 2017 period when running “hugo/hugo server”.

For bad use of ordering posts … set dates to made up years (temp thing).

Kaicastledine · May 25, 2018, 11:39pm

Rolled back to an older hugo version 0.30 - linux 64bit and ran -v --debug on the current source code

I actually shows all of the pages being read as source

/hugo server -v --debug
INFO 2018/05/26 00:34:49 Using config file: /root/Documents/IVW/IVWSource/config.toml
INFO 2018/05/26 00:34:49 /root/Documents/IVW/IVWSource/static/ is the only static directory available to sync from
INFO 2018/05/26 00:34:49 syncing static files to /
DEBUG 2018/05/26 00:34:49 Load I18n from [%!q(*source.Filesystem=&{[] /root/Documents/IVW/IVWSource/i18n [] {0xc420373020 0xc420073980 map[] en}})]
WARN 2018/05/26 00:34:49 No translation bundle found for default language "en"
WARN 2018/05/26 00:34:49 Translation func for language en not found, use default.
WARN 2018/05/26 00:34:49 i18n not initialized, check that you have language file (in i18n) that matches the site language or the default language.
DEBUG 2018/05/26 00:34:49 Load templates from path "/root/Documents/IVW/IVWSource/layouts" prefix ""
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/404.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/404.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/_default
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/_default/list.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/_default/list.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/_default/single.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/_default/single.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/index.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/index.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/partials
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/partials/footer.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/partials/footer.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/partials/head.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/partials/head.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/partials/header.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/partials/header.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/partials/js.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/partials/js.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/partials/off-canvas.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/partials/off-canvas.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/partials/pagination.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/partials/pagination.html
DEBUG 2018/05/26 00:34:49 Template path /root/Documents/IVW/IVWSource/layouts/partials/post-list.html
DEBUG 2018/05/26 00:34:49 Add template file from path /root/Documents/IVW/IVWSource/layouts/partials/post-list.html
Started building sites ...
DEBUG 2018/05/26 00:34:49 Load Data from 1 source(s)
DEBUG 2018/05/26 00:34:49 Read 33 pages from source
DEBUG 2018/05/26 00:34:49 Reading from posts/About/About.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Events/TechXLR8-2017.md
DEBUG 2018/05/26 00:34:49 Reading from posts/LAB/Metasploitable2/Metasploitable2M.mdown
DEBUG 2018/05/26 00:34:49 Reading from posts/Events/EventsList.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Events/IFSEC-2017.md
DEBUG 2018/05/26 00:34:49 Reading from posts/LAB/Metasploitable3/Metasploitable3M.mdown
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Metasploit/UsingMetasploit-ReverseShell.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/NetCat/NetCatCheatSheet.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/LinuxEnumExploit/kernelpop.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Metasploit/MSFVenom/Payload Generating.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Nmap/Nmap.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Pivoting/Basics.mdown
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Pivoting/LocalPortForwarding.mdown
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Nmap/NmapCheatsheet.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Nmap/NmapNSE.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Pivoting/meterpreter.mdown
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/ReverseShells.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/TheFatRat/FTREasyPayloads.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Pivoting/proxychains.mdown
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Pivoting/sshuttle.mdown
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/TheFatRat/TFRBackdoorEXE.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Website Pentesting/sqlmap.mdown
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/WindowsEnumExploit/netbiosX-SC.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Website Pentesting/XSS.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/Website Pentesting/nikto.md
DEBUG 2018/05/26 00:34:49 Reading from posts/Pentesting Tuts/WindowsEnumExploit/netbiosX-WKE.md
DEBUG 2018/05/26 00:34:49 Reading from posts/RFID-NFC-Research/ImplantSeries.md
DEBUG 2018/05/26 00:34:49 Reading from posts/RFID-NFC-Research/Implants.md
DEBUG 2018/05/26 00:34:49 Reading from posts/RFID-NFC-Research/ImplantProjects -KeyCard-Access.md
DEBUG 2018/05/26 00:34:49 Reading from posts/RFID-NFC-Research/ImplantProjects-Computer-Unlock.md
DEBUG 2018/05/26 00:34:49 Reading from posts/RFID-NFC-Research/MyImplants.md
DEBUG 2018/05/26 00:34:49 Reading from posts/RFID-NFC-Research/proxmarksetup.md
DEBUG 2018/05/26 00:34:49 Reading from posts/RFID-NFC-Research/proxmarktags.md

but reports the following,

Built site for language en:
0 of 5 drafts rendered
0 of 23 futures rendered
0 expired content
6 regular pages created
42 other pages created
0 non-page files copied
1 paginator pages created
11 tags created
6 categories created
total in 18 ms

Not sure why but the current brew install hugo version on mac when running -v --debug doesn’t show it loading the page sources at all.

But does report the following

+------------------+-----+
  Pages            |  29  
  Paginator pages  |   0  
  Non-page files   |   0  
  Static files     | 132  
  Processed images |   0  
  Aliases          |   1  
  Sitemaps         |   1  
  Cleaned          |   0

budparr · May 25, 2018, 11:52pm

Looks like a bunch of your content is dated far into the future:

Kaicastledine · May 26, 2018, 12:01am

Yeah this theme sorts itself by the date, so bodged it by messing with the post dates just to get the correct order on the posts.

I’ve got a project to redo a lot of the code and remove how it sorts the posts… even potentially a new theme entirely.

Is this for some reason now the cause of the issue ?

On the live site it hosts it with all the pages regardless of the ridiculous dates.

I’ll have to run a few tests and see what’s gone on.

Is there a way for it to build with all dates present past and future or to bypass any date requirements or specific ranges etc ?

RickCogley · May 26, 2018, 12:34am

Do hugo --help and look at the switches --buildXXX. You can get it to build drafts or future posts etc.

Kaicastledine · May 27, 2018, 6:52pm

Well done you’ve nailed it!

Any idea how this has only started happening ?

Last time the site was edited was nov 2017, new feature or will it be one of those weird mysteries haha?

OwenFM · September 4, 2021, 1:01pm

After suffering the same problem, it turns out the issue was that I’d set the date for the post to be a few hours in the future.

Upon changing the date, everything was working fine again.

Topic		Replies	Views
Site is working in local server but not after build support	5	2362	June 11, 2019
[SOLVED] About page not being generated for GitHub Pages support	9	1578	October 8, 2018
The Website that I uploaded is not rendered same as the one I saved in my localhost support	4	930	February 2, 2022
Issue with viewing /posts on Hugo site, GitHub pages support	5	1376	June 14, 2019
Generation of pages fails on Gitlab site support	0	414	April 8, 2021

Issue with generating site - missing posts

Related Topics