Dynamic API-driven content

The title is cut from the Hugo home page as a feature of Hugo.

I’m wondering if there are established patterns / best-practices for accomplishing this.

I am looking for a publishing platform to create an online book. The first several chapters will be free, and you’ll need to create a paid account to access the rest of the book.

I love the culture and technology of Hugo and have used it already to create a website, but am unsure if Hugo really supports my use-case well. Certainly I can have a button on the free portion of the site inviting you to purchase or login to access the rest of the book, but if the rest is also static-content authored with Hugo, it would seem that once you have a URL to this content, you’ll be able to pass on that URL and anyone can access the protected content. So perhaps the paid portion of the book, even if authored with Hugo, will need to be served by a dynamic host (e.g. Node, Rails, etc) that can authorize before delivering content.

Ideas? Examples?

Thanks!
Thomas Blom

1 Like

This is a little beyond my expertise level, but have you seen: https://github.com/netlify/gotrue

I think that “slogan” is slightly misleading, or promising more than it fulfils. I think @rdwatters added that sentence, and should consider changing it.

Hugo is a static generator, so:

  • You can include data from JSON etc. into your content as part of your build.
  • You cannot generate content files based on JSON etc.

Thanks @Jonathan_Griffin. gotrue appears to provide “authentication as service”, and there are similar projects gojoin and gocommerce that provide subscription and one-time-payment proxies to Stripe, all intended for static sites.

So the actual authentication/payment can be done with these technologies. The part I’m struggling to get clear on is how the paid-for content is served only to those persons who have paid for it.

It would seem that this content must necessarily be served by a “dynamic” server, that is, one that first authenticates and then either does or does not deliver the content.

This seems opposite of what static sites want to do, which is to push all their content out on CDNs to deliver quickly with no processing. I think: you can’t put content on a CDN if it requires per-user authentication.

So, since my “paid for” content will necessary live behind a dynamic server, and not on a CDN, I’m wondering if it still makes sense to author with Hugo anyway since I like it, and then deploy the resulting content to a non-static endpoint, which the static portion of the site could request via javascript…

Since I have no experience with this and may be barking up the wrong tree, just wondering if others have experience thinking through this.

Thanks!

Thanks @bep.

Certainly I can include javascript in my static site that makes requests to an API to receive additional content, and use that content to dynamically populate HTML elements on the page.

So this is possible, but now we are inserting dynamic features into our static site, and depending on how much of this is required, one begins to wonder if “static site” is really the right technology.

My book project is technical and will be written by scientists, not developers - so a workflow that is fast, local, authoring in Markdown with support for MathJax, is all ideal - which is why I am looking to Hugo as a possible authoring/publishing technology. But the paid content issue is going to require some dynamic backend.

Thanks!

1 Like

This seems to be somewhat rooted in the definition around “static,” which is harder to define in 2017, but I’m happy to change to whatever you want via issue assigned to me @bep

Maybe “flexible” would be a better term

For me the confusion starts when a technology advertises what is possible vs. what it was designed to do well.

Most devs realize anything is possible on just about any platform/framework/toolchain, it’s just a question of how convenient it is – was this tool really built for this purpose?

To have the home page hero text call Hugo a static-site generator, and then just below have some bulleted key features that includes “dynamic API-driven content” is confusing because these two descriptions seem to be add odds, at least with respect to what the main thrust of Hugo is.

You then get devs like me saying, ok, of course I can do this, as I can from any static site with javascript calls. But is it really meant for this? If so, give me docs and examples that indicate best practices. If not, don’t include it on your home page of top-tier features.

Just my $.02. I love Hugo and the whole culture around it.

1 Like

I believe the phrase in question refers to Hugo’s getJSON/getCSV functionality. Perhaps we should link to it to avoid confusion. But I think you’re asking about server-side functionality rather than dynamic content.

I think companies like Netlify and Aerobatic have some solutions for password protecting portions of your site, and I imagine you already understand the stuff about payments, etc.

It might be worth reaching out to the Netlify community or the Aerobatic community and asking there.

  1. It’s a new site
  2. It’s only 3 words :wink:

https://github.com/gohugoio/hugoDocs/pull/59

Like @budparr would distinguish both. IMHO your submitted change “data-driven content” would refer to data files, while API-driven content is content fetched from an API outside of Hugo, namely with getJSON and co.

The problematic word for many here is “dynamic” – and maybe also “driven”; Hugo is static and if anything drives the page creation it is the content (Markdown) files. There are an open issue somewhere to get Hugo to create pages from data files (i.e. /products/product1) – and with that in place we can maybe talk about “data driven content”; right now it is more “data enriched content”, but that is a bad slogan …

That said, @spf13 is extremely good with getting the terminology just right.

I dont think that static sites are less well suited for dynamic content than other server architectures. There is complexity in working with either pattern, neither method is inherently better at the surface.

I dont think this question has as much to do with hugo as it does with the general ideal of how the client side connects to the various api’s that it needs to populate the various types of data you have throughout your site(s), and there is no inherent relationship between a server side framework and how a client side application accesses various API’s to populate data. In modern web applications all the dynamic magic, all the key security stuff, it all happens on the client side, and, as it relates to your use case, the server side simply feeds javascript, html and css to the client side but all the processing still happens on the client side. In more traditional server-side frameworks, the client side may still build a page where the client goes directly to a google maps api or through a central chokepoint, likewise a server side framework does not force or even typically rely on web-app-user authentication to be handled through the server-side. Where traditional server side frameworks are more substantial is if you are a big company and you are writing a bunch of distributed microservices and you had to think about things like distributing api server configuration and secure keychain data during server-side dynamic scaling events for distributed microservices applications.

Fortunately for those of us that arent doing things that are anywhere close to needing massive scale custom complex apps like that, even for the most giant they use web services available over from public web services and saas companies as much as possible, and whether a client side page directly accesses various API’s to get data to build the page or goes through a central server-side api gateway is simply a matter of how the javascript on the client side is configured, and hugo can serve that javascript just as well as anything else imho.

The most important point here is having a server side framework does not at all change the need to build a client side application. You can write your client side application in a different language that will dynamically build the client side app in html/css/jss or you can work with any number of different ways to get the html/css/js and then serve it in any number of ways, you may find that certain features that other server frameworks may be more compelling to you, but personally I think you will be better off focusing on getting the client side app the way you want it first and then going from there.

For your use case the problem isnt so much that static-site generators cant do what you need as much as there arent as many of these types of templates for hugo but you may end up serving the web app you build in another framework with hugo, the central questions are how you can create and manage secure sessions in a client side web application, and how you provide authentication services and other services which in any case until you are massive scale will make most sense to get as a service and not try to build your own microservices.

For your use case, I would suspect that a Single Page Application (SPA) pattern may be ideal … dont let the name fool you, its not really meant for a single page, I think of the pattern more like a single web application, like for example your website. And the SPA pattern can support your website getting really big like to the point if you had to worry about it again you would have your own team of web devs. The benefit of the SPA is, it maintains availability of all your client object model, script objects and your security posture across all of your various pages, because as you know javascript can make entire pages or directory structures appear and disappear, like for example a page that has multiple tabs may actually be a single page where javascript exposes the text when you click on the different tabs but its actually all one page. So the SPA pattern is actually using javascript to embed an entire web application in the context of an entire page which makes it much easier to do things that involve using scripts to provide user interaction features, produce dynamic content, or just maintain security continuity as the user browses through your content. You could also use a SPA for only a portion of your website, or have some pages serve html that was generated by a SPA framework by hugo and hugo serve other standard static and md generated content, but then you would have to worry about how you would pass objects and security context between different areas of your site and maintaining continuity of appearance between the frameworks and pages. If you can meet all your needs in the context of a SPA and I strongly suspect you can, it will greatly simplify your life and you shoudnt have to think about growing beyond the framwork until your massive enough to have a dedicated web team :slight_smile:

And once you create your SPA, you can still choose to serve it with hugo on github pages, its just html/css/js.