Hugo with Dropbox and iPhone <long>

tips & tricks
1

Hugo with Dropbox and iPhone

I thought I would do a long overdo write-up of how I use Hugo with Dropbox and an iPhone to create blog posts and update my site. I have integrated 2 Hugo sites with a Ghost site of travelogues, but this approach has nothing to do with Ghost at all–I just linked the three sites into one with a navigation menu. I’ll also point out that git is not used in this approach at all (though there is nothing wrong or especially challenging about git.)

Just check it out at (https://a-view.org). In the top menu, the links for “Florida Coast” and “New England Streams” take you to Hugo blogs. You can skip around the Ghost site and go directly to Hugo with (https://a-view.org/floridacoast) and (https://a-view.org/nestreams) --this one is just started so doesn’t have many posts yet.

To author a blog in Hugo, there are several things to decide about the mechanics, leaving out your choice of theme and other design decisions:

  1. You need a place where you enter and edit blog posts, including placing your images.
  2. You need a place where you run Hugo to generate the static pages of your blog.
  3. You need a place on the internet where you host the static pages generated by Hugo.

There are a couple of considerations that influence these choices. I wanted to be able to author blog posts while on the go, using my iPhone and including pictures I’d taken with the iPhone. I wanted to be able to do this with mobile internet, which varies a lot in speed depending on where I was traveling. Here is the key: I didn’t want to sync the generated web site resources each time I created a post. Though Hugo is terrifically fast and only updates/generates changes, the result is the ENTIRE site. I don’t want to have to synchronize stuff that has not changed. It makes sense to only synchronize the new post I’ve authored because it is comparatively tiny. What this means is that places number 2 and 3 need to be the same place.

This means that the blog posts need to get to the web hosting service where I host my Hugo web site. Therefore, I must run Hugo on my web host’s server. This is no problem: the server is quite secure and only the www directory is exposed to the internet. Hugo runs easily on Linux. I can enter the Hugo command via my iPhone because it is only one line so this works even if the connection is very slow.

You may be wondering where place 1 is–where I edit blog posts–and how those blog posts get to place 2–where I run Hugo to generate the site. Place 1 is on my phone–just local files in markdown along with local photos. I use an iPhone app that syncs with Dropbox. There are quite a few and because a single blog post, possibly with a small size photo, is quite small the sync from my phone to my Dropbox doesn’t require much bandwidth at all. Next, I need to get new posts from Dropbox to my web host’s server. Turns out Dropbox has a Linux client. So, the Dropbox client is synchronizing from Dropbox’s servers to my web host’s server. This is server-to-server across the fast commercial internet and has nothing to do with my iPhone’s connectivity. Finally, I just need to be able to run Hugo on the web server to update the static website.

Be sure that in no way are we hosting the generated, finished Hugo website on Dropbox. Once upon a time you could get away with that, but Dropbox will block you now. You wouldn’t want to do that anyway as you’d have to sync the generated Hugo site across the internet to Dropbox. You wouldn’t get to have your own URL for the website. So, no–the website will NOT be hosted on Dropbox. Only you will sync the content you author to your own Dropbox account. Now that you understand the why and basics of the approach I’ll explain the details.

I will present the details at a bit of a high level. Your Dropbox and your webhost are going to be a bit different so I can’t describe exact commands with exact directories. I also don’t describe any of the steps to configure your Hugo website. Other than a couple of key points, I don’t describe how to setup a web server. There is nothing special for Hugo in using this approach. Using the 3 places above, here is what you do.

Place 1: Dropbox and phone

I use my laptop to create a folder in Dropbox called site-content. Within folder site-content you will have a sub-folder for each Hugo website. I have one called “nestreams”. Within your site folder you will have the template for your site including all of the folders required for Hugo: archetypes, content, data, layouts, resources, static, themes, and the config.toml file.

What do you need to do on your phone? I’ll keep this simple and assume you won’t make any on the fly changes to your theme, config, and you won’t create pages, only posts. I use an app called 1Writer. For all of the buzz about various markdown clients (all overblown in my opinion), I think 1Writer is the best. You can sync multiple folders from Dropbox to your iPhone. You only want folders that are absolutely needed and relatively small. So, I setup 1Writer to sync the folder Dropbox/site-content/nestreams/content/post. For your setup, replace nestreams with the name of the folder for your Hugo blog authoring. This is great: I create and edit a post and it will sync to Dropbox. If I get access to my laptop, I can author a post there and it will show up on my phone later to edit, if I wish.

There is more to do on the phone that I will describe as part of place 2, running Hugo, because we need to be able to access place 2 from the iPhone.

Place 2: running Hugo to generate pages of your site

This gets a little bit trickier. 1) We need a webhost that lets us install some software and run it, as well as host a website. Typically, a low-end shared hosting site, of which there are thousands, won’t let you install any software you want. We need a virtual private server (or VPS) where we have control of the server. I’ve used Linode and Digital Ocean with total satisfaction. There are many more. 2) We need to install and run Dropbox here. 3) We need to install and run Hugo here. 4) We need a web server (not going to say a lot about this–you need this for any usage of Hugo). 5) We need to actually run Hugo from time to time to add new posts or edits to old posts to the website.

The VPS

Let’s use Digital Ocean as an example. You’ll get an account and create a VPS. Digital Ocean has “droplets,” which are pre-configured Linux servers with essential software pre-installed. This makes life easy. I used a droplet with nginx (a web server), Let’s Encrypt (tools for installing and managing certificates so you can use https instead of http), and pretty much every other standard web tool commonly used. You will need telnet or ssh access. You’ll be using a commandline terminal to access your vps–I’ll just say “server” from now on. In effect, you have your own Linux server “box” through the magic of virtualization. It’s better to use ssh than telnet–telnet involves using a login and id and password to access the server–you probably don’t have any state secrets in your blog, but it’s better not to tempt hackers. So, set up ssh. Digital Ocean, Linode, and others have cookbooks for all of this but you need to know your way around the commandline and some basic Linux commands.

On your iphone you’ll need a terminal emulator to access this server. People like apps such as Termius, Shelly, or Blink. These are all beyond just fine. But, it’s nice to also be able to edit code files so I use Textastic. Does a lot more than the terminal or ssh shells and has a built-in terminal. You don’t need to do much. We’re not administering the entire server on an iphone–you can… …but we’ll do the big things on a laptop. All we need for Hugo is to be able to connect to the Hugo authoring directory (with the new posts synced from the iphone) and to run Hugo itself. This takes something like 5 seconds each time.

Installing Dropbox

There are some tricky aspects to this, but this is a one-time job. Dropbox has a super simple commandline utility (no GUI–this is not a laptop) that does the bare minimum you need to do. Dropbox suggests a python script for managing Dropbox. You want that. Your Linux distro on your server will need to have Python 3 installed. Any that you install today will. I just followed Digital Ocean’s instructions here: (https://www.digitalocean.com/community/tutorials/how-to-install-dropbox-client-as-a-service-on-ubuntu-14-04). As with all things Linux, no instructions are completely correct. I have a path to the dropbox.py command and only need to type dropbox. The command generates great help text and you hardly need anything.

Here is the rub. I have a HUGE Dropbox account and only a tiny VPS. When the dropbox client starts running, it will probably hang because it runs out of disk space. Just start it again and start pruning away directories you don’t need (nearly everything). As a smart alternative, create a new free Dropbox account. You probably won’t need much storage. Then, just link an empty account.

To prune your dropbox, you’ll need to do this:

dropbox exclude add "dumb folder" "baby pics" ...

You are adding folders to the exclude list. At the end, you should only be syncing the folder site-content/mysite/content/post.

The only commands I find I need are:

dropbox status  # tells whether dropbox is running and if it is up to date
dropbox start   # start dropbox if it stopped for some reason--mine's been
                  running for years without ever stopping
dropbox autostart  # do this once so that if your server is restarted for 
                     maintenance--it will be--dropbox restarts

If you’ve done this correctly, dropbox will start syncing the few folders needed for your Hugo site. I find that Dropbox syncs a new post about as fast as I can complete it on my iphone.

Installing Hugo

This also is a bit tricky, or perhaps unfamiliar. Your Linux distro might have an install package (apt on Ubuntu) for Hugo, but it will inevitably be out-of-date. So, don’t use that. snap is a package manager for many different distros of Linux. The good Hugo folks keep their snap package of Hugo up-to-date.

So, you need to install snap. snap was pre-installed on Ubuntu 16.04 so I didn’t have to do anything. If you are not so lucky, follow the instructions here: (https://docs.snapcraft.io/installing-snapd). It’s just a one-liner.

Then, use snap to install Hugo:

snap install hugo

I am a bad boy and manage the server as root. That’s bad, but since I am using ssh, I hope I get away with it. You’d normally sign is as a user with lower privileges and run sudo snap install hugo assuming you are a sudoer user (beyond this discussion…).

There are many cool things about snap. One is that it keeps the apps it manages up to date so you don’t have to–it’s automatic. Second, it provides a sandbox for files that snap-installed apps need to use. This avoids the entire privileges complexity on linux and let’s you keep your file system pretty much all locked up. This is very important: this means that Hugo will generate your completed static website into the snap sandbox. The directory will be something like /var/snap/hugo/common. Each snap managed app gets its own “safe” directory. When you tell hugo the target directory for publishing your finished website, it will create the needed directory the first time and then it will just update.

This is the line you need in your config.toml (using the name you want for your websites directory):

publishDir      = "/var/snap/hugo/common/sites/nestreams"

Place 3: Web Server and access to the finished hugo static website

This is where the server and website configuration will diverge the most across users. I’ll describe the key things I did for nginx, but you’ll have to adapt this to the environment you are using.

The first thing we have to do is symlink a directory that is published to the web with the target where our generated hugo site is published. For nginx on ubuntu, we want static pages to be served from a directory at /var/www/. To do that:

cd /var/www
ln -s /var/snap/hugo/common/sites/nestreams nestreams

Now, you have a symlink that behaves just like a real dictory at /var/www/nestreams. And this will be the base url of your hugo website that you must set in config.toml as

baseurl = "https://a-view.org/nestreams/"

Of course, use the name you’ve chosen not nestreams.

We have to tell nginx that this is a website with static files. I have a url for a website called “a-view.org”. I am not going to describe how to set that up; I am only going to show the modification to have the hugo website appear as a subdirectory of this url. nginx enables you to setup sites-available, which might include some inactive websites or experiments, and sites-enabled, which are actually served to the web by nginx. In my case there is a complication that has nothing to do with hugo. I have a series of travelogues that I created with the blog platform Ghost. At some point I decided Ghost was overkill (more below), so I decided to graft in Hugo websites. If you only have Hugo, you would edit config files for nginx in /etc/nginx/sites-available. If you have enabled https and have certificates for https and also allow http then you’d create two config files: a-view.org-ssl.conf and a-view.org.conf. Of course, your domain won’t be a-view.org. In my case, I put these config files in a directory managed by Ghost and symlinked twice: once to sites-available and again to sites-enabled. But, that is a detail.

Here is what both files look like with the addition needed for Hugo:

server {
    listen 80;
    listen [::]:80;

    server_name a-view.org;
    root /var/www/a-view/ghost/system/nginx-root;

    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $http_host;
        proxy_pass http://127.0.0.1:2368;

    }

    location ~ /.well-known {
        allow all;
    }

    location /floridacoast/ {
        alias /var/www/static-test/;
    }

    location /nestreams/ {
        alias /var/www/nestreams/;
    }

    client_max_body_size 50m;
}

And the ssl config file:

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    server_name a-view.org;
    root /var/www/a-view/ghost/system/nginx-root;

    ssl_certificate /etc/letsencrypt/live/a-view.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/a-view.org/privkey.pem;
    include /etc/nginx/snippets/ssl-params.conf;

    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $http_host;
        proxy_pass http://127.0.0.1:2368;
        
    }


    location /floridacoast/ {
        alias /var/www/static-test/;
    }

    location /nestreams/ {
        alias /var/www/nestreams/;
    }
    
    location ~ /.well-known {
        allow all;
    }



    client_max_body_size 50m;
}

You can see that all I added was this:

location /nestreams/ {
        alias /var/www/nestreams/;
    }
Blogging and running Hugo

Phew. A lot of setup. Let’s author a post. Because I like to enclose images with the blog post where the images will appear, rather than in a central directory I use this structure:

~/Dropbox/site-content/nestreams/content/post/the ride/
            index.md
            The_Oxbow.jpg

This creates a post that will be called “the ride”. The markdown is obviously in the file index.md and I’m using the .jpg as the header image for the post. I just create index.md with 1Writer on my Iphone or I could use a text editor on my laptop. As soon as I save it, the iPhone app will sync it with Dropbox. Then, the other Dropbox clients, including the one we setup on our server, poll Dropbox for changes and receive (and/or send) changes. For a couple of blog posts, even with images, this happens faster than you’re reading the description–or maybe as much as 3 seconds if your phone has a slow connection. There is nothing to do to sync the content. You write it and Dropbox propagates it. Done.

Now, we have to log into our server and run Hugo. Remember, I am using the Textastic app on my iPhone. I have a little text scrap in a local file of what I have to enter in the terminal (below). I select it and copy it. Then, I ssh into my server and paste it. Done.

The text scrap is:

dropbox status   # look for the reply Up to date
cd Dropbox/site-content/nestreams
/snap/bin/hugo

After all that setup, I just write a post, move an image into the folder (if any), login, and run hugo. It really is like 5 seconds.

That’s really all there is to it. Hope this was useful. Of course, it works just as well from a laptop where you have more tools.

Postscript: Why Ghost? Why Hugo?

I started out using WordPress to create blogs. WordPress is possibly the greatest piece of open source software ever created, if you exclude browsers. It’s great. In its early days it was sometimes a bit complicated and themes needed hand-tweaking. So much has improved.

I saw Ghost and I liked the looks of the final result and the editor:

  • some really nice themes, so the finished site looks great
  • a very simple editor and site manager
  • markdown text to create and edit posts

After converting some of my WordPress blogs to Ghost and using it self-hosted for new blogs, I realized that for my use there were some downsides:

  • It’s still complicated and overkill for simple “read-only” blogs. I don’t need SEO (no one reads my blogs except friends and family members). I don’t do interactive form-driven commercial blogs. Ghost holds content in a database, generates the content, caches it (so it’s sort of like a static generator from a database), and serves it.
  • It’s a bear to install and maintain. They won’t say that, but at the same time they heartily recommend their hosting service.
  • You need Mysql, Let’s Encrypt, special Ghost command line tool that is always getting in trouble, and carefully managed directory privileges (of course, you always do…). You need to upgrade Ghost itself, which hasn’t always gone smoothly though it has always worked in the end.

Realizing that my blogs were just stuff that normal people put on Facebook, I decided to try static site generators. I am Python handy so I tried Pelican. I never fell in love with it and I was still in my WordPress stage. After Ghost, I heard about Hugo. I had tried IOS update of Ghost blogs and it sort of works, but only sorta because the IOS Safari browser is not a good target for lots of heavy javascript. Not Ghost’s fault, but an obstacle for the Ghost browser based client (which is as nice as it gets). So, I thought Hugo would do the job. I saw that a friendly volunteer had created a dead-ringer for the default Ghost theme, called Casper (which I like a lot). It was a bit hard to work through the somewhat cookie-cutter options for syncing and hosting that Hugo provides. But, when I realized what Hugo did–and because I self-hosted Ghost at Digital Ocean, where I could install and run arbitrary code, I realized some of the canned approaches were making actual Hugo usage seem more complicated than necessary. I worked out the Dropbox approach largely on my own. The description of the setup above is perhaps more involved than it needs to be. Once you realize where things need to go in various directories, the whole thing comes down to 2 symlinks. And then it’s just easy. The good news is that you automatically get a backup of your authored content because it is saved in Dropbox. Your website is fast and super-easy to maintain. You give up some stuff if you were a commercial blogger, but I am not. Other than some difficulty modifying the theme I chose, it’s been duck soup.

8 Likes