Move to AWS

In late 2016 I moved the site from being served from a little VPS+ngnix to S3+CloudFront. The site was basically all that was running on the VPS and why pay $5/month to serve up static files when it could cost pennies. The site now averages about $0.70/month to serve (on a CDN), $0.5/month of which is just hosting the DNS for doshitan.com on Route53. Cheaper and better performance. Win-win.

I went into AWS initially planning on just setting things up by hand, but at the time there was a neat “Host a static website” quickstart thing that sets up the basics for a static site really easily and even provides a nice status page. I started with that.

It wasn’t perfect, had to make some tweaks and add some stuff (like www → root redirecting), but it was cool. I can’t find the slick status page anymore, or a link to the quickstart, so maybe they dropped it.

But managing things through a GUI long term is not something I wanna do, so eventually encoded the setup in some terraform (see /infra/). Importing the existing resources into terraform was pretty easy, props to them for supporting that. At least most of them. The TLS cert for the domain, managed by ACM, was harder to deal with. In general these can be a bit of a pain for config tools because of the verification step, but I could not get terraform to import my existing verified certificate. Not a big deal though, currently using a data source to pick out the right cert, so at least I don’t have to specify the ARN by hand.

Pretty happy with things. I wish I could use a CloudFront Origin Access Identity to restrict access to the S3 bucket so that all requests are forced to flow through CloudFront, but it breaks the S3 website hosting stuff. So if someone manages to guess the S3 website URL, they could side step the CloudFront stuff, which is a) more expensive to me and b) misses out on some security settings.

May setup a shared secret in a header between CloudFront and S3 such that S3 would reject requests without the right value to at least block serving up actual content to good guessers. Might eventually write my own little server to perform the S3 website duties and misc. other things. But I can live with things for now.

Most folks are probably better/more easily served using something like Netlify, but I like doing things myself and my needs are simple.

Better security

While overkill for a static site like this, the move to S3+CloudFront did mean I lost an easy way to specify recommended security headers, most important to me is HSTS. In December 2016/early 2017 AWS launched Lambda@Edge, which allows one to hook into the CloudFront flow and make changes, most importantly for me, it gives us the ability to add these headers back. Feels like a kludge to setup a lambda to set some static headers (honestly Amazon, just provide a way to set static headers in CloudFront¹), but it’s easy enough and I finally got around to setting it up for the site redo. The lambda is very simple.

Also added a Content Security Policy (CSP), but instead of as a header, it’s a HTML <meta> element since it kinda belongs with the content. I could probably be convinced to go either way with it, but it’s in the HTML for now.

See the Mozilla Observatory tests.

Repo organization

Rearranged some things in the repo. Broke apart the structure, with site executable bits in a top-level executable/ dir and the content (posts, pages, etc.) in content/. The nix build setup has also changed to accommodate the new layout as well as being reworked in general. The nix stuff is also locked down to a specific version (should have done that from the beginning).

I’d love to use nix even more to build and publish the site and there is a derivation to build the content, but because I rely on the files timestamps to show when a post or page has last been modified and those are lost/reset during the build that way, I pretty much have to build the site from the repo. Maybe I’ll move the last modified date into a meta field in the files themselves, but for now it’s not that big of a deal.

Dropped Bootstrap and therefore the submodule for it.

Moved from using Google Fonts to just subsetting the text fonts² myself and serving them with all the other site content, which required some build scripts. This was interesting and probably not quite as good as using Google Fonts, but I like having everything self contained.

Also moved from a full FontAwesome to just a subset with fontello, which has a nice setup for automation. See the fontello-* targets in the makefile.

New look

I tend to waffle on about things, so I wanted the site optimized for long-form reading. That has really always been the focus, but I think the new design is better than what I had before, with a bit wider main column for text and sidenotes support. It’s based on tufte-css, though with a good amount of additions and tweaks to make it work better for a whole site instead of just an article.

I’m not using the upstream CSS directly, but rather I ported it to Clay to be easier for me to work with. “Easier to work with” in some ways, being embedded in Haskell is nice, but the Clay files are compiled with the site executable, not by it, so CSS changes take a site rebuild to take effect. It’s easier to set up this way, but I should probably move it to be compiled separately at some point.

Tried to reduce and simplify everything. I kinda want to have the site navigation vertical on the side instead of horizontal across the top. May move the table of contents into the sidenotes area, but I’d need to avoid any sidenotes that are on early content, which is a complication.

I’m not very happy with how code blocks look. The Tufte and Pandoc markups kinda collide. Also not sure I agree with the Tufte approach of not setting off the blocks with a different background. Will need to keep tweaking things.

New markup

The underlying HTML markup is more compatible with h-entry.

Finally set a favicon.

I had two constraints I tried to respect in this whole process 1) only care about modern browsers and 2) no JavaScript. For certain design elements in the future, especially supporting them across all devices, I may relax the second constraint, but if I could get something I liked without JS, then I wanted that. And the first one both helps with that and makes my life easier.

Wrap up

I’d like to go even more IndieWeb in the future with “notes” (think of a tweet, short unnamed content) and more social support (distributed commenting/liking/etc., with things like Webmention and ActivityPub), but those don’t fit well with a static site³ and I like the simplicity of what I’ve got for now.

The goal was to update the site and get it to a place I was happier with. That goal was achieved. Now to finish the slew of half-written posts I’ve accrued.

HTTPS Everywhere

I’ve wanted to enable HTTPS on doshitan.com since I first started the website, but didn’t really want to shell out much money (especially reoccurring costs) for something which should really be free/one-time cost. The leading free option (that I know of) for a long time was StartSSL, which has a slightly better setup, with free 1-year certificates or charging just for identity verification (which actually costs something) and not for every single certificate you want to generate (which really doesn’t cost anything), but I didn’t want to have to remember to login and renew/regenerate a free certificate every year or give a bunch of personal information to an Israeli company to store for years.

In short, I was waiting for Let’s Encrypt to be generally available. I think there were other automated CAs issuing free/cheap DV certificates before Let’s Encrypt, but Let’s Encrypt is more my style (smart, open, principled, and free). With it entering public beta on December 3rd, it was just a matter of finding some time to play with it. I’m happy to say it was fairly painless to get it all set up and with the proper config settings for nginx, doshitan.com now has an A+ on the SSL Labs test (give it a run yourself).

I’ll go through the general setup, but it’s not terribly interesting so feel free to skip.

Let’s Encrypt Setup

First step was getting the Let’s Encrypt client running. I’m currently running from git, though there is a packaged version in the experimental repos for Debian Jessie. As the client stabilizes I’ll move over to the packaged version, but for now I want to be running the latest code and using letsencrypt-auto instead of letsencrypt. So just clone the repo somewhere:

$ git clone https://github.com/letsencrypt/letsencrypt /wherever/

This site is served by nginx, and Let’s Encrypt doesn’t yet have automagical support for nginx like it does for Apache, but all it takes is a few extra command line flags to get it working. Let’s Encrypt certificates are valid for 90 days, so you’ll want to renew them at least every couple of months (~60 days) to be safe. I setup a simple systemd unit to run the requisite commands and a timer unit to trigger it every month, which I’ll move out to every two months after a while. They are as follows:

[Unit]
Description=Generate/renew SSL certificate for example.com

[Service]
Type=oneshot
ExecStart=/wherever/letsencrypt-auto certonly --renew-by-default --agree-tos -a webroot --webroot-path /var/www/example.com -d example.com -d www.example.com --email webmaster@example.com
ExecStart=/bin/systemctl reload nginx

[Unit]
Description=Run letsencrypt-example

[Timer]
OnCalendar=monthly
Persistent=true

[Install]
WantedBy=timers.target

Then enable and start the timer unit:

$ systemctl enable letsencrypt-example.timer
$ systemctl start letsencrypt-example.timer

And that’s it, automatic generation and renewal of a certificate for example.com and www.example.com.

A more general approach

On a bit of a tangent, if you had lots of different sites to generate/renew certificates for, you could break some of this out into template unit with some environment stuff.

[Unit]
Description=Generate/renew SSL certificate for %I

[Service]
Type=oneshot
EnvironmentFile=/etc/sysconfig/%I
ExecStart=/wherever/letsencrypt-auto certonly --renew-by-default --agree-tos -a webroot --webroot-path /var/www/%I --email webmaster@%i.com $OPTIONS
ExecStart=/bin/systemctl reload nginx

[Unit]
Description=Run letsencrypt for %I

[Timer]
OnCalendar=monthly
Persistent=true

[Install]
WantedBy=timers.target

Drop a config file for each domain you’re interested in

$OPTIONS=-d example1.com \
         -d www.example1.com \
         -d sub.example1.com

Followed by a:

$ systemctl enable letsencrypt@example1.com.timer
$ systemctl start letsencrypt@example1.com.timer

If all the sites follow the same pattern (as most do), you could change a couple of the service file lines to something like:

EnvironmentFile=-/etc/sysconfig/%I
ExecStart=/wherever/letsencrypt-auto certonly --renew-by-default --agree-tos -a webroot --webroot-path /var/www/%I --email webmaster@%i.com -d %i -d www.%i $OPTIONS

So no config file is needed if the site only needs the common stuff.

Nginx

We need to allow nginx to read the certificates. For me it’s easiest to just give the www-data user read access with:

$ sudo setfacl -R -m user:www-data:r-x /etc/letsencrypt/archive/ /etc/letsencrypt/live/

A better approach would keep the stuff (or at least the private keys) strictly readable by root only (i.e., the web server starts under root but forks a less privileged process to handle requests and such), but as I only run a static site under nginx, I’m not too worried about something happening that exposes the private key.

After the certificates are readable by nginx, we need to have nginx actually use them. Somewhere in the blocks for both www.example.com and example.com add:

...
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
ssl_trusted_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
...

The rest of the ssl settings are gonna vary depending on your needs, but I generally follow the h5bp config with whatever tweaks I need. An important step is to generate a unique Diffie-Hellman parameter with something like:

$ nice -n19 openssl dhparam -out /etc/nginx/ssl/dhparam.pem

Or:

$ nice -n19 certtool --generate-dh-params --outfile /etc/nginx/ssl/dhparam.pem --bits 4096

Once it eventually finishes, gotta make use of it for each SSL-enabled block in nginx with:

ssl_dhparam /etc/nginx/ssl/dhparam.pem;

Ensure it’s all good:

$ sudo nginx -t

Then:

$ systemctl reload nginx

And now the site is set.

Email

I also setup email for the doshitan.com domain. The hardest part was deciding how I wanted to host the mail server. I considered a couple options.

Google

The easiest option is probably just Google Apps as I’ve already been using Gmail has my primary mail service for years, but I really wanted to detangle from Google. While I’ve never had an issue with their services, they are rather omnipresent and I’d rather not contribute to a proprietary mono-culture if I can help it (which is also part of the reason I primary host my code on GitLab instead of GitHub and use DuckDuckGo as my primary search engine).

Dreamhost

I currently register my domains through Dreamhost, partly for historical reasons, but they are also cheap and reputable, so no real reason to change. Dreamhost offers email hosting for any domain registered through them, but I’ve heard some not so good things about their email services. Mail not getting sent, mail not getting received, etc., so I wasn’t too keen on even trying it out.

Self Hosted

I’ve tossed around the idea of self hosting an email server for quite a while actually as it’s not terribly difficult to get something basic setup and I certainly have all the relevant skills to do so, but there are a lot of downsides to hosting it myself as well. The main downside is simply not getting outgoing mail flagged as spam by other mail services. The other big downside is the maintenance. Self hosting gives you a lot of control, but also a lot of responsibility and potential headache that I don’t really want to have to deal with for something I just want to work.

FastMail

I finally settled on FastMail. They are a dedicated email service that strongly supports open standards and have been around forever (well, since 1999). They also have some support for calendar and contacts that I might give a shot, but their email support is the main focus.

The rest

After the host was decided on, it was just a few simple DNS changes to get the mail flowing. And flowing it is! I’ve been quite happy with the performance of FastMail so far, in both their Web app and Android app. IMAP has been good so far too. Haven’t yet given the calendar or contacts stuff a good run, but I suspect those will be just fine too.

Wrap up

Not complicated stuff, but I’m happy to have it all set up now.

So, seeing as I wasn’t really interested in game development – and all the major “free” game toolkits (Unity, Unreal, etc.) don’t play nice with GNU/Linux or are in a language I don’t enjoy, e.g., Phaser – I decided if I was gonna do it I wanted it to at least be interesting (to me if no one else). That meant choosing some tools I was actually interested in and at least knew enough to get started in them, which boiled down to either Haskell or Elm. I ended up deciding on Elm, mainly because I’ve been wanting to do something with it and this seemed like a good opportunity.

I came up with a few abstract concepts I felt I could have a chance at actually making and either adapt to whatever theme was chosen or just ignore the theme and do my own thing (^_^).

The night before the event started I whipped up a little prototype (if it can even be called that) of one of my ideas just to make sure I had all of the Elm platform working together and got over the initial language speed bumps (“How do I compose functions again? Oh, << or >>, cool.” and so on).

Given the technical restrictions I had decided upon and my attitude towards the whole affair, I was fully prepared to do something on my own; very few people would want to learn a language they’d (likely) never heard of, in a style of programming they’d (likely) never used, with someone who didn’t want to stay up all night and didn’t care about winning.

The first night of the event (a Friday) was mostly spent just hanging out with the folks I knew (including giving a quick into to Haskell to one of them). Towards the end of the night, there was actually a group of folks on board with building out one of my ideas (the one I had “prototyped”) in Elm. More precisely, they were on board with me building the game in Elm and them doing everything else (^_^). I made it perfectly clear that I couldn’t even promise we’d have something deliverable by Sunday! I was pretty confident that we’d have something, but I’d never done a game before and never seriously worked in Elm, so I wasn’t about to have people depend on me to knock it out of the park in 24 hours. The crazy fools went along with me anyway. So we fleshed out the idea a bit and made it in theme for the event. Then we all went home around 10pm.

Started up the next day around 9-10am and quickly got to work. I spent a good amount of time, I think it was like 3-4 hours, getting the gameplay up to the point of my “prototype”, but of course in a more modularized/nicer fashion with a start screen that had buttons and such. The thing that took the longest was just reading the Elm docs about the “Elm Architecture” and trying to map the Mailbox/Address and Signal stuff into working code, threading the stuff through to the now separate Game module. By the end of Saturday, the basic gameplay was pretty much done, a lot of the themeing was in place – thanks to the other members of the team – and we were pretty happy. We all went home by 10-11pm.

Started Sunday again around 9am. We dropped in all the audio, tweaked some of the art, and just tried to polish the rest off. We were to deliver our game for the judge’s to play by noon. It wasn’t long after that the public started coming in and playing all the games. Of course it’s hard to just stop working on the thing, especially when Elm makes it hard to get something running that’s broken, so I kept pushing some updates after the “deadline” (with official approval) incorporating some of the early feedback we were getting. They were just minor things, a small progress bar across the top that gave a sense of how much of the conversation was left to listen to, a tip on the home screen on how to play, getting the game audio to restart on every new game instead of having to reload the app and a slight tweak to some colors. Around 2pm it was our turn pitch our game to the judges and around 4pm they got up and announced the winners for the Community and Judge’s Choice awards¹, among other things. Then we all went home.

We haven’t done much with it since, though we do plan on making it better at some point. I’d like to add some localStorage support to store your scores and other state. In the mockups we had a richer “debriefing” section where you have to answer questions about key points in conversation to actually “win”, that would be neat to implement. More “levels”/situations/conversations would also be good. Having the ring wiggle to the actual sound wave of the conversation would be pretty sweet too, though that’s pretty much a pipe dream, dunno if I really want to dive into implementing that. If the Touch library was well supported in mobile browsers, we’d actually automatically have a mobile game as I made everything scalable from the start, so that would be nice to get figured out.

All in all it was pretty fun. All the code was done by me from scratch starting Saturday morning and the other guys did a great job making/collecting all the other assets and really making the game what it turned out to be. It really would have just been super simple geometric shapes and solid colors if I had done it myself. The atmosphere is what makes the game at all interesting to play and none of that was me.

All the other teams had some fun ideas too and I think everyone really enjoyed the event. If you’re interested, you can take a look at the code and give the game a try from the project page.

Now the discussion here is going to be biased by a couple things:

1. My own personal preferences; I like strong statically typed, functional languages (e.g., Haskell)
2. The needs at work. For the work situation, we need (in my opinion) something that is ‘simple’, not necessarily a shallow learning curve, but rather an execution model that you can understand, easy debugging and help making sure things don’t break. Another factor is the development team at work is pretty small, a couple to a few developers at any given time, so one person needs to be able to accomplish a lot and maintain the product over time; we need something expressive.
3. I haven’t actually used most of these tools. I’m making judgments based on the project’s documentation and the chatter on other sites (HN, Reddit, etc.) from people who have used them.

It should go without saying, but the contents of my personal blog are my own opinions and in no way represent the opinions of my employer. Furthermore, I in no way intended or wish to misrepresent any of the tools in this post, if you find an issue, please get in touch.

Plain JS solutions

AngularJS 1.x

Homepage

AngularJS is a sophisticated framework for building web applications which emphasizes a bi-directional dataflow and testing. It’s one of the oldest of the ‘new’ JS libraries/frameworks and as such, has some inertia to it, though I feel it really only started picking up steam in the past couple of years.

Pros

• Already working for a few things at work
• Lots of drop in components
• Two-way data binding
• Backed by Google
• Emphasis on testability
• Fairly straightforward design patterns to follow (and published guidelines)

Cons

• Can be complicated and fragile (we have already seen this at work)
• Two-way data binding
• Likely large and incompatible 2.x coming within the next year, maybe with migration tools, but likely to be a very different framework (i.e. knowledge/experience gained on 1.x may not directly translate into productivity with 2.x)
• Can have performance issues
  • Is depending on browsers implementing native code (Object.observe) to alleviate some of its problems
  • See this presentation on Object.observe

Flux + React

Flux Homepage • React Homepage

One of the newer kids on the block, React is ‘View’ layer for web applications which utilizes a ‘Virtual DOM’ to build up and swap out parts of the DOM efficiently. Flux is an application architecture (as well as the name of Facebook’s implementation of the Flux architecture) which emphasizes a uni-directional dataflow which works well with React.

Pros

• One-way dataflow
• Generally more performant (compared to AngularJS)
• Backed by Facebook
• Flexibility in implementation/architecture, Flux provides some structure
• React Native

Cons

• More immature ecosystem (compare to AngularJS)
• No single best-practice/do-this architecture to follow, though Flux provides some structure

Misc

• While one may end up having to roll their own solutions to some things, Yahoo is rewriting their mail app with React and are open-sourcing a number of library in the process so the ecosystem is likely to be continue growing with industrial quality components
• In general, the React blog is worth reading, particularly the community round-up posts

More info

Tools are available built around these components:

• Marty.js, HN Discussion
• Fluxxor
• RefluxJS
• ReactFlux

Mercury

Homepage

Pros

• Extremely modular, can use which parts one wants/needs
• Generally very good performance (in contention for title of highest performing library currently available I’d say)
• Flexibility in implementation/architecture

Cons

• Extremely modular
• No single best-practice/do-this architecture to follow
• No big corporate backing?

Mithril

Homepage

Opposite end of the spectrum from Mercury with regards to modularity: Mithril is one file. Its goal is to be small and simple, both in the code itself and the cognitive load required to use it.

Pros

• Very small and ‘simple’
• Very good performance, up there with Mercury
• Flexibility in implementation/architecture

Cons

• I feel it could be a lot of boilerplate/lack of convenient functionality
• No single best-practice/do-this architecture to follow, there are some official suggestions on how to structure an app, but you would largely be developing our your way to structure the code
• No big corporate backing?

Typed JS tools

Plain JS will probably one day get type annotations¹ likely based on whatever syntax/semantics Typescript and Flow can come to agree on since these are the two working tools (that I know of) that have the types sprinkled in otherwise vanilla JS syntax.

Typescript

Homepage

TypeScript is a superset of JS that gives developers a more familiar OOP syntax to use that compiles down to plain JS. It was one of the first typed JS supersets and also brings in ES6/ES 2015-like features (e.g., modules and classes). The forthcoming 1.5 release will bring the compiler to almost complete parity with ES6/ES 2015 and the 1.6 release will start pulling in ES7 features. It’s one of the best ways to get the JS-of-tomorrow today.

Flow

Homepage

A static type checker for JavaScript

Flow is practically brand new, just open sourced at the end of 2014, though Facebook has been using it internally for a while. All it does is type checking (unlike TypeScript) using annotations similar to TypeScript. From what I’ve seen Flow’s type system is a bit better and more expressive, though TypeScript is catching up (e.g. TypeScript 1.4 will include Union Types and Type Aliases, which Flow already supports).

The downside is that the code Flow type-checks has to be pretty much plain JS (with type annotations) so you can’t really combine it with other tools (like Sweet.js) unless those other tools will preserve the annotations when transforming the code. One could potentially work around this with something like flotate (some of its rationale discussed here) as long as the other tools preserve comments. Some tools (like Babel) already support Flow and I imagine that broader community support for type annotations will grow quickly.

Elm

Homepage

Note: When referring to Elm, generally I mean Elm + the elm-html package (which is a virtual-dom library similar to React).

Elm’s a modern language designed to make writing code for interactive applications short, fast, and maintainable. It’s well thought-out, pragmatic, and tries to structure its design so it’s easy to pickup/understand.

Pros

• “Better” language than JS to work with
• Many of the same benefits of Flux + React or Mercury
• Interoperable with plain JS (via a language feature called ports)
• Can have even better performance than React because of language features (immutable data structures, optional lazy evaluations, etc.)
• Don’t have to deal with npm and bower while also getting some really nice tooling
• A natural way to structure applications

Cons

• Like React, more immature ecosystem
• Needs a runtime in the browser

PureScript

Homepage

PureScript is a small strongly typed programming language that compiles to JavaScript.

Taking many cues from Haskell, but making them fit in the JS world (like strict-by-default evaluation), PureScript compiles down to a plain JS with no runtime (unlike Elm). It has existing bindings to things like React and Angular, though they are both in the alpha/experimental stages. They also push against some of the functional features of PureScript so using something lower-level and more friendly to immutable data like virtual-dom is probably a better approach.

Paired with something like the signal library and you have a virtual-dom FRP implementation to play with.

Pros

• “Better” language than JS to work with
• Flexibility in implementation, it’s a full-blown general purpose language so less choices are made for you, unlike something more focused like Elm
• No (extra) runtime

Cons

• Still somewhat experimental

Other Haskell Stuff

There’s a lot of awesome stuff going in the Haskell world to improve JS development, mostly in the form of Haskell-to-JavaScript compilers. As of today, most of it probably isn’t ready for prime time/big stuff, but is improving and I expect things to get a lot better here quickly.

See this post for a further discussion of some of these options and the reddit post has some good discussion.

Full Haskell

• GHCJS - Version of GHC that emits JS instead of native code, compile any Haskell to JS
• Haste - Compile almost any Haskell to JS

Haskell interop

• Fay

Other

• Inferno (formerly SJS) - type inference for JS
  • Introduction post
  • Reddit discussion

Other JS Language Tools

Sweet.js

Homepage

This is a macro system for JS. It would probably be extremely useful in cutting down boilerplate if one chooses to use plain JS-of-today for a project. The biggest downside with Sweet.js is that it doesn’t (yet) support type annotations so doesn’t integrate super nicely with the other static-analysis/type-checking tools.

Bacon.js and RxJS

Bacon.js Homepage • RxJS Homepage

Both Bacon.js and RxJS provide “reactive” components, which, roughly speaking, are things that have some notion of time built into them. They use different terms, but I believe the libraries are fairly equivalent in functionally. The Bacon.js docs have a bit on their compatibility.

Immutable and Mori

Immutable.js Homepage • Mori Homepage

Provides immutable data structures. Immutable data is great for a variety of things and in particular in can be a big boon when working with rendering stuff since determining if anything has changed is a cheap check.

Babel

Homepage

Babel makes next generation JS features available now by compiling them down into today’s JS. The currently broadly supported JS lacks a lot of modern features, this can be a great way to utilize some of them (like modules) until the browsers/your user base catch up. Used to be called 6to5.

CoffeeScript

Homepage

I mention this merely because it is somewhat popular, but I don’t really consider it an option as it doesn’t bring a whole lot of important things (namely static type checking) that the other solutions provide, it is largely just syntax sugar.

Everything Else

There are a bazillion other JS frameworks/tools out there that I didn’t look at closely, some linked below:

Frameworks

• Cycle.js
• Aurelia, Blog Announcement, HN announcement
• Durandal
• Vue.js
• Enyo
• Ember.js
• RightJS

Pieces

• Amperstand.js
• Backbone.js
• Knockout
• Ractive.js

Languages

• Roy
• LiveScript

Other

• Comparison of a bunch of frameworks (hint: tabs at the bottom)

TodoMVC Benchmark

There’s a TodoMVC benchmark with a variety of popular (and obscure) frameworks here:

http://matt-esch.github.io/mercury-perf

The benchmark consists of creating 100 todos, marking them as completed, and then deleting them. It aims to give an idea of how frameworks perform under real-world-ish conditions running idiomatic code (as opposed to micro-benchmarks, which tend to take advantage of obscure tricks and aggressive optimizations that sacrifice maintainability for extra speed). A more limited benchmark, but with a prettier graph at the end is also available; it was put together for the blog post announcing elm-html.

Best Bets Today

Flux + React + Immutable + Flow

Integrating a static analysis tool like Flow would be a requirement for me, but that could be dropped if your requirements are pure JS with no build step (and/or you subscribe to the pulling your hair out and yelling at the computer style of debugging/refactoring; static analysis really helps people). I’d probably also want to use some JS-of-tomorrow layer as well, either using Babel or swapping out Flow for Typescript.

To see how some of this stuff plays together, see this blog post which suggests making heavy use of the PureRenderMixin (among other things) and generally having pure functions and immutable data. With the introduction of React Native, this setup has a lot of potential to reuse all the application-y bits (Flux + Immutable + Flow + (Babel)) and swap out the React bit for a native view layer.

Elm

A clean, modern language designed from the ground up to make web development better. Immature in some ways and may not be “production ready” enough for all cases, but well worth a look and experiment. The language is evolving fairly quickly, so you’ll need to keep up, but that’s no different that most other things in the JS world right now (^_^).

TL;DR

If you choose to stick with plain JS, look at Flux (Marty.js being my suggested implementation) + React and try to integrate some sort of static analysis into your workflow; I’d suggest Flow. If you have/are given more freedom, explore Elm. If you want something small and simple, look at Mithril.js.

The main take away is: buy what the functional programming folks have been selling for decades, pure functions and immutable data. You have to go out of your way to do this in plain JS; using a language that has this style of programming built in will cut down the boilerplate and open up nicer abstractions.

TL;DR the TL;DR

Learn Haskell.

I hope this was helpful in some way. Thanks for reading/skimming/skipping to the end.

At some point in 2013, I stumbled upon an old post by Penny Leach¹. Like her, I’ve found I tend to not commit changes to this stuff atomically. I change something, intending to commit it once I test it out to make sure it’s working, only to get on with working and forget about the committing part. Then I do the same thing with some other tool’s configuration and eventually end up with weeks of changes in a bunch of files all in a single repository.

Since Penny’s post (over five years ago at time of writing), vcsh has evolved (the vcsh script she mentions was deprecated) and is a bit nicer to work with. I just followed the vcsh docs and used the ‘Steal-from-Template’ way to get setup.

Pretty straightforward after that. My dotfiles now live in separate repositories for each program/tool and are the repositories starting with ‘dots-’ on gitlab. One very nice thing about this setup is now each tool’s configuration can be on separate branches at the same time. For instance, I’ve been reorganizing my vim dotfiles lately and for systems where I want to test the new configuration I just git checkout wip/reorg in the vim repo; no other tools configuration is touched and if something is broken, I can just git checkout master and old vim config is restored. Similar story if I ever need different configurations for different versions of a tool, or on different machines, just checkout the needed branch for the respective tool. This is a capability that is lacking in the single dotfiles repo scheme and it’s really useful to have.

With myrepos, managing a bunch of separate repositories is easy enough and to configure a new system it’s just:

cd
vcsh clone https://gitlab.com/doshitan/dots-mr.git mr
<tweak symlinks in ~/.config/mr/{available,config}.d/ if needed>
mr up

Basically, get your myrepos config setup, then have it set everything else up. If you’re still doing a monolithic dotfiles repository and/or symlinking stuff around, I strongly recommend giving the vcsh/myrepos combination a shot.

As previously mentioned if you follow the blog, I have left WordPress for Hakyll. This is the technical followup going through how I went about setting the site up.

Hakyll Setup

So first things first, install Hakyll. Previously I used cabal-dev, but Cabal-Install 1.18 includes the Sandboxed Builds and Isolated Environments work so that is my sandboxing environment now. This post is helpful for folks switching over.

I thought it might be an experience to write everything for the site in Haskell, using Clay as the CSS preprocessor and Fay for JavaScript. I tried Clay, but it’s math operations on values just don’t work and that’s a deal breaker. I was going to try implementing the operators, but decided to just use LESS so I could move forward. I may visit Clay again soon.

I debated what framework to build the site on, if I used one at all. I’ve used Bootstrap before and like it well enough. I’ve heard great things about Foundation as well, but I’m not as familiar with the Ruby environment, so out of the two, I’m biased towards Bootstrap. For the most part they are pretty much the same, especially with the release of Bootstrap 3 whose revamped grid system borrows a lot from Foundations – and is better for it. But besides the big two, I thought about going leaner and different with things like Responsable, Framless, Golden Grid System, or Open DAWS. I mean, my site isn’t going to be a big thing with complicated interactions, so I could do most stuff by hand. Ultimately though, I think what I would build would come out pretty similar to Bootstrap anyway, so coupled with my familiarity with it and the better grid system in version three, I went with Bootstrap.

Because I’m using Bootstrap, I haven’t needed any custom JavaScript yet, but when I do, I think I’ll try Fay.

Write the code

Hakyll is less a static site generator and more a library to write a static site generator, as is usual in Haskell land. Not too hard to cobble something together from other folks’ example code though. Basic file layout for the project is:

.
├── assets
│   ├── js
│   └── less
├── drafts
├── libs
│   ├── bootstrap
│   └── font-awesome
├── pages
├── posts
├── src
└── templates

Most of it is fairly self-explanatory I think. pages/ holds content that doesn’t really have a “published” date or represent some snapshot in time and are generally kept current whereas posts/ holds the stuff that is time sensitive. drafts/ holds posts that I’m preparing to publish, there is a makefile target that links/unlinks them into the posts/ so I can enable/disable them easily. libs/ holds the git submodules of my front-end third-party dependencies. src/ holds the source code for the site generator. If you want to have a deeper look, the source code is on GitLab.

Moving the content

I used Markdown on Save with WordPress and I only had a few posts, so I just manually copied the markup over. Nothing fancy here.

Static Site Tradeoffs

Because I’m using a static site, there are a few things you don’t just get, such as search and comments.

Search can be done client side (lunr.js¹), through a third-party (Swiftype²), or through a server side program. Some more discussion on Stack Overflow.

A potential downside to the lunr.js approach is that the entire site index must be loaded into the clients memory. In principle this seems wasteful for every client to expend the energy to construct the entire search index compared to every client sharing one on the server. In practice, probably not a big deal for the scale of this site (extremely small).

I haven’t decided what I want to do about search yet. For now, people can just use DuckDuckGo/Google to search the site, but I’d like to have a built-in option as well. I’ll probably do a lunr.js thing or make a simple server side search daemon.

Comments can also be handled on the client side code and some third-parties provide easy solutions; some standouts being Disqus and IntenseDebate. Honestly though, this site is extremely low traffic and for the one person every five years who would want to comment, I’d prefer they just email me. Less for me to maintain.

Server Setup

I was hosting the old site on DreamHost and I know others have had mixed experiences with them, but their hosting was fine for me, never had problems. While the hosting was fine, it was shared hosting, which is pretty restrictive. I wanted more flexibility for my site, with root access to install whatever I need and dedicated resources so I can plan what I can/can’t run. So I wanted a VPS. I decided to go with Digital Ocean mostly because of what I think is an incredible value. I’m using the smallest plan currently (512MB memory, 20GB disk space, $5/month) as it suits my minimal needs perfectly and is cheap, cheaper than even the old DreamHost account.

Nothing else too interesting on the server setup, just the basics. Debian, lock down ssh (disable root, disable password login, etc.), enable dotdeb repo (for newer nginx), install nginx, configure nginx (I made use of POSIX ACLs for access to /var/www/ and such), moved DNS to Digital Ocean. For some extra security, I setup and configured ConfigServer Security & Firewall (CSF), which was painless. CSF is nice for non-sysadmin folks (like me) who don’t want to spend a bunch of time learning/setting up/maintaining the important, but often impregnible, software for firewalls, intrusion detection, and/or failed login attempts, as it does it all (mostly) automatically.

Wrap up

All and all, pretty painless migration. Playing with Hakyll is a fun way to learn some Haskell.

WordPress As A Barrier

WordPress is great for what it is and nowadays you can build just about anything with it, but for how I think, its work flow presented a problem. It seems silly, but having to sign in, type inside a rather crummy editor (compared to what I’m used to), and press buttons was a big hurdle to jump for me. It always put me off from writing, just that little bit of unpleasantness that would stop me from starting. As I said, kind of silly, but thems the rubs.

I’m at a terminal or inside Vim most of the time. If I could publish and control my website without leaving these environments I would be much more likely to keep up with blogging. So that’s what I wanted.

There are tools such as blogpost and WP-CLI, which would provide most of what I was looking for. But even if I could gloss over the interface with some command line tools, it would still come down to managing a collection of plugins and updates that you really ought to stay on top of for security reasons.

So it’s still a bit of a hassle to maintain a WordPress installation even if I could do most of it from the command line. I wanted something simpler, where I could just write, publish and get on with life; no managing a database or being bugged about updates.

Hakyll As An Enabler

Given I didn’t really want to deal with a dynamic blog engine, I was looking for a static site generator. These have been rising in popularity the past few years with things like Jekyll and hosting on Github pages. They are much more suited to my mindset.

There are a some good ones out there, but for a variety of reasons I chose Hakyll. This was mostly because I’m interested in Haskell (the language Hakyll is written in) and powering my website with it seemed like a good way to sharpen my skills. It also uses pandoc which is great and provides a lot of flexibility for input formats.

So now when I get the inspiration to write a post I can just pop open a new buffer, write some stuff, preview the output locally and push it to the server whenever I feel it’s ready. My content, all of it, is with me on my computer at all times. I can write and tweak things without an internet connection and just push everything out when I do. The same could be said with an local development environment once it is set up, but that’s the point, there is little to set up. Once Hakyll is installed you can just work. No databases, or server to have running (well Hakyll uses the Snap server for previewing, but it’s not always running, it starts when you need it and shuts down when you don’t).

It is also very nice to not have to worry about constant security updates. The attack surface for a static site is, in practical terms, nonexistent. Of course, you do give up a few things, like built-in search and comments, but client-side/third-party solutions can provide reasonable replacements.

I’m not going to get into that now as this post is just a bit about why I moved away from WordPress, but I’ve been happy with the switch so far. I intend to do a more technical post exploring the how of switching over and the way things are set up soon.

You can find pretty much all the work I’ve done spread across a few bugs:

• Bug 640966 - Favorite applications should have priority when searching
  • This is near-ready, just need a decision what exactly should get priority.
• Bug 681797 - Switch to a List for Search Results UI
  • Also near-ready, just a matter of how to properly handle keyboard navigation
• Bug 682050 - Hide overview elements while searching
  • This one’s a bit more tricky. The current patches on the bug work, but break the behavior of hiding most of the workspace thumbnails when they aren’t being utilized (when < 2 workspaces are in use). The best fix is to refactor the overview into a St.BoxLayout or something similar that is a lot more automatic/flexible in the layout. I’ve started experimenting with this and have a basic working version that should be ready for 3.8.

So a number of things are, for practical purposes, done, but not widely tested enough to go into a stable product, particularly at the last minute. I’m comfortable with this, I would rather 3.8 have a solid, cohesive experience than a 3.6 with some piecemeal patches.

Other work I hope to do in the coming months includes sorting remote providers in the right order, providing the user a way to pick which search providers they want enabled, and indicate why a result was returned (highlight terms). With my fall semester also starting tomorrow though, my time for hacking on the shell will be greatly reduced. Even so, I think I can still get some things done and at the very least make sure all the work I did this summer is in proper shape.

Thanks to all the people who helped me get, and get through, GSoC; in particular Marina, Rui, and Jasper.

Week 11

Started implementing a new layout for the search results. A simpler list layout that is less image based than the current grid. See the design page.

Week 12

Finished the new list view and general cleanup before review.

Looking Forward

The goal is to get stuff reviewed and into master.

(Short post this time, gotta get back to work!)

Week 9

Implemented show/hide methods for the Dash and made use of them while searching. Started work on treating the workspace thumbnails more like the Dash, i.e., an element added to the overview on its own, not tied to something else.

Week 10

Finished splitting the workspace thumbnails from the window previews. Added hide/show methods to ThumbnailsBox that slide the thumbnails out of/in to view and made use of them while searching.

Looking Forward

I need to package up all the separate work I have been doing and get it attached to some bugs for proper review.