Wordspeak

Exploring Australian indigenous bible translation

Edwin Steele — Mon, 05 Aug 2019 06:53:00 GMT

Everyone deserves to have access to the bible in their native language.

Given the large number of Australian indigenous languages, I wanted to see how far data could take me towards answering the question Which Australian indigenous language group is most in need of bible translation?

It turns out that data can go a long way. As to the question itself:

Spoiler: It's the Kukatja people in Western Australia.

I answered this question by writing some software to aggregate data on bible translation efforts, language groups, language relationships, speaker count and locations and English language competency. I used data from the Joshua Project, the World Atlas of Language Structure (WALS), the 2011 Australian Census, SIL's ISO language code mappings, Find A Bible, AIATSIS AUSTLANG and Tindale's Catalogue of Australian Aboriginal Tribes and after a bit of recent inspiration and rework I've published the output from the tool. Keep in mind that this was written as a proof-of-concept - the user interface has plenty of rough edges, and the data has only had limited review.

You can see it at https://language-explorer.wordspeak.org.

Why the Kukatja?

Most importantly, they don't have any of the bible in their native language. Given the first book of the bible to be translated is usually one of the gospels, a language group with even a single book has access to the key parts of the message of Jesus. Given everyone should have some chance to read about Jesus, it's important to prioritise a giving that first opportunity. Secondly, while the Kukatja aren't the largest language group without bible access (that would be the Jaru) their low (self-assessed) ability in Australia's national language, English, means that they can't use an English-language bible either, so they're quite stuck.

Here's output from the software showing only language groups without a bible (filter by No Scripture) and sorted by count of non English language readers.

The software also has a map of language groups, with size corresponding to speaker count and colour showing scripture access (red circles are groups without any scripture, yellow circles are groups with only a single book of scripture)

The data shown in the software is a few years old, so it's quite possible that the Kukatja and the Jaru people have a bible translation underway, but I think there's promise in combining a data-driven approach with God's leading as we work towards giving everyone access to the bible in their native language. If you're interested in current Australian indigenous bible translation efforts, take a look at Aboriginal Bibles.

The source code for the software is available on Github

Accessibility - the old gray mare, she ain't what she used to be

Edwin Steele — Sat, 06 Oct 2018 07:34:51 GMT

My eyes aren't getting any better as the years pass, and so I'm more aware of things that improve website usability for those who don't have awesome vision or who read the web through a screen-reader.

I've recently improved the accessibility of this site by increasing the contrast of page elements and providing better information for a screen reader (removed redundant information and added it where it was missing).

If you're wanting to improve the accessibility of your site (and it'd be great if you did), you'll get some practical, easy-to-implement guidance from the Accessibility section of Chrome's Lighthouse audit tool and from WebAIM's WAVE tool. Even if you pick a few higher priority suggestions and come back to the others, it'll be an improvement for someone with reduced vision who reads your site. Future you says thanks.

Making the Touch Bar useful

Edwin Steele — Fri, 11 May 2018 20:58:50 GMT

I've had a Touch Bar equipped Macbook Pro for about 6 months. Until recently, I only used the Touch ID sensor with any regularity; I couldn't see a use for the other buttons and the switch from buttons to sliders was a usability regression. I recently read Making the Touch Bar finally useful and discovered how BetterTouchTool can be used to customise the touch bar. Wow.

Here's my new touch bar:.

From right-to-left:

The escape key. As a user of an Apple iPad keyboard that lacks an escape key, I've mostly retrained muscle memory to use ctrl+[ for escape. I still use the key occasionally (and I'd still rather have a physical key).
Battery info. I had battery info in the menu bar but the Touch Bar is more noticeable, and the extra space allows for more info. This helps me be more aware of battery-hungry apps, and now my battery lasts longer.
ADSL connection info. My ADSL model exposes this info by SNMP. It was previously in the menu bar as a custom plugin for BitBar and, like the battery info, it's more visible and useful in the Touch Bar.
www.wordspeak.org ping time. This is an indication of my upstream connection quality, not as monitoring. When touched, it opens an Alacritty terminal to my hosting machine, with the appropriate colour scheme.
Gateway ping time. Like the other ping time widget, this gives me an indication of link congestion, which often happens when devices are doing cloud backups or uploads. When touched, it also opens a custom Alacritty terminal.
Coffee time! Puts the laptop to sleep.
Volume and brightness. Buttons, not sliders. The Touch Bar is a small target, and I found it hard to set the brightness or volume correctly with the sliders that Apple uses in the default configuration.
Weather. Live, local weather from the Bureau of Meteorology implemented in shell
Clock. It's in the menu bar too, but I notice it more in this location.

The Touch Bar is a well implemented piece of technology, with a poor default configuration. Paying a few dollars for a BTT licence to make it useful is a good move.

Smaller images with single-step compression

Edwin Steele — Fri, 13 Apr 2018 20:38:45 GMT

I'm reviewing image sizes to improve download times in my photo galleries and I've obtained the smallest file sizes by performing a single compression step rather than allowing each tool to perform compression during my image pipeline.

Each tool in my workflow has defaults that work well if there is no subsequent or proceeding compression, but produce sub-optimal results when used in an image pipeline where each tool performs compression. My workflow is:

Load and edit photos in Apple's Photos.app.
Export from Photos.app. I choose a "JPEG Quality" level at export time.
Stamp copyright and licencing info using ExifTool
Resize images as a part of image gallery creation using Pillow, which generally involves a compression step
Perform final optimisation using imageOptim. I've used this tool in the past to reduce JPEG sizes with great success and it's consistently given me the best image compression. Adding this step was the trigger point for this investigation.

I did experiments on my Arches gallery whose photos have a total uncompressed size of 103.6MB. I tried four permutations of compression with the results below, noting that the Pillow step also includes resizing:

Compression by Photos.app, Pillow and imageOptim: Final size 4.6MB
- Photos.app (medium quality) 103.6MB → 19.1MB
- Pillow (75% quality) 19.1MB → 5.2MB
- imageOptim (74% quality) 5.2MB → 4.6MB
Compression by Pillow and imageOptim: Final size 4.6MB
- Photos.app (maximum quality) 103.6MB → 103.6MB
- Pillow (75% quality) 103.6MB → 5.2MB
- imageOptim (74% quality) 5.2MB → 4.6MB
Compression by JPEGmini and imageOptim: Final size 4.3MB
- Photos.app (maximum quality) 103.6MB → 103.6MB
- Pillow (100% quality) 103.6MB → 27.8MB
- JPEGmini (no user-selectable settings) 27.8MB → 7.1MB
- imageOptim (74% quality) 7.1MB → 4.3MB
Compression by imageOptim only: Final size 4.1MB (best result)
- Photos.app (maximum quality) 103.6MB → 103.6MB
- Pillow (100% quality) 103.6MB → 27.8MB
- imageOptim (74% quality) 27.8MB → 4.1MB

The best result comes from performing a single compression step with the best compression tool.

I expected that the individual compression tools would have complementary compression schemes and chaining them together would give the best compression. I suspect now that imageOptim uses all the types of compression schemes available in the other tools and gets the best result because it can take an image with low entropy i.e. an uncompressed image and introduce all the entropy (compression) in a single step.

Disincentives and Photo Hosting

Edwin Steele — Fri, 16 Feb 2018 19:01:16 GMT

I've been searching for a simple, scriptable photo hosting tool so that I can move off Flickr, and I've found one. Photo hosting tools don't seem to have evolved much in recent times, which is unsurprising given social media does the job of photo sharing for most people. There are a few tools, for sure, but there's something fun about hosting my own photos so I can optimise, meddle and learn and still have the joy of sharing (and indeed sharing without ads).

So it's time for me to say goodbye to Flickr. It's been a good technical platform and seems to have a community in it, but I didn't like Flickr's injection of ads in my galleries and the platform does feel irreversibly stagnated - I was excited while when Yahoo gave it some focus a few years back, but there's nothing to indicate anyone's committed to developing it.

So I've moved my photos across to a self-hosted instance of Sigal. Performing the move was a delightful task as I relived some of the memories that I've previously published, and then discovered albums that I could have published, but never did. I realised my reservations about Flickr were a subconscious disincentive for publishing.

So, with this renewed vigour, I've published photos from my 2016 trip to Cyprus along with the rest of my photos. I hope you enjoy them.

Yak shaving with Vagrant, Travis-CI and AWS

Edwin Steele — Fri, 18 Nov 2016 06:01:00 GMT

Tldr; Don't use the vagrant package from your distribution if you intent to build plugins.

I've just finished setting up CI pipeline for a personal project. The project has an Ansible playbook that I want to exercise every time there's a commit or a PR. While completing the task I shaved a yak and narrowly avoided shaving a whole herd. I planned to use Vagrant in my Travis-CI pipeline to start an instance in AWS, run the playbook, look at the result and terminate the instance. Vagrant, Travis-CI and AWS are pretty common tools, so I was surprised at the wrangling involved before I ended up with a solution. I thought I'd document my findings to minimise the chance that others will have the same experience.

Finding #1: Travis' default build agent has an old Vagrant

The default build agent is based on Ubuntu 12.04 LTS Server which ships with Ansible 1.0 in its apt repo. The vagrant-aws plugin requires Vagrant 1.2. Fortunately they have a Ubuntu 14.04 LTS Server beta which has a newer Vagrant in the apt repo.

Finding #2: The AWS-Vagrant plugin won't build with a newer Vagrant because of missing libraries and tools

$ vagrant plugin install vagrant-aws
Installing the 'vagrant-aws' plugin. This can take a few minutes...
/usr/lib/ruby/1.9.1/rubygems/installer.rb:562:in `rescue in block in build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)

        /usr/bin/ruby1.9.1 extconf.rb
/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- mkmf (LoadError)
    from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from extconf.rb:4:in `<main>'

Searching for the mkmf LoadError in the context of Debian and Ubuntu gives recommendations to install a few devel packages including ruby-dev (some will say a specific version of ruby dev). It seems that AWS-Vagrant is a Ruby gem, and gems are built on-box (at least it seems so - I'm a Ruby rookie), so libraries and the ruby compiler are needed. These aren't installed on the build agent.

$ sudo apt-get install build-essential libxslt-dev libxml2-dev zlib1g-dev ruby-dev

Finding #3: The AWS-Vagrant plugin needs Ruby version 2.0+

$ vagrant plugin install vagrant-aws
Installing the 'vagrant-aws' plugin. This can take a few minutes...
/usr/lib/ruby/1.9.1/rubygems/installer.rb:388:in `ensure_required_ruby_version_met': json requires Ruby version ~> 2.0. (Gem::InstallError)
    from /usr/lib/ruby/1.9.1/rubygems/installer.rb:156:in `install'
    from /usr/lib/ruby/1.9.1/rubygems/dependency_installer.rb:297:in `block in install'
    from /usr/lib/ruby/1.9.1/rubygems/dependency_installer.rb:270:in `each'
    from /usr/lib/ruby/1.9.1/rubygems/dependency_installer.rb:270:in `each_with_index'

vagrant plugin install wants to use Ruby 1.9.1 to perform the gem build, but that version is too old. Fortunately the build agent has Ruby 2.3.

Finding #4: Ubuntu Vagrant can't load plugins built with Ruby 2.3

$ gem install --verbose vagrant-aws
...
Successfully installed vagrant-aws-0.7.2
41 gems installed
$ vagrant plugin install /home.travis/.rvm/gems/ruby-2.3.1/cache/vagrant-aws-0.7.2.gem
/usr/lib/ruby/1.9.1/rubygems/format.rb:32:in `from_file_by_path': Cannot load gem at [/home.travis/.rvm/gems/ruby-2.3.1/cache/vagrant-aws-0.7.2.gem] in /home/travis/build/edwinsteele/biblebox-pi (Gem::Exception)
    from /usr/share/vagrant/plugins/commands/plugin/action/install_gem.rb:36:in `call'
    from /usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/plugins/commands/plugin/action/bundler_check.rb:20:in `call'
    from /usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
    from /usr/lib/ruby/vendor_ruby/vagrant/action/builder.rb:116:in `call'
...

Now I'm running out of ideas and I'm considering less conventional means like building on the Travis MacOS environment where I can use Homebrew or moving to another CI hosting provider entirely. Fortunately I stumbled on the answer...

The Answer

From the Vagrant docs

Beware of system package managers! Some operating system distributions include a vagrant package in their upstream package repos. Please do not install Vagrant in this manner. Typically these packages are missing dependencies or include very outdated versions of Vagrant. If you install via your system's package manager, it is very likely that you will experience issues. Please use the official installers on the downloads page.

Yeah, I experienced issues. Once I followed the advice it was smooth. Perhaps I should have looked at the official docs sooner!

$ wget -O /tmp/vagrant.deb https://releases.hashicorp.com/vagrant/1.8.7/vagrant_1.8.7_x86_64.deb
$ sudo dpkg -i /tmp/vagrant.deb
$ vagrant plugin install vagrant-aws
$

And now I have a CI pipeline running on AWS after each commit. Nice.

Site Security Improvements

Edwin Steele — Thu, 06 Oct 2016 20:01:00 GMT

I did some security work on this site recently. I was able to get some nice wins without a great investment of time, in part due to the great resources that are available. Here are the areas of work, the resources that I used, and the outcomes:

Content Security Policy (CSP)

A CSP constrains the actions that a web page can take or the actions that can be performed upon it. It allows one to apply the principle of least privilege to a page and site. A CSP allows one to specify constraints like "Only Load CSS from these sources", "Don't allow this site to be embedded in frames" and "Don't allow inline JavaScript". I developed a CSP after reading a the HTML5rocks CSP tutorial and Scott Helme's CSP intro. I validated my policy using Google's CSP evaluator and Mozilla's Observatory tool. In order to apply best-practices, which include disabling inline JavaScript and CSS, I needed to make a simple changes to the site. I've been conscious to minimise JavaScript and CSS as I've developed this site, and it was great to see how that choice made the application of best-practices a simple task.

Miscellaneous security headers

I implemented X-XSS-Protection, X-Content-Type-Options and X-Frame-Options and while the effect of these headers overlaps a little with CSP, providing them is still a good idea because of inconsistent CSP implementations and benefits unrelated to CSP. I learnt about them from Scott Helme's Response Headers page and Mozilla's web security guidelines. I validated my setup with the SecurityHeaders validation tool and Mozilla's Observatory tool.

SSL

I already had a reasonable SSL setup but while looking at the Mozilla web security guidelines, I didn't consider how my list of cipher choices would need regular updating (I'd last reviewed them 2 years ago!). Mozilla are good enough to provide nginx config snippets for to help with good cipher selection, and their config snippet included an HTTP Strict Transport Security (HSTS) directive. I'd considered HSTS before, but found the SSL certificate renewal process to be complex enough that I was unsure I wouldn't accidentally take my site offline around renewal time. Having recently switched my site certificates over to the (awesome) Let's Encrypt renewal process, I felt comfortable activating HSTS at the same time. I validated my setup with the Qualys SSL Report

Outcome

It took about 4 hours to make the changes, and after the changes were applied, this site ¹ moved from an A to an A+ on the Qualys SSL Report. The Mozilla Observatory tool gives the site an A+ and the SecurityHeaders.io validator gives it an A. My nginx config is available on GitHub.

Actually, I use Cloudflare as a CDN, so I ran the tests against (my origin server). ↩

Travelling through LAX just became a lot easier

Edwin Steele — Thu, 21 Jul 2016 06:39:00 GMT

I've transited Los Angeles Airport for many years on my way to conferences and family visits and on a recent trip I was thrilled to find that its no longer necessary to go through airport security when moving between international to domestic flights. While the signposting is pretty bad, one can now move between Tom Bradley International and Terminal 4 (and thus the rest of the domestic terminals) without exiting the terminal and subjecting oneself, and ones family, to security screening and potential delays.

Hooray.

How I use the Apple Watch

Edwin Steele — Wed, 29 Jun 2016 20:09:59 GMT

I bought an Apple Watch a little while ago. I really like it and I find it useful, but I use it differently to most people and I almost didn't buy one because I wasn't sure that it could fit with my need. Here are a few of my "workarounds" and a few findings that someone might find useful.

A non-standard user?

I don't have an iPhone.
I don't want to charge the Apple Watch at night; I want to wear it to bed.

I'd love to report that the Apple Watch can connect to an iPad (I only have an iPad), but it doesn't. You still need an iPhone to configure it, and to serve as a conduit for data and App installs, but you don't need it to be with you all the time for the watch to be useful. In my case it doesn't even matter that the watch is hooked into my wife's Apple ID (she has the iPhone) because all the important data is still accessible¹.

What I planned to use it for

silent alarms during the day, and to wake me up in the morning
the time
fitness tracker (mainly step count)

If you're thinking that I could have used a Fitbit, you'd be right. I've had a few of them and the build quality on my bands was poor and the software gets in my way ², so I wanted something else.

What I also use it for

checking the weather when dressing the kids for bed and when getting clothes out for the next day
countdown timers
driving directions when I'm out with my family

I don't use it for anything else.

Battery Life

The big surprise is that battery life isn't a problem, even though the standard usage pattern involves charging at night and I have it on my wrist at night.

My observations are that recording activity drains the battery a little, the attempts by the phone to connect to the iPhone drain the battery, and being in contact with the iPhone drains it quite a bit more.

Battery drain rates: * being a couch potato in aeroplane mode - 2% per hour * normal activity levels in aeroplane mode - 2.5% per hour * normal activity levels (not in aeroplane mode) when the iPhone is not around - 3.5% per hour * normal activity levels (not in aeroplane mode) when the iPhone is around - 5% per hour

I have a 42mm watch. Bluetooth is off, brightness is at the lowest setting and haptics are at the strongest setting). I have one third party complication Pocket Weather AU.

Charging

I've observed the watch charging at about 1% per minute, just like David Smith, whose great idea allowed me to even consider an Apple Watch.

I don't really need the watch while I'm preparing for my day, so I put it on charge within a few minutes of getting up so it's charging while I do my morning routine (shower, coffee, eat, read) so it has about 75 minutes to charge.

I have the watch out of aeroplane mode in the evening (~4 hours) i.e. 4 x 3% = 12% and the rest of the non-charge time in aeroplane mode (~19 hours) i.e. 19 x 2.5% = 47.5%. This means I need about an hour of charge time to be constantly topped off. That has always been enough and consequently I'm not worried about battery life.

So, it works for me

Initially, I thought the watch would be of limited use without connectivity but it's not been a problem. Most of the features that require connectivity are for notifications and I'm happy to forgo them - I like not being interrupted, to be honest, and where connectivity is necessary for data updates, I've found a once-a-day refresh to be sufficient. So, I'm really happy with the watch. It's been a step-up from the Fitbit in terms of functionality and quality so it's been a good purchase and I'm looking forward to what's coming in watchOS 3.

iCloud shared calendars for the win ↩
Why can't I set an alarm when I'm not connected to the internet? Why can't graphs render without an Internet connection? ↩

Making Ansible, Doas and OpenBSD play nicely

Edwin Steele — Wed, 29 Jun 2016 08:50:00 GMT

Update 11 Jan 2020: The behaviour of doas changed in OpenBSD 6.6 so the environment specification described below is unnecessary. As of 6.6, doas sets HOME and USER to reflect the target user, like sudo. For details see openbsd-tech

A little while ago, OpenBSD replaced sudo in the base system with doas. This transition follows a typical path in the OpenBSD community where a large, difficult to audit daemon or key application is replaced by an auditable and securable alternate implementation. At the same time, features are pruned if they add significant complexity. I don't need all the features of sudo and I like the simplicity and security of doas so I wanted to switch. I use Ansible to provision and manage my small collection of servers and when version 2.0 added support for privilege escalation via doas (via the become keyword) I started to make my switch. The main use-case in my Ansible playbooks is where I run as root and become my non-admin user so that permissions are set correctly by default, and in this case doas behaves differently...

# /usr/bin/env | grep -E '^(HOME|USER)='
HOME=/root
USER=root
# sudo -u esteele /usr/bin/env | grep -E '^(HOME|USER)='
USER=esteele
HOME=/home/esteele
# doas -u esteele /usr/bin/env | grep -E '^(HOME|USER)=' 
HOME=/root
USER=root

The key difference is which environment variables are propagated from the original session. I find doas to be a little unintuitive in this respect, particularly the inability in the new session to write to the directory specified by $HOME and this behaviour causes problems with the Ansible git and the pip tasks. The git task expects $USER to correspond to the effective user id and the pip task expects $HOME to be the home directory of the effective user id. The unexpected doas environment meant the tasks were trying to write files, as my non-admin user, under /root which was unsurprisingly unsuccessful. Fortunately Ansible has a way to specify environment variables for a task and even though it'd be preferable for the become mechanism to take care of this, it's an effective workaround (albeit a slightly dirty one).

The example below, from my web host playbook, ties the environment specification and the doas become in a block

# doas preserves USER and HOME, which is different to sudo, however by                                                                                                    
#  specifying them as environment variables we can proceed.                                                                                                               
# USER is required for git, and HOME is required for pip                                                                                                                  
- block:                                                                                                                                                                  
  - name: Checkout wordspeak repo                                                                                                                                         
    git: repo=ssh://git@github.com/edwinsteele/wordspeak.org.git                                                                                                          
         dest=/home/esteele/Code/wordspeak.org                                                                                                                            

  - name: Setup nikola virtualenv                                                                                                                                         
    pip: requirements=/home/esteele/Code/wordspeak.org/requirements.txt                                                                                                   
         virtualenv_command=/usr/local/bin/virtualenv                                                                                                                     
         virtualenv=/home/esteele/.virtualenvs/wordspeak_n7                                                                                                               

  become: True                                                                                                                                                            
  become_method: doas                                                                                                                                                     
  become_user: esteele                                                                                                                                                    
  environment:                                                                                                                                                            
    USER: esteele                                                                                                                                                         
    HOME: /home/esteele

And having tied this together, I can remove the sudo package!

    - name: Remove sudo
      openbsd_pkg: name=sudo-- state=absent