ln -sf /proc/self/ns/net /var/run/netns/vpn

Months later, this resulted in a face-meet-palm moment as I realized that since /proc/self changes for every process, this link was meaningless. It didn’t prevent the rest of the setup from working fine, of course, as none of that uses the vpn name to refer to the netns. However, if I wanted to run a command inside, then I had a problem. I embarked upon yet another journey to see how I could name if /proc/self wasn’t option. A couple of ways came to mind:

/bin/sh -c 'ln -sf /proc/$$/ns/net /var/run/netns/vpn'
/bin/sh -c 'ip netns attach vpn $$'

The first option didn’t work. Since the process in question died immediately, the link would become invalid. The second option, in which ip netns attach does mount shenanigans, seemed like it should work. Apparently, it remounts /var/run/netns as a bind-mount to itself, making it shared in the process, so that mounts in it are propagated to child namespaces. Then it mounts the netns in a subdirectory there, so it can be accessed independently of any process. However, once systemd starts our services in private namespaces, it is too late - even using the + prefix to elevate our commands beyond these namespaces doesn’t seem to work, and the mounts aren’t propagated correctly.

So this should be something that’s done before our services start. I first tried using a separate netns-default service just for naming the original netns. Once the initial setup was done, one would think that ip netns attach should then start working even in restricted services if run with elevated privileges. Quelle surprise, ip tries to do the mount shenanigans all over again and fails:

ip[471]: mount --make-shared /var/run/netns failed: Operation not permitted

Then I fell back to using this in the VPN service override:

ExecStartPost=/usr/bin/ln -sf /proc/${MAINPID}/ns/net /var/run/netns/vpn

This way, the netns will be accessible at least as long as the VPN process stays alive. After thinking a bit more, I decided to go for a third service:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# /etc/systemd/system/netns-vpn-post.service
[Unit]
Description=VPN network namespace (post)
ConditionPathExists=!/var/run/netns/vpn
After=<vpn>.service

[Install]
WantedBy=<vpn>.service

[Service]
Type=oneshot
RemainAfterExit=yes
# Hat-tip to A.B. here: https://serverfault.com/a/1097323/229499
ExecStartPre=:/bin/bash -c 'declare $(systemctl show --property MainPID <vpn>.service); ip netns attach vpn $MainPID'

This service doesn’t have to deal with private namespaces, and just sets up the name using ip netns attach. Note the : at the start of the command so that systemd leaves $ alone. Now everything looks nice:

% ip netns list
default
vpn (id: 0)

In the end, though, I went with just disabling PrivateMounts for the netns setup service, for which it doesn’t really matter, and running the ip netns attach commands there. Having PrivateMounts for the services run in that service might be fine, but for this one, it really was more trouble than it was worth.

1. The /var/run/netns/vpn had to exist (the old unit ran ip netns add vpn to create yet another namespace).
2. Mounts are problematic with PrivateMounts=yes.

The latter only became a problem with systemd’s v254 release:

* PrivateNetwork=yes and NetworkNamespacePath= now imply
  PrivateMounts=yes unless PrivateMounts=no is explicitly specified.

Since I run Arch Linux, I get the latest versions of systemd, and one day this setup just started failing, because:

File system namespaces are set up individually for each process forked off by the service manager. Mounts established in the namespace of the process created by ExecStartPre= will hence be cleaned up automatically as soon as that process exits and will not be available to subsequent processes forked off for ExecStart= (and similar applies to the various other commands configured for units).

— man 5 systemd.exec, PrivateMounts=

This meant that while I had a vpn netns created, the commands that I was running assuming that it was in the vpn netns were actually being run in the unit’s private namespace. So, when the VPN interface started up, there was nothing useful in the netns. No veth devices, no route to the internet.

I solved this problem by:

1. Using symbolic links to name the namespace, and
2. Using JoinsNamespaceOf instead of using name for the namespace in the other units.

This actually improved and simplified the setup, in addition to not having to create the vpn netns unnecessarily:

1. JoinsNamespaceOf directly expresses the relationship between the VPN service and the netns service.
2. Any other services that should be in the namespace of the VPN service can now use JoinsNamespaceOf=openvpn-client@<whatever>, instead of having to join some arbitrary namespace.

This creates a nice tree of namespace relationships. The vpn name for the netns now exists only for convenience, for when (if) I need to run ip netns exec to do something in it.

(Note that each unit that uses JoinsNamespaceOf must also have PrivateNetwork enabled.)

Some time ago, I bought a subscription for Tunnelbear, and they support Linux via OpenVPN. I started using it on my Pi, but to my dismay, I found that the VPN routing prevented incoming connections from the internet from being properly handled. This caused a couple of problems:

1. This broke SSH from outside.
2. I could no longer run an externally-accessible web server from the Pi. As a side-effect, Let’s Encrypt certificate renewal via certbot could no longer be automated, since it would only work when the VPN was off.

The first could have been something of a deal-breaker, but since I accessed the Pi via SSH from the internet only rarely, I was willing to let the VPN run as-is for the time being. I investigated online, and most solutions seemed to recommend setting up another routing table and using iptables to mark the traffic (via, e.g., matching the user) to use the new routing table. I didn’t find this particularly appealing - I didn’t want to go about running some process under a different group or user, and I didn’t want to mess with whatever routing OpenVPN deemed appropriate. Different strokes and all…

Enter network namespaces aka netns. The idea itself is old (see, for example, this 7-year-old post); I just happened to stumble upon it recently. In short, the key element driving the netns solution is the Virtual Ethernet Device aka veth. These are pairs of linked devices, where:

Packets transmitted on one device in the pair are immediately received on the other device.

By keeping each interface of a veth pair in different namespaces, we can have easy communication between the two. So, we’ll have the default namespace, which is where we normally operate, and we will have a new namespace where the VPN (and any applications that need the VPN) will operate.

The arcane incantations (to be invoked as root) involved are:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
NS_NAME=vpn
ip netns add "$NS_NAME"
ip netns exec "$SHELL"
ln -s /proc/1/ns/net /var/run/netns/default
ip link add dev veth1 mtu 1500 type veth peer name veth2 mtu 1500
ip link set dev veth2 netns default
ip addr add dev veth1 10.0.0.1/24
ip link set veth1 up
ip netns exec default ip link set veth2 up
ip route add 192.168.1.2/32 dev veth1
ip route add default via 192.168.1.2
ip netns exec default ip route add 10.0.0.0/24 dev veth2
ip netns exec default iptables -A FORWARD -i veth2 -o eth0 -j ACCEPT
ip netns exec default iptables -A FORWARD -o veth2 -i eth0 -j ACCEPT
ip netns exec default iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE
sysctl -w net.ipv4.ip_forward=1

What do these commands do? Let’s examine them block by block.

1. Set up the namespace and enter it:

    NS_NAME=vpn
    ip netns add "$NS_NAME"
    ip netns exec "$SHELL"
    ln -s /proc/1/ns/net /var/run/netns/default
   

   Here, we create a new network namespace named “vpn”, and start a shell in it. Since the default namespace probably doesn’t have a name, we set a name for it (oddly enough, default).

2. Create the veth pair and move one to the default namespace (we’re already in a shell in the new namespace, remember):

    ip link add dev veth1 mtu 1500 type veth peer name veth2 mtu 1500
    ip link set dev veth2 netns default
   

3. Bring up our veth interfaces and assign them IPs:

    ip addr add dev veth1 10.0.0.1/24
    ip link set veth1 up
    ip netns exec default ip link set veth2 up
   

4. Route to the external network via your proper network interface (here, I’m assuming its IP is 192.168.1.2) and route back:

    ip route add 192.168.1.2/32 dev veth1
    ip route add default via 192.168.1.2
    ip netns exec default ip route add 10.0.0.0/24 dev veth2
   

5. Set up packet forwarding using iptables:

    ip netns exec default iptables -A FORWARD -i veth2 -o eth0 -j ACCEPT
    ip netns exec default iptables -A FORWARD -o veth2 -i eth0 -j ACCEPT
    ip netns exec default iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE
   

6. Enable IPv4 forwarding using sysctl:

    sysctl -w net.ipv4.ip_forward=1
   

Then run OpenVPN in this network namespace (for example, by running the openvpn command itself here, or by using systemd to link it to this namespace).

I personally use systemd to set the whole thing up at boot. First, there’s the one-shot service to set up the namespace:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# /etc/systemd/system/netns-vpn.service
[Unit]
Description=VPN network namespace
ConditionPathExists=!/tmp/netns-vpn-concluded

[Service]
Type=oneshot
RemainAfterExit=yes

# Ask systemd to create a network namespace
PrivateNetwork=yes
# But not private mounts
PrivateMounts=no

ExecStartPre=/usr/sbin/ip netns attach default 1
ExecStartPre=:/bin/sh -c 'ip netns attach vpn $$'
ExecStartPre=/usr/sbin/ip link add dev veth1 mtu 1500 type veth peer name veth2 mtu 1500
ExecStartPre=/usr/sbin/ip link set dev veth2 netns default
ExecStartPre=/usr/sbin/ip addr add dev veth1 10.0.0.1/24
ExecStartPre=/usr/sbin/ip link set veth1 up
ExecStartPre=/usr/sbin/ip netns exec default /usr/sbin/ip link set veth2 up
ExecStartPre=/usr/sbin/ip route add 192.168.1.2/32 dev veth1
ExecStartPre=/usr/sbin/ip route add default via 192.168.1.2
ExecStartPre=/usr/sbin/ip netns exec default /usr/sbin/ip route add 10.0.0.0/24 dev veth2
ExecStartPre=/usr/sbin/ip netns exec default /usr/sbin/iptables -A FORWARD -i veth2 -o eth0 -j ACCEPT
ExecStartPre=/usr/sbin/ip netns exec default /usr/sbin/iptables -A FORWARD -o veth2 -i eth0 -j ACCEPT
ExecStartPre=/usr/sbin/ip netns exec default /usr/sbin/iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE

ExecStart=/usr/bin/touch /tmp/netns-vpn-concluded

It differs a bit from the code above, because this uses the network namespace that systemd can set up for a service. The sysctl knob can be set at boot using a file in /etc/sysctl.d.

Then use a drop-in to modify the OpenVPN service to use the same namespace:

1
2
3
4
5
6
7
8
==> /etc/systemd/system/openvpn-client@/override.conf <==
[Service]
PrivateNetwork=yes

[Unit]
JoinsNamespaceOf=netns-vpn.service
Requires=netns-vpn.service
After=netns-vpn.service

This runs all instances of the openvpn-client template service in the namespace created for the one-shot service. You can, of course, make a template of the network namespace setup service, and have one for each instance of the OpenVPN client.

A similar drop-in override can be used for other services (e.g., Kodi or a desktop session).

This doesn’t protect against the VPN dying and leaving your services directly connected via your network. I think that can be bluntly handled by having an ExecStopPost for the OpenVPN client which removes the default route altogether (and correspondingly have an ExecStartPre set up the default route instead of setting it in the one-shot service).

Updates

Since the time this post was originally published, I have updated the unit files above twice. Details on the updates can be found in these follow-up posts:

1. Network Namespaces and systemd Private Mounts
2. Naming Network Namespaces in systemd Private Mounts

It spent nearly a month collecting dust before I finally got around to booting it. And when I did upgrade from polygonal tuits, I was faced with a tough problem: which OS do I use?

1. Raspbian is pretty popular, and perhaps the first among equals. But: Stable releases of Debian-based distros like Raspbian tend to be heavily outdated. One of my goals with the Pi was to play with new software. I could hunt down repositories, or build stuff myself, but ¯\_(ツ)_/¯
2. Ubuntu’s main repositories don’t support ARM. That’s over on http://ports.ubuntu.com/. Even so, support in PPAs for ARM isn’t that widespread.
3. Arch officially only supports the amd64/x86-64 architecture (they dropped support for i686 some time ago). ARM, like i686, is supported by the community. Mirrors are a bit of a problem. There are none in Japan.

Instead of going for Raspbian or Ubuntu, I opted for installing Arch Linux. I ended up picking Arch because it is my daily driver on my PC, and I already have a box running Ubuntu serving as a … server. It was time to see how Arch would hold up.

Installation

It’s been over a year since I installed it, so I don’t remember specifics of any problems I had during the installation. I don’t think I had many — except maybe if when I goofed up with some commands. I did have some concern that the ARMv7 build that Arch Linux ARM recommended wouldn’t be good enough, but I think things have mostly turned out just fine.

Ah, yes! Like an idiot, I tried my default way of booting for system installations: using an USB drive. And was greeted with a blank screen. I think I fiddled with the KVM switch at first, then tried connecting everything to the Pi instead of the switch and I even tried flashing the disk a couple of times. Finally I Googled and realized that a Pi does not boot from an USB drive unless it has been configured to do so — by booting from a microSD card. That meant buying one, and waiting a couple of days for its arrival. Booting from the card went fine: I finally saw the rainbow splash. (All told, it was about 6 weeks since I bought the Pi that I first successfully booted it. Damn, I’m lazy.)

As I said before, Arch Linux itself worked pretty fine. I’d give it a Gold rating (not Platinum), because I have had the occasional kernel panics over the past year, about once every couple of months or so. For the moment, I have stuck the panic=10 bandaid on it.

The main issue — I can’t even really call it an issue — is that the community’s preferred kernel is a bit old. They track an LTS kernel, something to do with custom firmware needed for some components. Since I don’t use most of it — not the camera input, not the audio out, I don’t even run a GUI on it — I think I could get away with running the latest, mainline kernel. Something to test soon. Shortly after I got the Pi, I wanted to try iwd, the new wireless network management daemon. However, the kernel in use then (4.12-ish, I think) didn’t support it. The kernel is currently 4.19 (following Arch’s linux-lts), where my PC runs 5.1. I think it does support iwd now, but I haven’t gotten around to testing it yet.

Using a microSD card

Since I had to boot from a card anyway, I decided to go with it completely, instead of configuring USB boot. This proved to be a mistake. I don’t know if it was the card I’d bought (a Toshiba M203) or something with the Pi’s I/O, but damn, the system was slow. Tab completion, listing files, running commands — everything had a very noticeable, multi-second delay. I mean, I understand that the ARM CPU isn’t comparable to the i7 on my PC, but this was slower than anything I’d used in the past decade, except for SSH connections over mobile internet. I thought it might be the CPU, but later, I realized it was the I/O from the card, when I attached a USB hard disk to it. Listing files on the disk was noticeably faster than doing so on the card. Then I noticed that essentially anything which touched the card turned slow, like it was nearing the accretion disk of a black hole. I decided to stick with the card for the time being, but it did lead to a lot of frustration.

I finally moved to a USB flash drive a couple of weeks ago, and things have drastically improved. I couldn’t use my 8GB drive - it turned out to be just a bit smaller than the 8GB microSD card. I could resize the partitions and all that, but meh … I did have another 16GB one lying around. That one worked OK. I had moved some stuff from the card to the hard drive for performance, but now I have I moved them back to the flash drive, simplifying my setup.

Power

I have a power strip with a couple of USB ports for charging, and I used one of those for powering the Pi. And so I have had warnings about “under voltage” pretty much from day zero. Even if I did buy a dedicated power adaptor for the Pi, I didn’t have spare slots on the power strip. I tried changing cables, using shorter cables, etc. to no avail. I don’t know if the under-voltage situations had any really impact on anything. They always seemed to be resolved in seconds, and I didn’t notice any relation between their appearance and the aforementioned slow I/O. The one problem that might be linked to it was a hard disk acting up.

I had two USB hard disks connected to the Pi, and the older one of them, a Toshiba Canvio bought in May 2012, started having problems. It would randomly stop responding. It could be the age of the drive, it could be the power — I’m not sure. IIRC a consumer hard disk lasts around 5 years, so this one had already gone above and beyond. I managed to get all useful data off via my PC, though, which is why I doubt if it was really failing. I had another aged disk fail on an unrelated system around the same time, and that one just started spewing I/O errors all over the place (couldn’t even copy data reliably off it). Or maybe I just experienced two failure modes of aging disks.

I decided to bite the bullet and get a dedicated power supply, but I didn’t feel like getting just power supply for the Pi. If I was going to have to mess with my current power setup, doing it just for the Pi felt like a waste. Instead, I got a powered USB hub a few weeks ago, which also had an additional port for USB charging. I moved all the peripherals to it (except the flash drive used to boot it) and used it to power the Pi as well. I haven’t had an under-voltage situation since.

What next?

So, what did I do with this Pi? I’ll talk about the specifics in my next post. But the tl;dr is this: I configured it to run Nextcloud, and in general function as a file server with data on the hard disk.

It also serves as the gateway to my home. I have set up SSH (with only public-key authentication allowed) and port-forwarded it over my router, so I can access the Pi, and by extension (with some help from Wake-on-LAN) my PC as well. A non-default SSH port means I don’t see many any bots knocking on the SSH door.

Overall, I’m pretty happy with the Pi. I’d like to upgrade to the Pi 4, mainly for USB 3 and Gigabit networking, but I can live with the current performance. The main bottleneck is my Internet (upload bandwidth a measly 10 Mbps), and there’s nothing that upgrading will really improve. I’d even say the Pi is underutilized — I’m barely using 200MB of RAM most of time, and the CPU’s mostly idle too. I’d say if you don’t have 4K media, a Pi 3B is a perfectly fine media server.

Shutting down

The Alienware series apparently has a fairly common issue of panicking on shutdown (1, 2, 3, 4, 5). The apparent cause is something in the I2C Designware module. The solution used to be [disabling it][21], but things have changed since then, and now the Arch Linux kernel has it baked in instead of as a module that could be blacklisted. With some help from the Unix & Linux Stack Exchange, that problem has been overcome. The solution is to blacklist a kernel function, as Stephen Kitt suggested in February, and the correct function was identified by Yurij Mikhalevich right around the end of May.

So, to fix the panic issue, I needed to add the following to the kernel boot parameters:

initcall_blacklist=dw_i2c_init_driver

In the intervening months, I had my own hack around this: while trying to figure out when exactly the offending driver was loaded, noticed that shutting down worked fine if I booted directly to the systemd poweroff.target. So I added an additional EFI boot entry, which started the kernel with systemd.unit=poweroff.target, and then wrote a short script that enabled a one-time boot to that entry and restarted. So shutting down took a while longer, but worked fine as long as I called that script.

Multi-monitor and audio

So my setup looks something like this¹ — with a horrible mess of cables connecting the whole lot:

     +--------------------+
     |     Alien          |
     |                    |                                         +--------+
     +--------------------+                                         |        |
                                                                    |        |
  +-------------------+                                             |        |
  |                   |                                             |        |
  |                   |                                        +----+---+    |
  |                   |   +---+                                |Speakers|    |
  |                   |   |   |                                +----+---+    |
  |                   |   | M |                                     |        |
  |                   |   | o |                                     |        |
  |                   |   | n |                                     |        |
  |         Table     |   | i |                                     |   TV   |
  |                   |   | t |                                     |        |
  |                   |   | o |                                     |        |
  |                   |   | r |                                     |        |
  |                   |   |   |                                +----+---+    |
  |                   |   +---+                                |Speakers|    |
  |                   |                                        +----+---+    |
  |                   |                                             |        |
  +-------------------+                                             |        |
                                                                    |        |
                                                                    |        |
                                                                    +--------+

So my monitor, a 24” BenQ, is visually below the 46” Sony TV. The monitor is smaller, but much closer than the TV, so reading is far easier on it. As a result, it is my primary display when using the PC. However, the speakers are connected to the TV’s audio out, since I also use the TV via a Chromecast, and in the past via an HDMI switch that connected it to my laptop. So it makes sense for whatever’s using the TV to have its audio go via the speakers.

This is where the annoyance begins. Since both displays are connected via HDMI², both are available for sound output (and the monitor has a built-in speaker). Since the monitor is the primary, sound is usually preferentially routed through it, on both Windows and Linux (with PulseAudio). Windows does a better job of remembering after I select the TV as the output once. However, on Linux, every time I switch display configuration (from mirror to extend and vice versa), which I do quite often depending on whether I am gaming (mirror) or watching videos (extend), output is reset to the monitor. (Or worse, sometimes it picks the PC’s S/PDIF output, which has nothing connected to it at all!)

Worse still, the sound settings are not at all helpful here:

Quick: which would be the TV and which would be the monitor? :unamused:

Thus, I have two problems (and regex ain’t one of them):

1. Quick switching between display arrangements (GNOME Shell doesn’t have anything built-in for that, as far as I know).
2. Ensuring that the TV is the sound source, no matter what the arrangement is.

I tried fiddling with ~/.config/monitors.xml, but it seems that file isn’t watched for changes. So modifying that file doesn’t help. I tried an extension for GNOME Shell, but I quickly ran in to problem (2) above. So once I again, I settled on writing up scripts. Setting arrangements is easy enough with xrandr:

xrandr --output DP-1 --primary --below HDMI-0     # extend above
xrandr --output DP-1 --same-as HDMI-0             # mirror

Here, DP-1 is the monitor connected on the DisplayPort, and HDMI-0 is the TV on (surprise, surprise!) HDMI. It’s not difficult to see which is which:

% xrandr                                                                                                                                                                                        :(
Screen 0: minimum 8 x 8, current 1920 x 2160, maximum 32767 x 32767
HDMI-0 connected 1920x1080+0+0 (normal left inverted right x axis y axis) 0mm x 0mm
   1920x1080     60.00*+  59.94    60.05    60.00
   1440x480      60.05
   1280x720      60.00    59.94
   720x480       59.94
   640x480       59.93
DP-0 disconnected (normal left inverted right x axis y axis)
DP-1 connected primary 1920x1080+0+1080 (normal left inverted right x axis y axis) 527mm x 296mm
   1920x1080     60.00*+  59.94    50.00    60.05    60.00    50.04
   1680x1050     59.95
   1600x900      60.00
   1280x1024     75.02    60.02
   1280x800      59.81
   1280x720      60.00    59.94    50.00
   1024x768      75.03    60.00
   800x600       75.00    60.32
   720x576       50.00
   720x480       59.94
   640x480       75.00    59.94    59.93
DP-2 disconnected (normal left inverted right x axis y axis)
DP-3 disconnected (normal left inverted right x axis y axis)
DP-4 disconnected (normal left inverted right x axis y axis)
DP-5 disconnected (normal left inverted right x axis y axis)
DP-1-1 disconnected (normal left inverted right x axis y axis)
HDMI-1-1 disconnected (normal left inverted right x axis y axis)

The (far newer) monitor has far more supported modes. So it’s easy to parse them out and see which is which:

% xrandr |
   awk '/connected/{name=$1; next} name && /x/{count[name]++} END {for (i in count) print i, count[i]}'
HDMI-0 5
DP-1 11

PulseAudio

The tricky part, however, was PulseAudio. I find it confusing. There’s pactl and pacmd. There’s all those configuration files, default.pa and whatever else, written in a custom format, from what I can tell. Sinks, sources, ports, cards, …! ‘Tis enough to make one throw up their hands in despair!

Also, looking at the pamcd and pactl outputs, it’s obvious that trying to parse that deeply-nested structure using regex is just asking for trouble. However, PA doesn’t have first-party libraries for accessing this data from friendly languages like Python. There have been a few projects trying to expose such an API, but they don’t seem well-maintained or extensive in coverage. Worse yet, even though the pactl manpage says:

 pactl  only  exposes  a  subset  of  the  available  operations.  For the
 full set use the pacmd(1).

pactl actually seems to expose more information than pacmd. For example, here’s part of the output for pactl list cards:

        Active Profile: output:hdmi-stereo-extra1
        Ports:
                hdmi-output-0: HDMI / DisplayPort (priority: 5900, latency offset: 0 usec, available)
                        Properties:
                                device.icon_name = "video-display"
                                device.product.name = "BenQ GW2480
 "
                        Part of profile(s): output:hdmi-stereo
                hdmi-output-1: HDMI / DisplayPort 2 (priority: 5800, latency offset: 0 usec, available)
                        Properties:
                                device.icon_name = "video-display"
                                device.product.name = "SONY TV
     "
                        Part of profile(s): output:hdmi-stereo-extra1

The equivalent on pacmd list-cards:

        ports:
                hdmi-output-0: HDMI / DisplayPort (priority 5900, latency offset 0 usec, available: yes)
                        properties:
                                device.icon_name = "video-display"
                                device.product.name = "BenQ GW2480
 "
                hdmi-output-1: HDMI / DisplayPort 2 (priority 5800, latency offset 0 usec, available: yes)
                        properties:
                                device.icon_name = "video-display"
                                device.product.name = "SONY TV
     "

Two things of note:

1. A human-grokkable name (the product name) is available³ from PulseAudio! (Those are, in fact, what Windows shows!) Why TF doesn’t the sound settings use those?
2. pacmd, ostensibly the more “powerful” command, doesn’t show the profile associated with each port. It’s probably hidden away in some other sub-command, and I’d need to parse out the port name matching the product name and then go hunting.

At any rate, getting the correct profile for the output I want to use is now easy with pactl:

tv="SONY"
pactl list cards |
  awk -v "tv=$tv" '$1 ~ /Name:/ {name = $2} /device.product.name/ && ($0 ~ tv) {p = 1} p && /Part of profile/ {print name, $NF; exit}'

This gives the card name and the profile name, then I can do:

tv="SONY"
read -r card profile < <(pactl list cards | awk -v "tv=$tv" '$1 ~ /Name:/ {name = $2} /device.product.name/ && ($0 ~ tv) {p = 1} p && /Part of profile/ {print name, $NF; exit}')
pacmd set-card-profile "$card" "$profile"

Now all I need to do is wrap these up in scripts, have the display arrangement script set the sound output as well⁴, and use this neat extension to add a couple of icons to my panel, and bam! A click is all I need to switch profiles or fix sound if some bug caused the sound to go wandering around.

I think, all said and done, a better way would have been to use PA configuration files to set a higher priority for the TV. But I can’t be arsed to learn the configuration language. So: ¯\_(ツ)_/¯ My scripts work and I postpone PA for a time when I have far more free time.

This time around, I decided to go for the best graphics card I could sanely get (so, no Titan). That’s the 1080 Ti at the moment, with 11 GB of GDDR5 graphics memory! I considered System76 once again, which did have models supporting the 1080Ti. But … Dell had a discount going on the Alienware Aurora R7 (Ars Technica review of the 2016 model). After considering the discount, shipping charges and support options (Dell had a good discount going on accidental and on-site support coverage as well), Dell was proving to be cheaper, and I finally opted for it. Ordered, shipped and received in 2 weeks!

Specs:

• Intel® Core™ i7 8700 (6 core/12 threads, 12 MB cache, 4.6 GHz max)
• NVIDIA® GeForce® GTX 1080 Ti 11GB GDDR5X
• 256GB M.2 PCIe Toshiba SSD
• 2TB 7200RPM SATA 6Gb/s Seagate HDD
• 16 GB dual-channel DDR4 @ 2666 Mhz (2x 8 GB)
• A Blu-ray reader/writer²
• 7 USB 3.0-A ports (3 in front, 4 back)³
• 5 USB 2.0-A ports (all in the back)
• 2 USB 3.0-C ports (one each front and back)
• 1 HDMI port and 4 DisplayPorts
• Killer 1535 802.11ac 2x2 WiFi & Bluetooth 4.1
• 850 W PSU, liquid-cooled chassis
• 14 kgs!⁴

It came with Windows 10 Pro, and setting it up was pleasant. Logged in with my Microsoft account synced my theme and wallpapers (a nice touch). Set about installing Steam, other software I commonly use on Windows, and, of course, WSL. Installed HITMAN, cranked up the settings to Ultra at 1080p, and ran the benchmark to get an average of ~59 FPS! That’s head and shoulders above whatever the 670MX could do at low-mid graphics. (The min was 4 FPS, but that came from a stutter right at the very beginning). Even discounting the FPS values, the benchmark looked smooth and clear in a way it never did on my laptop. Installed Assassin’s Creed: Unity - no problems with that one either.

Of course, my setting up didn’t end there. The main quest was yet to be done: installing Linux.

Setting up Arch Linux

I must admit I had my worries about how well this thing would run Linux. The network card name (Killer?!) didn’t particularly inspire confidence, and that’s where most problems occur in a Linux setup - the LAN, WiFi and Bluetooth. And you don’t often to get hear about people running Linux with a 1080Ti. But if my experience with the 670MX meant anything, graphics wouldn’t be too much of a problem.

UEFI

Booting with a USB drive with Arch on it didn’t prove troublesome. Both the usual methods (use Windows’ advanced reboot option to get to UEFI, and pressing Esc during boot) worked and got me into the UEFI boot menu. As an aside, the Alienware boot splash is a nicely understated logo that I really liked. Couldn’t find an image that looked just like it, so I took a photo of it and processed it a bit to get this:

But that’s probably the best bit about the Alienware UEFI setup. The UEFI menu itself is … disappointing. It didn’t support widescreen monitors, and I have seen a few UEFI systems that do, so there’s that. Many UEFI systems also support screenshots, but this one doesn’t.

I needed to disable Secure Boot to get Arch to boot, as expected. Disabling Secure Boot enabled legacy boot as well, and the legacy boot splash is somewhat crappy. I didn’t want legacy boot, so I disabled that.

Once I selected the USB disk, Arch booted up right quick.

Partitioning

I’d shrunk the partitions that already existed, to split the ~200 GB Windows partition on the SSD in half, and to carve out a 500 GB space on the 2 TB disk. There used to be a time when I thought more partitions were a good thing, and I’d keep my data neatly divided into separate partitions (games on one, videos on another, etc.). When I last upgraded my laptop, adding a 1 TB disk to (in addition to the 500 GB it came with and the 500 GB disk from my older laptop), my partition layout looked like this:

~ lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT
NAME          SIZE FSTYPE            TYPE  LABEL       MOUNTPOINT
sda         931.5G                   disk
├─sda1       49.2G ntfs              part  Windows8
├─sda2        804M ntfs              part
├─sda3        500G ntfs              part  Downloads   /home/muru/Downloads
└─sda4      381.5G ntfs              part  Stuff       /home/muru/Stuff
sdb         465.8G linux_raid_member disk  ica-ext:0
└─md127     931.5G                   raid0
  ├─md127p1    40G ext4              md    arch        /
  ├─md127p2   100G ext4              md    devel       /home/muru/dev
  ├─md127p3   300G ext4              md    var         /home/muru/var
  ├─md127p4 483.5G ext4              md    archives    /home/muru/archives
  └─md127p5     8G swap              md                [SWAP]
sdc         465.8G linux_raid_member disk  ica-ext:0
└─md127     931.5G                   raid0
  ├─md127p1    40G ext4              md    arch        /
  ├─md127p2   100G ext4              md    devel       /home/muru/dev
  ├─md127p3   300G ext4              md    var         /home/muru/var
  ├─md127p4 483.5G ext4              md    archives    /home/muru/archives
  └─md127p5     8G swap              md                [SWAP]

In hindsight, that is just … insane. All those partitions meant that I soon ran out of space in some of them. And of course, naïve as I was, I only granted ~50GB to Windows. This time around, I eschewed all that:

~ lsblk -o NAME,SIZE,FSTYPE,TYPE,LABEL,MOUNTPOINT /dev/sda /dev/nvme0n1
NAME          SIZE FSTYPE TYPE LABEL       MOUNTPOINT
sda           1.8T        disk
├─sda1        128M        part
├─sda2        1.3T ntfs   part Downloads   /home/muru/Downloads
└─sda3        500G ext4   part dev         /home/muru/dev
nvme0n1     238.5G        disk
├─nvme0n1p1   500M vfat   part ESP         /boot
├─nvme0n1p2   128M        part
├─nvme0n1p3 124.5G ntfs   part OS
├─nvme0n1p4   450M ntfs   part WINRETOOLS
├─nvme0n1p5  11.9G ntfs   part Image
├─nvme0n1p6     1G ntfs   part DELLSUPPORT
└─nvme0n1p7   100G ext4   part arch        /

Aside from the partitions already present, I added just two. Not even a swap partition. Another upgrade I’d done to my laptop was to add 16 GB RAM to it, for a total of 24 GB. And since then, I noted that swap was never used and I don’t think I have seen it use more than 16 GB RAM. Even if I need swap, I’ll probably use a swap file. No hibernation for me.

Installing

After mounting, I realized that I’d forgotten to check the part that I’d worried most about: the network. And to my pleasant surprise, both LAN and WiFi were working perfectly fine OOTB:

# lspci | grep -i net
05:00.0 Network controller: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter (rev 32)
06:00.0 Ethernet controller: Qualcomm Atheros Killer E2500 Gigabit Ethernet Controller (rev 10)

I’d never heard of “Killer” network cards before, and I didn’t look them up before I ordered, figuring I’d work something out if the card turned out to be poorly supported. But nah, we’ve plain old Qualcomm Atheros here.

It was a simple matter of:

mount /dev/nvme0n1p7 /mnt
mkdir -p /mnt/boot/EFI
mount /dev/nvme0n1p1 /mnt/boot/EFI
pacstrap /mnt base base-devel gnome

And the usual steps in the Installation Guide. (I’d accidentally mounted the Windows partition instead of the Linux one on /mnt, but pacstrap is non-destructive, so no harm done).

One thing I intended to do this time was to skip GRUB and boot directly from UEFI. This is possible because the Linux kernel from Arch has EFISTUB. While using efibootmgr to set this up, I realized that the ESP was best mounted on /boot, since the only thing /boot contained was the kernel image and the initramfs, both of which had to be copied to the ESP anyway. So, remounting /mnt/boot, and using these:

kernel_params="root=UUID=e5018f7e-5838-4a47-b146-fc1614673356 rw initrd=/initramfs-linux.img"
efibootmgr --disk /dev/nvme0n1 --part 1 --create --gpt --label "Arch Linux" --loader /vmlinuz-linux.efi --unicode "$kernel_params"

I got a new UEFI boot entry created and set as default automatically. The command is long-winded enough that I saved it to file in root’s home directory, which turned out to be pretty handy later.

Next step was to edit /etc/mkinitcpio.d/linux to get it to create the initramfs files in the locations listed above directly (using the mkinitcpio(2) method from the Arch Wiki).

Timezone set, fstab created, hostname set … and reboot!

Borking

Bork. Rebooting didn’t work, it looks like I messed up the boot entry. I’d missed the part where the paths were supposed to be rooted at the ESP partition. So, rebooting back to the USB, chrooting again, and:

kernel_params="root=UUID=e5018f7e-5838-4a47-b146-fc1614673356 rw initrd=/EFI/arch/initramfs-linux.img"
efibootmgr --disk /dev/nvme0n1 --part 1 --create --gpt --label "Arch Linux" --loader /EFI/arch/vmlinuz-linux.efi --unicode "$kernel_params"

Reboot again, and I’m dropped into a TTY. :disappointed: I had installed gnome after all. Of course, I forgot that Arch doesn’t automatically enable services, so logging in, and systemctl enabla --now gdm later, I had … GDM crashing in a loop. I had both my TV and my monitor connected, so I tried disconnecting my TV, which stabilized GNOME and allowed me to login. Turns out GDM itself runs in Wayland, and while Wayland didn’t have problems on a running on a single screen, a dual display setup caused it to crash.

So, getting it to use Xorg instead:

sed -i '/WaylandEnable=false/s/^#//' /etc/gdm/custom.conf

And GDM worked fine with dual displays from then on.

Logging in, my next point of trouble: no networking! :sob: Panicked, I checked ip a and lspci to realise that the devices were detected fine, but I’d forgotten to install NetworkManager. A quick run of dhcpd later, I had network.

Steam

After that, it was relatively simple to install the other software that I usually use, and setup a LAN cross-connection to my laptop to start copying files (I used tar c | nc/nc -l | tar x for speed). I ran into some trouble getting Steam to start, turned out to be missing 32-bit libraries:

libXtst.so.6
libXrandr.so.2
libgobject-2.0.so.0
libglib-2.0.so.0
libgio-2.0.so.0
libgtk-x11-2.0.so.0
libpulse.so.0
libgdk_pixbuf-2.0.so.0

Later on, vdui2.so couldn’t be loaded. Then curl wasn’t loaded.

All told, I needed to install:

# xargs -L1 -a ~muru/dev/steam-libs pkgfile -s | grep multi
multilib/lib32-libxtst
multilib/lib32-libxrandr
multilib/lib32-glib2
multilib/lib32-glib2
multilib/lib32-glib2
multilib/lib32-gtk2
multilib/lib32-libpulse
multilib/lib32-gdk-pixbuf2
# And
lib32-openal
lib32-curl

steam-native still refuses to start, but steam-runtime works well enough.

And finally:

54 FPS! A 10% drop compared to Windows, but still pretty good for a port. My gaming will no longer be as frustrating as it was before. :sunglasses: :neckbeard:

Who knew an Alienware would be so comfortable for Linux? Besides goof-ups I made because I hadn’t installed Arch in ages and didn’t pay attention to the Wiki, it was a pretty easy install. It would have been easier still if I’d gone for Ubuntu, I suspect. A far cry from some of the laptops I have installed Linux on for friends, maybe the year of the Linux desktop is indeed at hand.

Whenever someone asks, I usually say I preferred IITG to IITB (and I guess, as my alma mater, it will always be №1 for me). However, in many ways, my time IITB was richer.¹ I feel like I experienced more here, that the highs were higher and the lows deeper, but that could simply be the effect of a slight increase in senility. IITB also felt a lot more like home, probably because home itself is so close. Like the moon giving a pale reflection of the sun, the campus had a comfort, reflecting the comfort I felt at home. It helped that most of my time here was spent doing things I enjoyed (cough not Mechanical Engineering cough :grin:).

So what did I do on my last morning here? I climbed a hill, of course. :stuck_out_tongue: It was an endeavour I’d been putting off for nearly year; time ran out and now my hand was forced. The last time I was there, I got a few delightful pics of the black kites that frequent the campus. And since these birds didn’t seem to mind the rain in July and August, I hoped to see a few of them today, since a sunrise seemed hopeless. Well, not one turned up. I barely heard the echo of a kite’s scream today. :unamused: It was a very humid, very wet morning, and I didn’t get much to photograph.

In a way, the trip reflected both what I wanted from time at IITB, and what it actually was. I wanted to set goals, put in the effort and achieve those goals. Instead, I spent most of time coasting, leaving me with regrets and missed opportunities. And much like the trip up the hill, I’m not sure my time here was all that fruitful — I leave here with the distinct impression that I haven’t achieved what I came here to do.

A few pics to cheer up a depressing post²: :smile:

For example, the main IITB CSE website (http://www.cse.iitb.ac.in) is HTTPS-only, however this is only enforced for the department website. User pages and parts of the site proxied to other servers aren’t HTTPS-only, much as I’d like them to be. I had initially tried enforcing HTTPS for the entire site, but it broke some old pages that the users were unwilling or unable to fix.

Of course, it’d be hypocritical of me to not offer or enforce HTTPS on my own site. That, however, is difficult to do. Github Pages didn’t support HTTPS at all earlier. (Now it does, albeit unofficially.) It still has no option to support HTTPS for custom domains.

Recently, two things happened:

1. I had occasion to visit a friend’s website, and he had HTTPS setup and enforced.
2. Gitlab started offering their own Gitlab Pages service, which did support HTTPS for their Pages domains from the start.

Ever since I installed Gitlab for the CSE Git server, I’ve been a fan of their work. Their monthly release schedule meant that bugs were fixed and features added regularly. Integration of a CI implementation and Mattermost was a plus. So, when Gitlab Pages was announced, I was interested. It supported more Static Site Generators (SSGs) than did Github (which only supported a crippled Jekyll). You could use practically any Jekyll plugin you wanted, since the command to generate the site was specified by you as CI configuration. I didn’t see much reason to migrate at the time, since I’d have to add CI configuration, and there seemed to be not much benefit with having my page available from four (!) places (#visitors ≤ #sites :stuck_out_tongue:).

Seeing the friend’s site gave me impetus to investigate my options for HTTPS once again. This time, there seemed to be some updates:

1. Using CloudFlare to provide HTTPS is a commonly-suggested method.
2. Other options offering a similar workaround have come up, like Kloudsec.
3. Let’s Encrypt has taken wing.
4. And finally, Gitlab Pages started supporting custom domains with HTTPS.

First, I tried the CloudFlare way. That was fairly easy, following this blog post. Register with CloudFlare, add your domains, switch nameservers with the ones provided by them, and then start toggling the options. Of interest:

1. You have to enable both DNS+proxy with CF for HTTPS, HTTPS redirection and HSTS to take effect.
2. The Full (Strict) option is what you need to make CF use HTTPS with the actual server and check certificate validity. This won’t work with Github Pages.
3. As usual, changing nameservers takes a while. So, for immediate testing, I added the CF nameservers to /etc/resolv.conf. (And looked up the IPs for use in /etc/hosts, just in case.)

The annoying bit is that the Github Pages site, which redirects to my custom domain, still prefers HTTP. So: https://murukeshm.github.io → http://murukesh.me → https://murukesh.me. :unamused: And if you do enable Full (Strict) SSL, down goes your site.

And so the hunt continued. Like I said, Gitlab Pages had started some time ago, but I didn’t know that it support custom domains with HTTPS support. I learnt about that while browsing the Github issue about HTTPS support on Pages, from @konkone’s comment.

So, off I went to Gitlab.com, where I already had an account I made for participating in Gitlab CE issues. I imported the repository from Github, renamed it, and added a .gitlab-ci.yml to kickstart the Gitlab CI’s page building process. I lifted the contents from the default Jekyll setup for Gitlab. The build started, and took a couple of minutes to finish. That’s not a big deal, since updates to Github Pages may take upto 5 minutes, and with Gitlab, you can see the status of the build, and the live log output as well. In particular, the latter is very useful to spot bugs.

Now came the most tiresome part: actually getting a certificate for murukesh.me. There are a few options for getting a free certificate. I decided to try Let’s Encrypt (instead of StartSSL, just for the sake of it). So, I installed the letsencrypt package on my Arch Linux system, and ran the tool. And, well, apparently by default it’s designed to run on the server hosting the site you want to encrypt. :neutral_face: You have to pick the manual option, and this, I discovered too late, is already well-described in Gitlab’s tutorial on using Let’s Encrypt.

It’s still cumbersome though:

1. You have to add the domains on Gitlab without the certificates.
2. When you run the letsencrypt tool, it generates some text you have to add to your site. Annoyingly, it does so once each for each domain listed, and it waits for each addition to finish. So, two domains, two commits.
3. Then, after LE has generated the certificates, you have to remove and add back the domains on Gitlab, since it doesn’t support editing an added domain to add a certificate. :tired_face:

After all that, I do have HTTPS on both murukesh.me and murukesh.gitlab.io. (Side note: Yay for getting in early enough to get murukesh as my ID!) However, Gitlab still doesn’t support redirection of HTTP to HTTPS. So, I’m back to using CloudFlare to add HTTP-to-HTTPS redirection and HSTS for my site.

There’s also the problem of renewing the Let’s Encrypt certificate, which lasts around three months. Apparently there are plans to integrate LE with Gitlab Pages to automate the whole shebang. There’s also a recent feature request to enable redirection of HTTP to HTTPS.

Here’s to hoping both of them getting implemented!

Update (June 13, 2016)

On June 8, Github announced official support for HTTPS on their user pages. Not only are *.github.io pages available on HTTPS like *.gitlab.io ones, you can also force them to be HTTPS-only, as long as you don’t use a custom domain. That means I’ll probably have to remove the CNAME file from the repo. Your move, Gitlab!

I needed to set it up so that the site would work across the three ways it can be accessed:

• Github Pages
• Personal domain
• CSE homepage

Github Pages, while decidedly convenient for starting out, is not so convenient when it comes to expanding. Not that Jekyll is all that convenient either.

1. You cannot easily publish your blog posts under a subdirectory
2. Posts cannot have a counter-based ID. The permalink can only be based on the date and title.
3. You have to rely on external providers for comment support. Maybe Disqus, or Facebook, or something else.

But…

1. Markdown is fun to write in. It is clear, logical and the source is easy to read.
2. Jekyll seems to do a good job of templating, without too much boilerplate.
3. Having things like code highlighting taken care of is quite convenient.

Setting up Jekyll is quite easy - the site has good instructions. I’ll just focus on things I had trouble doing.

Page permalinks

On Github, pages could be accessed both with and without an extension (/index and /index.html both worked fine). Naturally, I stuck to the extension-less form. Running Jekyll locally, I ran into trouble - it did not support extension-less access. I’d guess this is presumably because the Github Pages setup has something like this (in terms of nginx config):

try_files $uri $uri.html ...;

Annoyed, I resorted to specifying the permalinks manually. After all, I did only have four pages - the rest generated as posts. And index.md didn’t need any help, so the mess is down to three pages.

Code Highlighting

Displaying code proved to be a tricky thing. I’m software guy, naturally my post will include bits of code here and there. I want it to be pretty (syntax highlighting, line numbers, the works). Jekyll offers two ways for prettifying code: Pygments and Rouge. Pygments is written in Python, and Rouge in Ruby. So, naturally, you might think Rouge is the way to go, since the whole damned thing will be in Ruby. Nope.

The thing is, sticking to what Github Pages offers really does constrain you. And Pages doesn’t support Rouge. So, Pygments is the way to go, and is the default.

The first time I tried it out, the result was some consternation. You see, back when I first made this site for the CS699 course, I’d set display: block for span tags. And Pygments relies heavily on span tags with CSS classes for Highlighting. So, where I’d hoped to see something like:

I got (without the colours):

Flabbergasted, I decided to try out Rouge, and it seemed to work fine, except I had barely any highlighting - the output was marked up, but I didn’t have corresponding CSS. The docs suggested picking up a sample syntax.css which is purportedly close to Github’s own style. Ah… But I wanted a dark theme, and I wasn’t too fond of Github’s theme as it is. After a bit of going around in circles, I realised Rouge wasn’t supported by Github, and so I went back to Pygments. A quick inspection showed me what the problem was, and just as quickly I deleted the offending CSS rule. I had no idea why it was around, since I used barely any span blocks - at least, I couldn’t see any visible effect on the site! With that problem solved, two remained: the colour theme conundrum, and line numbers.

Like with Rouge, the next problem was CSS. Thankfully, the step to get a CSS file for Pygments was easy:

pygmentize -S <theme> -f html > foo.css

After a few trials, I chose the normal theme.

Line numbering was harder. Oh, both could do numbering, but the numbers would be selected too when you selected the code, and worse, copied. Apparently, Pygments supported a table option to linenos, but it looked weird. There was a delightful solution that involved using lineanchors instead of linenos, and then applying a CSS counter to create the numbers. Unfortunately, lineanchors doesn’t work when Jekyll runs in safe mode, which it does on Github. Then I applied the same technique to linenos, and hid the numbers generated by Pygments :mask::

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
pre {
    counter-reset: line-numbering;
}

pre span.lineno::after {
	content: counter(line-numbering, decimal-leading-zero);
	counter-increment: line-numbering;
	opacity: 0.4;
	-webkit-touch-callout: none;
	-webkit-user-select: none;
	-khtml-user-select: none;
	-moz-user-select: none;
	-ms-user-select: none;
	user-select: none;
	visibility: visible;
	margin-left: -1em;
}

pre span.lineno {
	visibility: collapse;
	text-align: right;
	display: inline-block;
	min-width: 1em;
}

Sectioning

By default, the entire post content is stuffed into the content Liquid variable. If you want to split your post into sections, tough luck. This StackOverflow post helped me out. One of the answers talks about the post.excerpt feature, where you can use an excerpt_separator to demarcate out the post blurb, in case you want something different from the default. Turns out, the idea can be easily extended to a generic separator. For example, add to your _content.yaml:

section_separator: "<!-- section -->"

And create a new layout (say _layouts/sectioned_posts.html) containing (aside from the boilerplate):

1
2
3
4
5
6
{% assign sections = content | split: site.section_separator %}
{% for section in sections %}
<section>
{{ section }}
</section>
{% endfor %}

Your source will look like:

1
2
3
4
5
6
7
8
9
10
11
12
Call me Muru.

<!-- section -->

I am an aspiring BOFH. Often called a psycho. Now in my third year of the Master
of Technology in Computer Science and Engineering course in IIT Bombay, and
working for the department as an RA in System Administration, I get plenty of
opportunity to hone my skills. :)

<!-- section -->

I share the hobby of the masses - reading. :P

See my front page for the output. :)

Comments

The last thorny issue was comments. I wanted to enable comments, even if the site had tumbleweeds rolling around. Given the static nature of the site, storing comments here was out of the question. That left me with external providers.

I tried Google+ comments, but text area never loaded properly. Using Github issues seems to be a nice idea, but I’m too lazy to follow through on it. I tried Facebook comments but an odd issue popped up: when I clicked on the text box on my phone, the orientation was forced to landscape, even though the device hadn’t rotated. So, I ditched it and went to Disqus. After styling it nicely, I opened my phone to test … and ran into the same problem. Soon after posting on the Disqus forums, I found the problem: Android silliness. The soft keyboard eats in to the page display area, thus changing the height and hence the orientation.

I wanted to consider both the height and width, so the accepted answer (using max-width) was not acceptable to me. The next-best solution used boundary values for aspect-ratio different from 1/1. After some more digging around, I decided to use device-aspect-ratio instead.

Of late, with Vim.SE for support, I’ve been customizing Vim more and more. I’ve made a Git repo of my Vim files, taken baby steps in automating tasks I often do, and so on. While looking through the recent posts in Unix.SE, I came across this post which suggested using your editor as the pager. That kicked up the dusty cobwebs in my decrepit memory module, and I remembered that old attempt at using Vim for reading manpages. So, I set about trying to make Vim man’s pager. Why did I submit myself to such cruel and unusual punishment?

1. I like Vim.
2. I have it customized to my liking.
3. It is powerful. The search is way better than anything less or your average manpage browser (like yelp) can offer.
4. It can browse to other manpages mentioned using tag navigation (<c-]>, <c-t>).

The post suggested setting $MANPAGER to a combination of col and vim:

export MANPAGER="col -b | vim -c 'set ft=man nomod nolist ignorecase' -"

For decidedly non-obvious reasons, it’s not likely to work for you. Why? Because GNU man doesn’t support piped commands in $MANPAGER – BSD’s man does (that’s +1 for you OSX folks). From man man:

MANPAGER, PAGER
   If $MANPAGER or $PAGER is set ($MANPAGER is used in preference),
   its value is used as the name of the program used to display the
   manual page.  By default, pager -s is used.

   The  value  may  be  a  simple  command  name  or a command with
   arguments,  and  may  use  shell  quoting  (backslashes,  single
   quotes,  or  double  quotes).   It  may not use pipes to connect
   multiple commands; if you need that, use a wrapper script, which
   may  take  the  file  to  display  either  as  an argument or on
   standard input.

I tried the suggested solution (using a wrapper script), which worked fine. However, it created a problem: I use Git to manage my dotfiles. I’d rather not rely on stuff outside the repo. Stuff installed by package managers and differences per distro are a fact of life and have to be handled, but I’d rather not take pains over what I add to it. One obvious solution is to wrap the command in sh -c:

MANPAGER='sh -c "col -b | vim -c \'set ft=man nomod nolist ignorecase\' -"'

Ugly. I also hate having to deal with quoting.

At this point, it struck me: Why should I run this via a pipe? Once Vim starts, I can perfectly well use %! col -b to do the job. So:

MANPAGER='vim -c "%! col -b" -c "set ft=man nomod nolist ignorecase" -'

Nice!

Now, other considerations started popping up. You can easily quite less (and by extension, man), by pressing q, or CtrlC. Vim usually considers a buffer read from stdin to be modified. Therefore, to quit a manpage, you’d have to do :q!, not just :q. Thankfully, one of the options set (nomod) tells Vim that the buffer hasn’t been modified. Therefore, we can just use :q:

nnoremap q :q<CR>

Other considerations arise:

• The buffer is modifiable. There’s no reason for it to be so.
• The buffer doesn’t have a name. It would be convenient to see the name of the manpage.
• You don’t want swapfiles hanging around from manpages.

As I pondered over this, I realised that these are settings I’d want to apply to a manpage no matter how I opened it. Hence, they should really be in Vim’s filetype settings for man. So, I created a ~/.vim/ftplugin/man.vim, containing:

1
2
3
4
5
6
7
8
9
10
11
12
function! PrepManPager()
	if !empty ($MAN_PN)
		silent %! col -b
		file $MAN_PN
	endif
	setlocal nomodified
	setlocal nomodifiable
	setlocal readonly
	setlocal nolist
	setlocal noswapfile
endfunction
autocmd VimEnter * PrepMan()

I picked VimEnter since it runs after any commands specified using -c are run, so I can get it to run after the filetype has been set.

However, I realised that:

• I wanted to apply some of these settings to manpages irrespective of how they were opened; and
• I’d rather not specify set ft=man from the command line, keeping an eye on using Vim as a general-purpose pager;
• Using VimEnter * felt wrong.

A bit of experimentation later, I found that:

1. man doesn’t seem to ever provide a filename as an argument, irrespective of what the manpage says.
2. man sets MAN_PN to the manpage name (man(1), for example)

Knowing that I’m reading from stdin and that MAN_PN is set (to the manpage name!), I came up with this version:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
" vimrc
if !empty($MAN_PN)
	autocmd StdinReadPost * set ft=man | file $MAN_PN
endif

" ftplugin/man.vim
setlocal nolist
setlocal readonly
setlocal buftype=nofile
setlocal bufhidden=hide
setlocal noswapfile
setlocal nomodifiable

function! PrepManPager()
	setlocal modifiable
	if !empty ($MAN_PN)
		silent %! col -b -x
	endif
	setlocal nomodified
	setlocal nomodifiable
endfunction

autocmd BufWinEnter $MAN_PN call PrepManPager()
nnoremap q :qa<CR>
nnoremap <Space> <PageDown>
map <expr> <CR> winnr('$') == 1 ? ':vs<CR><C-]>' : '<C-]>'

with:

export MANPAGER="vim -"

Beautiful!

What does this do?

1. In the main vimrc, I check if I’m reading from stdin and if MAN_PN is set. If so, set the filetype to man and the filename to the contents of MAN_PN.
2. In the filetype-specific setting, use an autocmd the relies on the filename being $MAN_PN to apply col -b.
3. Set nomodified to tell Vim that the buffer hasn’t been modified, and make it a read-only, non-modifiable, scratch buffer.
4. Also, map q to :qa, so that I can quit all opened manpages, and Space to Page Down, in keeping with the usual behaviour of less.
5. col -b’s use of tabs led to messed up alignment. I had to use -x (replace tabs with spaces) so that, for example, man ascii showed up properly.

Finally, man man opens up pretty much as I’d like it to.

Why “pretty much”? man obeys MANWIDTH, so I can get a manpage formatted exactly as wide as I want. If open a manpage within Vim, however (by navigating the tags, for example), the page is formatted for the full width of Vim. :( Secondly, Vim leaves this annoying message:

$ man man
Vim: Reading from stdin...

$

For the moment, I’ve adopted the decidedly un-Vim-like solution of opening a split before navigating to any tags - half the terminal width is fine for me. That’s what the last mapping in the above snippet does: check if I have only one window open, and if so, open a new window before jumping to the tag - with the added benefit using to the thoroughly intuitive (to me) Enter key for jumping to the named page. As a happy side effect, I get to see exactly where I was in the new window! :)

I have no idea how to suppress the stdin message from Vim itself.

All told:

Footnote

This is my first blog post using Jekyll. Writing it, I have learned quite a bit, which I will write about in another post soon.