Piperka blog

New site, first impressions

I'd say that the site transition went quite smoothly. The crawler had a hiccup where one faulty parser code failed silently with old library versions but killed the whole run with the new ones. Some redirects went nowhere but I caught on to that bug in short order. The Perl crawler code used a funny memcached key name for notifying about crawler progress and I needed to switch to another library with that. This is one of those situations where I was happy to receive just a couple of emails from my users. All in all, it all was comparatively boring. Boring is good.

Last week, I've been setting up various background jobs and testing the old site maintenance scripts with the new site and making some updates to them. One thing that didn't quite work was the submission handling code. The users with moderator rights weren't actually able to make edits. That bug was to be expected since I had written that part in a hurry and very late in the project when I thought I was about to get the code live.

There was a problem with users' timestamps with regards to comic additions. You may have noticed that Piperka was telling you about a couple of new comic entries for most of a day last Thursday and that notification didn't reset properly. I added a new timestamp column for that logic and now they seem to work properly again.

I found out that the script that I use for adding the comic's entry after the initial crawl that adds the archive page's to Piperka's index failed badly with comics with banners. The error looked very much like a stack trace, I didn't even know that Perl had those. I very quickly decided to not even try to fix that code and instead made the first fully new feature for the new code base: Adding comics to Piperka now works via the web page. In its current form it still is only usable by me since the initial crawling needs to be done with the old maintenance scripts on a shell account. Nonetheless, it's a step towards reducing site maintenance being tied solely to me.

Piperka now offers HTTPS connections. Thanks to Let's Encrypt setting it up was really easy. I could have done it with the old site already but I wanted to have one fewer moving part for the transition, despite the long delay. I still didn't set up a redirect from the HTTP side to HTTPS. I want to do it on a Sunday when there's less traffic but I was busy coding yesterday. Next week, then. Feel free to update your bookmarks to use "https://" already.

I'm afraid the unofficial Piperka Android App is currently inoperative. It's due to changes to Piperka's backend. I tried to retain compatibility but with other time constraints I didn't use all that much time on it or test it. I'm not its author and I can't as such quite offer support for it but I'd still rather keep it working. I may need a few days to find time to figure out what's going on with it.

I just figured out that Piperka's email sending was offline for last week. The SMTP process was supposed to be running and I did test it but apparently it was shut down at one point. The transition process was a whirlwind and it may have been due to some half finished action of mine. I'll keep an eye on it. Piperka uses email for password recoveries and for confirmations of newly added comic entries so no that large harm with that. I assume people would have told me if they tried password recovery and never received one.

I've talked much about OAuth2 and it didn't immediately work since I still needed to refresh the configuration on the providers' side. It should work now. I'll yet add at least Facebook as a login option.

The new site keeps a couple of Piperka server processes running and nginx acts as a proxy to forward traffic to one of them. Deploying changes involves zero downtime as I replace the secondary server process with an updated one and once it's done, just update the proxy port in nginx's config and reload it. I also have the option to instantly rollback to the last version if the new one turns out fail somehow. A sysadmin at work suggested this setup for me and it seems to work just fine, so far.

The point of no return was on last Friday. I had three weeks of overlap with the server contracts. I was a bit hasty with signing up with the new one but it's all well in the end. The contract period for the old server has expired and it's gone now, forever. Luckily it was soon clear that I had no need for that contingency plan. Whatever odd ends there still remain are well manageable. I was able to retain user sessions and nobody (as far as I know) was kicked out and needed to relogin. One issue that I suspect is that the account page won't accept password without logging in once, first. No need to log out even, it can be done with an incognito window. The account page's password check doesn't know about old style passwords.

I'll be giving a talk at work on an internal functional programming event day about Haskell and how to write a web site with it in a couple of days.

I'm considering that the next feature to implement is a ticketing system. I still need to act on them with the same old clunky scripts and raw SQL queries and the uptake may still not be all that great initially but I know that crawler bug reports very likely end up being just ignored if I take any more to my email. My goal is, naturally, to have much less to ticket about in the end. It's all too easy to spot stale comic entries currently.

At first, I was simply exhausted after getting the new code live but I'm slowly starting to see the new possibilities. I'm happy about the new backend code and it's much faster to do new feature development based on it.

submit to reddit
Mon, 19 Feb 2018 20:23:34 UTC