Piperka blog

Piperka migrated, part 2

I'm writing this one so that I might remember it better: Piperka's backend process runs as a systemd user service. The default mode for them is to shut down when the last user session closes. That means that everything is fine as long as I'm logged in but it shuts down down when the last ssh connection closes. The command to change services to persist is loginctl enable-linger which will make Piperka run even after I close the laptop lid. At least I had the fortune of having an SSH open from another computer until I did a reboot this morning to change the hostname. I had almost two hours of outage due to this today but I suppose I got lucky since I didn't cause any more of that. Sorry about this, this was a pure user error and I'll try to not make the same mistake the next time around. I should write some documentation about the process or even automate it somehow, this certainly wasn't the last server migration I'll be doing.

I'm not the first user of my new IP address. I don't mean to reflect badly on my new hosting provider but looks like the last holder of the address gathered some bad reputation. I was told that an anti-malware site was flagging Piperka after the migration. I contacted them and they told me that they had blacklisted my IP. I'm hoping that they'll clear the status soon if they haven't done it already.

Also, Google is rejecting all email from piperka.net at the moment, citing that my IP has sent bulk email before. I added DKIM, SPF and DMARC (email security and anti-phishing features) for my domain and I hope that that's enough to clear that one up. At least my IP isn't included in any of the public anti-spam blacklists.

Hopefully I'm not seeing any other issues like these. I'll just resolve them and it's not worth it to consider requesting a new IP over this or even to switch providers this soon. It's just frustrating that these aren't anything that I have direct control over.

Here's to hoping that I'll have a dreadfully uneventful and boring time with my new hosting and can concentrate on some feature development again.

Addendum: This one slipped my mind. I knocked out the crawler for more than half a day on Thursday. It happened at a bit inopportune time for me. Something in the new environment didn't agree with one of the parsers and that aborted the whole run. Sorry about that.

submit to reddit
Fri, 21 Jun 2019 17:37:37 UTC

Piperka migrated

I moved Piperka to Hyperhost.ua today. I've had a few problems with my last hosting provider. The server I've been using has had a number of hang ups that I've had to resolve by issuing a remote reset on the control panel and their network has had more outages and slowdowns than what I'd like. Though as far as I can tell they've improved on that last respect lately.

I was hoping to stay on my previous server for a while longer but I decided to take the leap already now. I had no real need for the dedicated server and this time Piperka's on a VPS. I suppose it's okay. The hardware's newer and page renders are noticeably faster with it since that's mainly single core load. If you don't see it yet then you're likely still using the old DNS entry and have your traffic proxied through the old server. I had extravagant amounts of RAM and disk space on the old server but I was hardly using them.

My sysadmin friend at day job talked to me about streaming replication for my DB but I didn't listen to him this time around. As far as I'm concerned the 15 minutes of outage it took to dump, copy and restore the DB was quite acceptable. Everything seems to work now on the new server. It took a bit of manual work to set up everything on the new server and I hope to not have to think about this in a long while again. At least moving over was a lot less work than last time around.

If you're in need of any hosting for yourself then feel free to use the affiliate link that I gave above. Time will tell how this'll work out but it's looking good so far and they have received some very good reviews.

I'll keep the old server for the rest of the week but I certainly hope to not have to revert this change. I even went as far as to upgrade my PostgreSQL version along with the migration. I'll have to go with stale data if I must go back.

To top it off, I'm even paying less for the new hosting. Let's see if I can outgrow it.

If you see anything out of the ordinary then please let me know. I've tried to grab everything relevant from the old server and I'll have a second look tomorrow but it's still easier to fix any straggling issues while I still have it.

submit to reddit
Tue, 18 Jun 2019 18:21:26 UTC

App done (for now)

I've ported the Piperka client app to Android and it's been available on Google Play as of May 20th. I put the badge on the site template for a while but I'm moving that to just the new user landing page and the about page a while ago. I think you know that it exists by now. I marked the app as having ads but I didn't yet actually implement those. They'll be added in a later update, perhaps.

As of now, the uptake has been quite low. I'm seeing much more activity from the Sailfish version. There's still more I could do with the app, like using notifications to alert about updates, but I'm moving on to other things for now. If and when the app would gain some degree of popularity I would return my attention to it. I suspect that I'm just old school in that I don't have much of an intuition about what makes mobile devices' users tick. I don't use them much myself and as Piperka's design is mostly guided by what I'd like to use myself so I wasn't on a familiar territory when making the app. I hope it'll find some users at least, this kind of was what took most of my development time this spring.

I'm retiring the unofficial Piperka app that's been around for a few years now. Other people are welcome to write third party apps for Piperka (or to send pull requests to mine) but I had to poke a few holes to Piperka's authentication system that I never liked to make that particular app work and now that I have mine published I'm going to require other implementations to have that same base level. If there's something that it did that you'd like the new app to do then please let me know.

I'm going to focus on crawler improvements next. Much of that will be invisible and aimed at catching or even automatically fixing issues before anyone even submits a ticket but one feature that I plan to get out of this is to show comics' update schedules. I have enough data to tell if a comic updates on, say, Sundays or on Mondays and Fridays. I'll just have to look it up and display it.

submit to reddit
Wed, 12 Jun 2019 16:17:04 UTC

Next step, Android

First off, a couple of notes about monetization. I promise I'm not trying to annoy you with it. I've added Google's AdSense ads as an opt in feature. They're not the kind of ads I'd really want for Piperka so I'm not making them a default, but since they were pretty quick to add (along with the code to not show them) I went with them. Enable them if you don't mind them and don't feel bad if you don't.

I've joined Piperka to Brave's verified site program. If you don't use their browser then this doesn't concern you in any way. If you like, use this referral link and install and use it and I may earn some of their BAT. It's a privacy oriented browser but I can't vouch for it personally as I'm not a user.

I didn't post anything about it on the blog at the time but I released Piperka client for Sailfish a bit over month ago. My goal for May is to port it to Android. I've already got that far with it to get the C++ portion of the client to run on it with some very basic QML layout, which is already quite a lot since the compiled code does most of the heavy lifting. I'm afraid it won't look as pretty as the Sailfish version, at least at first. Their QML objects were opinionated and I could line my app pretty well along their assumptions and as a result I got quite a few things for free for my app. I'm happy with the reception I've got there so far. It's a small platform so the volume was never going to be that big but it did okay, considering.

I had some unplanned service outage on 2nd of April. The whole data center where Piperka's hosted was offline for some 7 hours. I've been toying with the idea of moving Piperka to a fault tolerant distributed cluster with a hot spare node ever since. Some would say that it would be overkill for a site like Piperka and it could come with possible new modes of failure of its own. But it's not a near term thing, either way.

I made a new small addition to sort types. Sort by new subscriptions shows the comics which have gained and lost readers recently. The data is refreshed daily after the readers count history data collection. It's something I've been wishing to keep track of myself and there was no reason not to make it available for everyone to see as well.

After the Android app I'm thinking of doing further improvements to the crawler. I've outlined some of the things I could be doing with it earlier on this blog. Just the kinds of things that you'd never see or notice if they work as they should.

I suppose I was a bit exhausted after getting the Jolla app published and haven't had much focus in April. I expect to pick up the pace again.

submit to reddit
Wed, 01 May 2019 07:28:14 UTC

Mobile progress

I've been busy with coding the mobile client and I'm hoping to release a first version in a couple of weeks. I've just published what I have so far on Gitlab. I put it under GPLv2 (or later) this time around.

This shows the app running in minimized mode, or in cover mode as Sailfish calls it. Tapping on the button on it will open the app and open the first comic on the list in reader mode. The default sort mode for comics is to put the comics with least new pages first. In this case, that would be the second newest page on this blog.

This is the app in reader mode. Clicking on the button at the bottom opens a list of all pages in the comic and the navigation buttons work as you'd expect. In this case, the comic would have two unread pages (never mind that this is with stale data from my development environment). Tapping the forward button once would show the newest page in the archive and update the user's bookmark. Another tap would mark the whole comic as read and open -HEID for reading.

I've had all sorts of fun with connecting signals with all the ways how subscriptions can change and that all seems to work now. All together it's at over 4000 lines of QML markup and code. The main feature still missing before pushing this to openrepos and Harbour is bookmark storage and handling for unregistered users. I'm hoping that it would make the app more approachable towards new users to not insist on coming up with a user name and a password right away.

I still haven't tried the app on an actual phone. I have one but at this stage it's easier to just run it in the emulator. The WebView component used by Sailfish is pretty old and I hope it won't cause trouble with viewing comics.

It's been a learning experience but it wasn't too hard to get started with Qt. It's been a long while since I've last used C++ but it wasn't too hard to pick it up again. Somehow the language had gained lambdas since I last looked at it.

Once the dust has settled somewhat and I've seen what the reaction's been to my app (hopefully not nil) I'll start looking into making an Android version. I'd expect it to be a lot smoother sailing than with the initial Sailfish version. Not that this was all that difficult either, it just took some time to get here. I'm not promising any time frames for that version and I may need to take a bit of a break.

If anyone's interested in providing translations for the app then you're welcome to. I'll maintain the Finnish version myself.

submit to reddit
Sun, 10 Mar 2019 20:53:10 UTC