Piperka blog

Mobile progress

I've been busy with coding the mobile client and I'm hoping to release a first version in a couple of weeks. I've just published what I have so far on Gitlab. I put it under GPLv2 (or later) this time around.

This shows the app running in minimized mode, or in cover mode as Sailfish calls it. Tapping on the button on it will open the app and open the first comic on the list in reader mode. The default sort mode for comics is to put the comics with least new pages first. In this case, that would be the second newest page on this blog.

This is the app in reader mode. Clicking on the button at the bottom opens a list of all pages in the comic and the navigation buttons work as you'd expect. In this case, the comic would have two unread pages (never mind that this is with stale data from my development environment). Tapping the forward button once would show the newest page in the archive and update the user's bookmark. Another tap would mark the whole comic as read and open -HEID for reading.

I've had all sorts of fun with connecting signals with all the ways how subscriptions can change and that all seems to work now. All together it's at over 4000 lines of QML markup and code. The main feature still missing before pushing this to openrepos and Harbour is bookmark storage and handling for unregistered users. I'm hoping that it would make the app more approachable towards new users to not insist on coming up with a user name and a password right away.

I still haven't tried the app on an actual phone. I have one but at this stage it's easier to just run it in the emulator. The WebView component used by Sailfish is pretty old and I hope it won't cause trouble with viewing comics.

It's been a learning experience but it wasn't too hard to get started with Qt. It's been a long while since I've last used C++ but it wasn't too hard to pick it up again. Somehow the language had gained lambdas since I last looked at it.

Once the dust has settled somewhat and I've seen what the reaction's been to my app (hopefully not nil) I'll start looking into making an Android version. I'd expect it to be a lot smoother sailing than with the initial Sailfish version. Not that this was all that difficult either, it just took some time to get here. I'm not promising any time frames for that version and I may need to take a bit of a break.

If anyone's interested in providing translations for the app then you're welcome to. I'll maintain the Finnish version myself.

submit to reddit
Sun, 10 Mar 2019 20:53:10 UTC

Towards mobile

I'm not a big user of mobile devices myself. This old thing is what I usually have in my pocket. It is much more pleasant to hold in it than a large smart phone and I'm not going to worry about any scratches on it. I do have a couple of smart phones as well but they're usually at home unless I know I need to access Internet on the go. Which isn't at all often. But I suppose I wanted to get to learn something new. This is what my browser's tab bar looks like as of now:

I'm making a native Piperka client app. I've pretty much just got to begin with developing it. I've only cursory knowledge of Qt previously which I've chosen for the job so it's a bit slow start. What I have so far is a test app which downloads the list of comics and shows it as a list. For Sailfish, of all things. Qt is cross platform and I'll target Plasma Mobile or Kirigami later on but I have to start with something and I'm just fond of the platform. I still have and use the original Jolla phone. I'll bring it to Android and perhaps even iOS later on.

I know of one Piperka user who used a Jolla phone (other than me). He tested how water resistant his Jolla was. Not much at all as it turned out. I may not reach all that many users on Harbour but I like the idea of how any apps on it stand out a lot better.

There has been an unofficial Piperka client for Android for a long while. This'll be a completely separate implementation from it. I've never used it myself and I don't know if it even works currently. I was contacted once after my backend overhaul a year ago about it being broken and I took measures to unbreak it but I don't think I ever got a confirmation that it worked again. I expect to break the unofficial client for good when I'm done with my client app and I'm hoping that nobody will mind it at that point.

I'll put my work in progress code on a public git repository once I'm a bit further along with it. I'd like it to have some basic features like logging in first. Once I have the basics in place I'll have to figure out how a Piperka client app could be useful.

I can't seem to resist the niche (am I even one to talk about them). I have bought a Purism Librem 5. I suppose it'll be released in time for me to build the client app for it as well.

I've a couple of new moderators. I'm grateful for them for volunteering.

submit to reddit
Thu, 31 Jan 2019 20:20:50 UTC

Update watch

The first new feature of the year is update watch. I've added a checkbox on the updates page and clicking it will set the page to automatically wait for updates. In practical terms, it'll wait for a signal from the hourly update run which will trigger it to reload the updates list. It'll even add an exclamation mark to the favicon to show if any new comics had been added to the list.

This feature went live a week ago already. I didn't announce it right away and I haven't checked from the server logs whether any of you noticed it yet. Looks like it's worked well so far. At least for me. With this, you won't need to hit F5 anymore.

I'm still mulling over the watcher's UX. When it's enabled, it gives no indication about what it does unless it finds updates. It does show an error message if it fails to re-establish connection and reconnects get a message as does the actual moment of downloading a refreshed list. A possible future feature would be to (optionally!) push desktop notifications about updates.

Other than the update watch, I've been working on the moderator interface. I'm storing past entries in a history table now and next up is some sanity checking for moderated content. As it is now, moderators can inject any content as comic descriptions and I'd rather safe guard that it's at least valid HTML.

Editing comic entries and moderating change requests just is something I've wanted to delegate to other people. I'd rather focus my limited time on things that'd demand a bit more than just basic HTML knowledge. "Other people" has been a few local friends so far but that hasn't really worked out anymore. I can't blame them since they've been volunteering for the task in the first place and I'm grateful that they've been at it for so long.

I'm calling for new moderators at this time. You'd get access to the moderator queue and get direct access to edit comic's info pages. Archive and crawler maintenance are still limited to just my own account. The interface for those is in a lot better shape than what it used to be but there's still a few quirks to its use and I think it's still better to keep it more restricted.

If you're a long time user who'd like to help me with this then drop me a message. Or even a fresher one but I'm more comfortable to give edit privileges to a name I recognize. I'll still want to add a few more checks to the moderator interface first but I hope to have a few names to hand roles to in a week or two. I'm hoping to get to hand off editing comics' info pages and not have to think about them much. Piperka's not about comic reviews and I'm not going to try to cater to anything like that but it would do to have a few words on the site about what a comic is about. I don't have much in the line of moderatorial guidelines. I've pretty much just held a policy of rewriting a comic's description if it's using first person nouns. It's fine for a comic's own site but that same text won't suit Piperka.

Speaking of communication, Piperka has an IRC channel on OFTC. Feel free to drop by on the webchat and join #piperka to say hi. I've been idling on the channel for a long while but I don't think I've actually advertised that fact anywhere. No wonder it's just me there.

I took a break from Piperka and wrote a patch for Heist during Xmas. Though it is still tangentially related to Piperka. I'm not about to explain what a monad transformer is on Piperka's blog.

submit to reddit
Mon, 07 Jan 2019 18:50:04 UTC

Personal recommendations

I've set up a personal recommendations page. This time I resisted the temptation for implementing my own algorithm and just used R's recommenderlab library as is, with default UBCF settings. As far as I see the results look pretty reasonable, though not particularly striking. It won't necessarily offer xkcd to you in the hundred results it shows which I consider a success but it's unlikely to suggest anything outside of the top 500 of comics either. I'm not going to win any Netflix prices with this one but it's good to have some baseline.

The only input data the algorithm uses is plain user subscriptions, with no consideration for anything like the date of addition. As such, it's unlikely to suggest anything particularly new. Currently, there are 33 comics with over 200 readers on Piperka and the last one to reach that threshold was Stand Still, Stay Silent which was added five years ago. There's only a handful of comics from the past three years that have even reached the top 500. Inactive users are dropped from the counts after half a year of inactivity so it's not just that disused accounts with old subscriptions on old comics are inflating the measures. It would do well to give a bias to comics a user doesn't necessarily know yet.

I think my per comic recommendations (with the totally custom implementation I wrote for it) does a better job at picking more specific results when it doesn't get suffocated by the strong nucleus of most popular comics. But I have no clear idea how to turn that into giving per user results and as much as I like having Piperka as my personal playground I'm not sure it's worth it at this point.

With respect to comic discovery, I would like to add overlays to Piperka Map. To color the comics listed on it according to some variables. Like from PCA. I would like to come up with some application for ANNs also. I'm going to need an embedding. I'm not a data scientist by any measure but they do have some cool toys.

In other news, I'm hosting an ad for December. Piperka's been adless since Project Wonderful's demise but I'm still considering what to replace it with. I'd rather have ads that target my site over more bids, with no user tracking or personalisation and there's no obvious choice for that after PW. Web comic authors are always welcome to advertise on Piperka as far as I'm concerned. I do get some pretty imaginative offers for ads from time to time, just by the virtue of running a web site but I don't think you'd care to read about gambling and what else. This is an one off thing at this time and I'm setting the ad up manually but I would like to eventually have something just as convenient as PW.

If anyone else would like to run an ad then just drop me a message. I'm afraid I'm still a bit rubbish at replying to queries, especially if I'm in the middle of something.

It's been a fun month but I think I'll content myself with leaving Piperka to maintenance mode for a while. I guess I'll play Half-Life: Opposing Force next, I've had it waiting for the right moment for a while. I'm glad they still make good games.

submit to reddit
Fri, 30 Nov 2018 16:36:15 UTC

Crawler health check page (mostly) empty

I've had more and better data from crawler's actions since I reimplemented much of it this fall but I didn't do anything new with it until now. A week ago, I added a view to Piperka that shows the issues it has found in the log. I made it so that it shows all the comics that have had errors when the crawler tried to find that next page during the last week and no successes during the same time. I didn't want to see every transient timeout but only those that have little chance of resolving on their own. I was in for a ride. I've removed 884 comics during the last week and reindexed or removed disappeared pages from Piperka's index and made the crawl run again for a good couple of hundred. I didn't keep an exact count of that latter group.

I didn't quite get the list empty yet as some of those reflect bugs with the crawler itself and not real issues with comics. And I still found a few cases it should have caught but it didn't, I'll just need to adjust my query a bit. Even in the best case, not all crawler issues will show up on my health page so there's unavoidably still an element of waiting for Piperka's users to report about any issues. But there should be much less of those now that I've turned the crawler finally to flag me before anyone even necessarily notices.

I get to see the date of last successful crawler action for a comic from the log too. It's invaluable to know when deciding what comics to eject or just to monitor. The base rate at which the crawler checks on a comic if it doesn't update regularly is at about once every 20 hours.

I still don't have any kind of messaging functionality built in to Piperka. Removals for comics that you read ought to raise some kind of notifications. That will still have to wait for another day, but I added something that should provide pretty much the same thing. My removed comics page lists any removed comics that you were subscribed to. I didn't necessarily look everywhere over the Internet for whether they had new homes somewhere. One thing I won't do is to make an entry point to a former site that still hosts an older copy of their archive with some message saying that new updates will be on a hence gone site.

I didn't even implement my idea about running the crawler to check on old pages to see whether they can find a known subsequent page yet. When I have that I should catch even more dead comics. Not all domain squatters are nice enough to return an easy 404 error for a former comic page.

Piperka's comic index has never been in this good state. I got curious and took a few statistics from the database: For a removed comic, the average count of pages is 189.6 and the median page count is 91. For live comics the respective values are 394.7 and 166. Not surprisingly longer running comics are likely to have a longer life.

You may have noticed that I've added a bit of styling to comics in listings that have more frequent updates. I experimented a long while with CSS styles until I settled for a white corner to mark the more active comics. I try to avoid information overload but this felt like a valuable addition.

I've been coding and upkeeping Piperka pretty much non-stop for three weeks. I could easily have ready plans for another month but I'll need to ease a bit for now. I'll consider later on what to do with the thumbnail functionality I implemented early this month.

submit to reddit
Sun, 25 Nov 2018 10:29:25 UTC