Tickets list empty
Too bad that I didn't want to punch two holes to the opposite ends of Piperka and didn't have a handy river to divert. Piperka has, to date, had tickets opened on 1664 comics listed on it. The ticket system was launched at the end of May. During the last month, I've closed tickets on 1364 of them and performed 710 comic removals. There's only one ticket remaining where someone asked me to update a comic's archive a month ago but the site's disappeared since and I'm leaving it to remind me to check on it later to see whether it has reappeared or needs to be removed after all.
Most of my recent development efforts have gone to further improve the crawler web interface. It's not perfect yet but it was in a good enough shape to just get on with applying it instead of waiting for it to do the job even better and more easily and automatically. There's certainly room for making the crawler smarter about any update issues it may face and have it directly ask me to fix things that it couldn't solve by itself, but at this point I just ended up with a crowd sourcing solution. Thanks to everyone who ticketed Piperka and my apologies for taking this long to get hold of the situation. I just didn't have the tools to do a proper job about that part until now.
Getting the crawler back on track was only a part of what I needed to do as the archives may have reorganised or removed or added content. Trying to match users' bookmarks was another part of what I did. Sometimes the old links were dead and gave no idea of what the page content was and I had to go as far as to try to see what the pages were on archive.org. Catching the newly updated page from the archive head is just a part of what I want to do with Piperka.
There's just something poignant about seeing a comic stuck in 2010 to see new updates after resetting the crawler, only to have them end in 2014. Piperka's old. Long running comics may have user bookmarks sprinkled all along the length and I doubt I'll see most of them ever move again. I'm not publishing exact user counts but it's not difficult to tell that Piperka's user base has long been in a decline. I can hope that getting the basic functionality of the site to a better shape can stem some of that. I'll admit that I tend to find creating new features more fun than finding bugs to fix even if those annoy users more than lack of new features but at least the new features have finally made bug fixing less toilsome.
I would have liked to have some sort of intra site communications method set up before going through all the tickets but that would have been another couple of months' delay. I think you would have preferred to see comics update sooner rather than later. What's more bothersome is that I had to do some removals and it would have been better if you got a message about those that you read. Some of those even were active but weren't suitable for Piperka for some reason. Either they had some overly creative page navigation solutions or they used crawl preventition measures, like Incapsula. I doubt they're targeting Piperka specifically but I'm not going to go out of my way either to request them to grant access to my crawler.
Piperka's had some connectivity issues lately and they culminated last weekend when it was offline for hours at a time on a couple of occasions. First my hosting provider wanted to relocate the rack where my server was (if it was announced beforehand I missed it totally) and then they had a major power outage that knocked even their own web pages offline. They only give a 99% SLA which amounts to almost 15 hours of outage each month. Budget hosting can be like that. Looks like the network's been working better this week and I'm hoping that it'll last. I may or may not move back to Hetzner at some point.
I tried reaching out to /r/haskell to seek out contributors for Piperka but my text post only got instantly shadow banned. Gee, thanks. Finding some Haskell enthusiast seeking for a project who'd care about web comics seems more likely than hoping to find someone technically inclined and available from my own user base. Especially as it involves Haskell though I can think of plenty of improvements that would only involve Javascript. I took the effort to lay down the installation steps for both the backend and the crawler and provided a DB schema snapshot to help get a full development environment of the site up and running. I don't think anyone's yet tried it. I'm not sure where I would go ask for contributors next. Piperka's open source and I'm hoping that that would lower the barrier for hopping in and would make me asking for help less of a "work for me for free" thing. My next blog post may be one targeted at a wider developer community and not just to my users.
On a personal note, I'm on a leave of absence for all of November. It may or may not help Piperka. This is not a luxury I can take very often but it felt like the right thing right now.