September updates
Looks like I got a bunch of new users just after I used a good part of my last blog post to complain about the lack thereof. I suppose someone went out to scout for them. I didn't quite mean it like that. I'll say that I'm grateful yet embarassed about this. I hope I'm doing something well enough with Piperka to keep you around.
I don't have any particular new feature to announce at this time so this is a grab bag of smaller things. 1st Sep update resulted in a considerable speed up with page loads. When the user is logged in, each one of them computed the total page count from the updates table for several comics potentially multiple times each time. It was pretty smart about using database indexes but it all still added up. Adding the expected updates table slowed the main page refresh further and I had a look at what I could do about it. I found a nice way to cache those results. I had tried to do a similar thing with the new backend from the start but that lead to some database deadlocks and didn't yield that much of a benefit after all so I rolled back it. Looks like I got it right this time. Faster page loads is a yay.
The mobile site version's been the default for mobile users for a while now. It's seen some slight adjustments since, thanks to everyone who gave me feedback on it. On a less cheerful note I noticed that I had broken logging in with the mobile app in June only on 5th Sep. I wouldn't want to find surprises like this but what's done is done. I had made a change and didn't test it properly. Nobody's sent me a message about it so I only noticed it myself later than what I'd like. But it works now again.
I've expanded the update frequency estimates to consider monthly schedules as well. As of this writing, 670 comics have a weekly schedule recognized, 46 have weekly update counts and 120 have monthly counts. I had a user contact me about this so I'll have a word about what they're about. I'm trying to figure out their current rates of updates, not what they've done over their whole run. If a comic has averaged to 3 pages each month over the years but has done 4 recently then that's what I show. If they, against all odds, start to update each monday at 5 AM then that's what I want to capture.
I've added some filtering options to the per user recommendations. I thought that newer comics are underrepresented in the results so I added the option to show only results among the latest 200 / 500 / 1000 / 2000 added comics. If you don't feel like sifting through all the newest additions then this may be of help. Even though someone still has to do that for them to show up as a recommendation at all.
I updated the crawler today to try one page backwards if the initial load of the last known page results an error or a redirect. Webtoons has been prone to getting stuck due changes to archive pages, I hope that this helps with it. It's a longer term goal to add more smarts to the crawler but this is something at least. It'd be cleaner to have all the compensatory actions and regular crawls separated and I've begun some work towards that but this latest change was added as a part of the regular crawls since it was simpler to do it that way at this point and this fixes some comics' updates now.
I've had some unnecessary personal drama going on lately and it's been a distraction. I'm hoping that I can set it aside soon enough and get to concentrate better on Piperka and other things I'd care more about.