New Message: How Manila makes me old before my time

[webmaster at userland.com]

A new message was posted:

Address: http://manila.userland.com/discuss/msgReader$1108

By: Matt Deatherage (frontier at gcsf.com)

In the spirit of helping UserLand understand why Manila gives me wrinkles and liver spots, I thought I'd share tonight's story.

We all hate spam commenters. Ever since disabling RCS on our server, the auto-spambots haven't been able to post spam comments, and that's good. They have, however, been able to post spam trackbacks. They don't care about nofollow; they keep doing it, and it tasks me and my users.

So, a few hours ago, I thought, "Hey, I have a little bit of time. I'll go fix trackback so that it won't accept trackbacks on posts older than ten days. This will be a bit of fun."

First, I set out to find the "date" verb that returns the difference between two dates. Then I looked more closely. Then I checked Matt Neuberg's book. I found out there's only date.sameDay - true or false if it's on the same day. Sigh. OK, so I use date.get and date.set to determine if a given date is less than ten days from clock.now.

Now it's time to make a trackback callback.

Oh, wait. There /aren't/ any trackback callbacks. I'll have to patch manilaSuite.trackback.respondToPing directly. I poke around all of the manilaSuite.trackback and system trackback scripts for a while to make sure this is where I want to patch, and it is. I just wrap the entire creation of the trackback entry in "if workspace.lessThan10DaysAgo(msgDate)", where msgDate is the lastUpdate field of the message receiving the trackback.

I'm notorious for getting the < and signs reversed in this stuff, so I test it with two sites on the server. As I watch, it's working - some spammer (probably for that mississippi river kard gayme) is trying to add trackbacks to really old messages every minute - from 2002, 2003, 2004, and they're all failing. Heavy sigh - success! After only two hours to write 12 lines of script.

Now I test it to make sure that /new/ trackbacks still work - and they don't.

Eeek.

I start digging through all the trackback scripts again, turning on debugging, stepping through things, looking at table after table after table. I eventually figure out that inside each messages trackback.outgoing.urls.[url] table, there are booleans describing if that URL has been pinged, if the site has been checked, and if the site supports trackback at all.

The "flSupportsTrackback" boolean is always getting set to "false", even though the URLs are for another Manila site on the same server. This is maddening. I'm thinking, "I can't possibly have broken this."

I eventually, somehow, find that this boolean is getting set inside trackback.discoverPingInfo. So, I set up the test code bundle and start stepping through it. It goes and grabs the HTML from the page we're trying to ping with trackback, successfully finds the RDF segment and trims out the rest, and finds everything it needs.

And then we get to this line:

 Bundle //JES 7/15/03: skip RDF snippets whose dc:identifier doesn't match the URL

And now I see what the problem is. Manila generates permalinks to date-based pages using anchors to identify the right spot on the page, like this. I don't like this style and never have, because on most browsers I've ever used, anchors don't work about half the time. The page loads and stays at the top, even if the anchored text is several screens further away. If the anchor text is on screen, the page doesn't move anyway, and I have to skim to find it.

Anchors are OK at times [1], but not for links that should be archived, IMHO. When I want to reference a message, I use its discussion group URL, like this [2]. Manila gives me a nice clean page with only that message on it, plus a ready-to-use blank discussion form. The permalink macro does not do this, and apparently can't be made to do this, but I don't care too much - I use the URLs I want.

Manila is putting the permalink macro in the <dc:identifier field in the trackback RDF. Since the subject URL of "http://friends.macjournals.com/mattd/discuss/msgReader$1300" does not match the dc:identifier URL of "http://friends.macjournals.com/mattd/2005/07/11#a1300", Manila decides that the trackback is invalid. It then continues the loop searching for the RDF after having discarded the "irrelevant" RDF. It doesn't find any further RDF snippets, so Manila decides that the server doesn't support trackback at all.

Or, after 3 hours of tracing, I can summarize thusly: trackback pings only work if the URL in the ping is the same as the permalink for the item.

I submit that this is a bug, and that at the very least, Manila should accept trackbacks if the URL is one of the (many) known formats that all go to some form of the same message.

I also submit that the permalink macro should generate discussion group URLs if the discussion group is enabled, and that there should be trackback callbacks so I don't have to patch manilaSuite to reject these spams on old requests. Some messages from 2-3 years ago literally have /hundreds/ of these trackback spams, and I never know it until I go back and see the messages, since manila does not notify anyone of new trackback pings.

(A utility to remove trackback entries dated too far after a message's last update time would be nice, too, esp. if I could run it on an entire Manila site.)

So, yeah - 3+ hours of work to write about 15 lines of script, including about an hour and a half of testing to prove I didn't break something because trackback only works with permalink-format URLs and I didn't have any idea this was true. This would explain why messages I've posted in other places don't generate trackbacks - I wasn't using the permalink URLs. Is this restriction documented somewhere other than in this message?

Now it's time to go home and go to bed. I want a cupcake.

[1] http://friends.macjournals.com/mattd/2005/07/11#a1300
[2] http://friends.macjournals.com/mattd/discuss/msgReader$1300

This is a Manila site.. http://manila.userland.com/.