Aggregator Fixed (Finally!)

The aggregator should be updating every hour again. Apologies for the ridiculously long delay in getting it working. If you are really bored or geeky and you want the technical lowdown on why it was so much of a pain to fix that my ADD brain couldn't focus long enough to get it going until now, read on.

Usually when the aggregator breaks it's because one of the feeds we aggregate has found a new and clever way to be non-compliant that I hadn't thought of yet. Because of Cheetah's not-so-great debugging facilities, the way I've usually dealt with this was to delete feeds until I found the culprit, then examine the offending feed to figure out what was wrong/different, and code around it.

This time, that didn't work. For some reason, several of feeds were giving the same problem. The maddening thing was that the error I was getting indicated that the date attribute for some entries was not being found. However, when I put code into the aggregator to print out all the entry dates, it printed dates for every entry! This was maddening.

After lots of futzing around and investigating the way Cheetah looks up attributes, I noticed something: if an object behaved like a dictionary, and the dictionary had a key that matched what it was looking for, it would use the dictionary interface, otherwise it would use the attribute interface. This was an "ah hah" moment for me. Mark Pilgrim's feed parser can be used as a dictionary with keys or as an object with attributes.

What I pass into the template is a thin wrapper around entries from the parser that handles a couple of things and passes the rest on to the parsed entry. The wrapper *should* get any attribute requests first and then pass any it can't handle on to the entry object. It turns out in this case that Cheetah's NameMapper was calling the has_key method, which my wrapper wasn't handling, to see if the 'date' key existed as a dictionary entry. Sure enough, some entries had sprouted this key. When it found it, it was calling getitem to get the value of the key rather than doing an attribute lookup. The template was getting entry['date'] rather than! To fix the problem, I added a has_key method to the wrapper and made it always return false for 'date'.

For some reason the aggregator still fails with the C version of NameMapper, but I'll investigate that later (maybe sometime next year :) )

Share this

Those are probably just

Those are probably just fancy words for the code equivelant of: "I bashed it with ma wrench and it got fixed" or along that line. You sly cat....

I have no idea what any of

I have no idea what any of that means, but way to go, Sean.

Naughty, naughty, must check

Naughty, naughty, must check for existence before using... ;-)

re: ...probably just fancy

re: ...probably just fancy words for the code equivelant of: “I bashed it with ma wrench and it got fixed”

I actually DID understand that (I'm a fairly experienced Python programmer), and, yup, that's pretty much exactly what it means :>