BlatherSource

IntelliJ, why do you insist on bleeding memory relentlessly as I use you?  I should not see your usage increasing by 3 megs every second.  Sure garbage cleanup is taking care of it, but geez...

Supposedly this was supposed to be fixed in 7.0.3.  I'll get back up with JetBrains when I have the patience to do so.  Generally it doesn't seem to cause any real problems, it's just damned bizarre!  See attached movie clip for fun.  ;)

You may have seen, in the past, my podcast about External vs Internal Gateways.  In that podcast I talked about a number of benefits that internal gateways have over external gateways.

It's not all peaches and cream though.  You see, the problem with being so close to the internals of the server with internal gateways is that you tend to hear and trigger far more than you intend.  I'm going to pick on Openfire a little here since that's what the IM Gateway plugin is using.  Some of these may simply be something that should be fixed in Openfire.  Being one of the developers of Openfire, yes, I will investigate them when I can.  =)

One of the issues is, when a roster item changes, I actually get a number of listener events that tell me this occurred.  It's difficult to know what the proper event is.  In some cases, I'll get an event with no groups and then another event with groups.  Hard to know which one of those is true.    All things that I need to run through and make sure I'm doing things correctly.

Likewise, any time I tweak something about a roster item, I get events telling me it's changed.  Yes, I'm aware, I just did them.  =)  So in that case I need to ignore events.

It's all very strange actually.  I switched to listening for packets for real IQ packets coming from the client.  Guess what, there's a lot of IQ packets that come from the client.  So that's not really making things easier.  In XMPP, groups don't mean as much.  Removing and readding groups is nothing big.  In AIM and friends though, it's dire.  You remove groups and you've moved things around on your roster a lot.  This is primarily because in XMPP, you have a roster item, and an attribute of it is what groups are in it.  In AIM you have groups, and an attribute of a group is who's in it.  (for nitpickers, I'm referring to attribute here as "something the entity contains",  not attribute in the literal XML form of the word)

So if you were thinking, wow, internal gateways sound so easy!  Think again.  =)  There's a lot that goes on behind the scenes that's more difficult to deal with than you might expect.  So to sum it up, I'd like to simply say, there's pros and cons of both.  =)  Don't think an internal gateway is a magic bullet.

I've been quiet for a bit.  Most of you know how blogging goes ... you get excited about it or have a lot to talk about for a time, and then you start doing less and less posting.  Though some of you are quite adept at regularly posting ... to you I give the virtual thumbs up.  =)  But what have I been up recently?  Quite a bit actually.

For those who don't know, I left my university job of 11 years at the end of October and joined Jive Software.  My work with Jive fell right in line with "what I really want to do" for my career.  =)  Plus, they're all really great people.  That said, I'm working from across the country (waves) and quite enjoying it.  Maybe Jive will open up a Raleigh-Durham (Research Triangle Park?) office at some point.  ;D ;D ;D  Anyway.  I have continued my IM Gateway work, become one of the primary developers of Openfire, and become the lead developer of Spark (amongst other things).  I've been enjoying all of these things quite a bit!  I don't get to spend a lot of time on Spark, but when I do, it's quite fun to be playing with the client side again.  It's been a loooong time.  At this point I get to play with all three of the main areas of XMPP (Client, Server, and Component)!  It didn't take that long to adapt to Jive's environment though.  I actually really dig more deadlines and scheduled releases and such.  Frankly, it helps me mind "pick a target" when I'm trying to decide what to work on next.

So what's happened to PyICQt and PyAIMt?  I've officially stepped aside.  I moved them to Google Code so that others can more easily become involved.  You can now find them at:

http://code.google.com/p/pyaimt/

and

http://code.google.com/p/pyicqt/

I haven't removed myself as one of the "owners" yet on there so that it's possible for me to facilitate others becoming involved in the project for the next month or so in case others have been dying to get involved and didn't think about approaching me before.  A couple of other folk have stepped up and are doing a few things with it!  Yay!   The new location of the py-transports "mailing list" is now a Google Group at http://groups.google.com/group/py-transports.  I must admit, at this point, I don't miss them.  I do hope that they keep going though as I do believe they are good projects.

What about the XMPP Script Repository?  Well, I moved that to SourceForge: http://xmppscriptrepo.sf.net/

I won't be actively managing it, but if you are interested in becoming involved with it, please ping me and I'll set you up!  (let me know what your sourceforge id is)  It's using a SourceForge tracker to handle new script submissions.  If people -do- submit things to it, that would be about the most maintenance it needs unless you really feel like doing some significant work with it.  So... lemme know!

So what's left of BlatherSource?  You're "looking at it".  All I'm using BlatherSource for at this point is my blog.  I may do something else with it at some point, but for now it's just my blog.  I did like the interface I had put together for it, but I just don't have time to keep developing it/to manage it, ya know?  Many kudos to my hosts at Modevia.com for putting up with my requests for the site.  =)

I do still have a site for JWGC, http://jwgc.sf.net/ (moved it to SourceForge).  No idea if I'll ever do much else with it, but I like to keep it around.  It's my first XMPP related project ever.  =)

So... why did I move PyAIMt and PyICQt to Google Code instead of SourceForge?  Well, mostly because I wanted a wiki for the documentation and I couldn't find one on SourceForge.  Of course, an hour after moving everything I found that SourceForge -does- have a wiki.  Color me blind...  Doesn't matter, was neat to see what Google Code offers in terms of services.  I really like their "easy on the end user" issues tracker.  I don't like their web space offering though.  Spaces I think it is?  Whatever it is I don't like it.  I couldn't do with it anything like what I wanted.  I hear they are looking into a better solution at some point, so that might be cool.

After looking around relentlessly or a Java OTR library .. something like libotr that Pidgin uses, I decided to request a project on SourceForge to write my own.  Hopefully others will find use in it.  It's not approved yet, but if it is, it'll be jotr.  (http://sourceforge.net/projects/jotr ?)  If anyone is interesting in helping me out with that, I would greatly appreciate it!  I was rather surprised to see nothing already exists for Java for OTR.  (then again, I could have just not found it, but I tried really hard to find it  ;D  )

The IM Gateway plugin ended up putting me in the lead developer role of Java-JML (http://java-jml.sf.net) (MSN messenger library for Java) and also a developer of Martyr (http://martyr.sf.net/) (IRC library for Java).  Things sure do pile up quick, don't they?  =)  Those have been fun though.  I've gotten to work with Smack a little more as well in my Jive work, so that's been cool.  At this point I feel like I'm touching just about everything!  lol  Who knew that I'd go from being irritated with Java to enjoying working with it so much.

Anyway, so that's what my life has been lately!  New job, new adventures, letting go of some of the old ... moving forward.  =)  Yes, I do still have some XEPs to write/propose on the backburner.  I just haven't gotten to the point where I need to write them.

You know, when you are working by yourself on an open source project, your schedule is your own.  If you decide you don't want to work on it for a couple of months, that's your perogative.  At some level there's no rules what-so-ever aside from not doing things that will drive folk away from being interested in your project (assuming you care).  One of the biggest things that drives the project, however, is what -you- want.  When you are working on an open source project where you are teamed up with others, there's some checks and balances over what you do and your partners do.  However, at the end of the day, it's the same type of situation, you all decide what the focus of the project will be.  I don't like to put it like this, but no one has a "right" to your time except for you.

When you start working with a company on an open source projects, things change.  There are deadlines that the company is pushing for help drive your own timeline.  There are things they'd like to see happen that you may or may not agree with.  Instead of saying "well unless you submit a patch, it's not happening", you tend to work something out instead.  It's an interesting adjustment to "normal open source projects".

Do I dislike either?  No.  In fact I've quite enjoyed the experience of it.  It's taught me a few things that I wasn't aware of.  Like I was rather blind to some of the requirements that corporations ask for.  Some of the things they ask for I would typically have thought "that's ridiculous" with some of my other projects, but here I see a lot of folk bringing the same issue up and I start to discover that the issue is more commonplace than I would have assumed.  I'm not citing any examples here.  Just suffice to say there's things that having a broader knowledge of the corporate world has made me reconsider some of the decisions I might have made with other projects in the past.

So many thanks to Jive Software for giving me this additional experience that I wouldn't have gotten probably with PyAIMt and PyICQt.  =)  It's been fun and I hope it continues to be fun!

You know, when I first became interested in Jabber/XMPP, the thing that drew me to it so much was the concept of the transports to AIM, ICQ, etc.  Why?  Because it meant, instead of having to remember my passwords and accounts every time I switched to a new client, I could now store all of that on my server and log into one single location, my personal Jabber/XMPP server.  In general, this has worked out well, I do not log directly into AIM/ICQ/etc unless I'm testing something or don't have another choice at the time (perhaps my transport is broken, or something of that nature).

That said, lets take a look at my Adium X list of accounts at this point.  Instead of having 5 different protocol accounts listed, I have around 6 Jabber/XMPP accounts.  Home server, another external server, university private server, university engineering server, livejournal, google talk, friend's server.  So what happened here?  My original goal was to have one and only one server to connect to.  Is it because there's no XMPP transport/gateway as of yet?  No, if that were the case I would have written one by now.  Is it because a lot of these servers do not have s2s enabled?  Some of them, yes, that is the case, but that's not why I use all of them.  I can't answer why I'm now connecting to -more- accounts than I was before, completely defeating my original purpose.  I also used to have a jabber.org account listed there.  At least I pruned a couple.

Does it bother me that I log into so many accounts?  Not really.  It actually helps me keep some things in perspective while I'm developing... like the fact that I'm not the only one who does this and I should support multiple accounts in clients I might write.  It does, however, limit me from being interested in clients that only support one account to connect to.  Note that this is one of the other big reasons for wanting to centralize everything.  If I ran into a client I liked that only could connect to one server, I could use it and be happy with it.  Eh.

So anyway, no takers so far on the "anyone want to help with the Pys?".  Oh well.  I'll get back to them at some point.

In working with the IM Gateway Plugin, there has been a pressure to try to improve the way transports are handled in general.  I've begun the process of seeing what could be done to improve things, but one of the big sticking points seems to be not having the transports in your actual roster.

Now many clients provide an option to simply hide transports from your roster.  IMO, this is not a bad way to handle it.  Leaves it up to the client to decide whether to hide things or not.  It also allows for the end user to decide whether or not to see the transports or not.  I, for example, like to be able to see the transports and how they reflect when I'm logged in, what my status is, etc.  (because, for example, AIM only does available and away, none of the other statuses, and this is neat to see in my roster because I know that AIM folk don't know that I'm set to do not disturb)  It also allows clients like Psi and such to provide me a way to right click and choose log out.  (equiv of sending a directed presence packet)

So how would you go about having a transport detect that you've logged in if it's not in your roster to be telling you so?  Well you can cheat for one.  The IM Gateway Plugin allows me to use internal listeners.  This solution, of course, isn't going to help external transports a lick.  But for the plugin, it's nice.  Another way might be to require some sort of ad-hoc command to log into legacy services.  Like instead of auto-logging in, you have to select "log me in".  But then how does it detect status changes and such?  More ad-hoc commands?  Eww!  How many freakin' ad-hoc commands does it take to screw in a lightbulb.    Another way could be to create some sort of 'trust' relationship with a transport.  So a transport could indicate to it's attached server that it wants to be notified of any presence changes.  Not so sure about that either.  I think that was already addressed a long time ago in regards to transports wanting to interact directly with a user's roster.

Long story short, I think this will be fine for the IM Gateway Plugin to ditch the roster, as it apparantly confuses some folk to see transports in their list, but I don't think this is viable for other transport implementations.

That said, some issues have come up that -should- help other transports.  I'd like to, for example, write up an XEP that focuses on how to handle user preferences.  I'm not talking things like, "what theme for your client" but rather things that are handled on the server end.  For example, mail notifications.  AIM has them.  Not everyone wants them.  User should be able to go in and choose "no or yes" for mail notifications.  =)  These are enabled or disabled on the transport end and a client shouldn't be expected to have to be bothered with ignoring lots of things it doesn't want in the first place.  This is basically an ad-hoc command type of thing, but I'd like to standardize it or make it a "best practices" type of thing.  That way client devs could fairly easily go "oh this one supports user prefs, I'll provide an easy link for that".  Something like that.

Another one is a XEP that I had intended to write a while ago... basically something similar to Jabberd2's component protocol where the transport itself can connect and tell the server what jid's it wants to be.  This means that the transport could say "Hey, I'm aim.jabber, chatrooms.aim.jabber, and fries.aim.jabber" and the server could go "oh ok, well you got 'em" or "bugger off, those are taken".  Wildfire actually implemented this in a slightly different way than Jabberd2, but I think the same XEP could address both just fine.  I believe I originally called this "Extended Component Protocol" or something like that.  Who knows.  It's been a while.

On a side note, I think I figured out what was causing problems with Java-JML.  Basically I kept getting a severed connection upon logging in.  Turns out I think that the server Java-JML has for "fast login" is not a good one.  Theoretically you are supposed to connect to a nexus first and it'll tell you a login server.  I'm doing that now.

BTW, Thanks to the Cenqua peeps for providing me with a Fisheye setup for Java-JML!  I'm going to link that off the Java-JML site in a bit here.  =)  I love me some Fisheye.

Why do I often find it so hard to clean up my roster?  Just about anyone who has ever talked to me sits on my roster forever and I typically only end up talking to them for one or two sessions and that's it.  What do I need them on my roster for?  It's just causing my server extra work every time I log in!

So anyway, I spent a good couple of hours here cleaning up my roster and now it's nice and small.  I'm not entirely pleased with Adium X's roster deletion handling because just about every time I deleted a roster item, it reappeared and I had to delete it again.  =/  That said, it's a beta, so what do I expect.  =)

Logging in is now lightning fast.  I know some folk take offense at being removed from a roster, but if I removed you, this gist of it is, we don't talk much, I have no reason to need to know when you are online and/or initiate regular chats with you.  Certainly doesn't mean I'm not willing to talk to you. 

Anyway, was kinda fun to do the housecleaning.  There's a couple of legacy network contacts I still need to boot, but that involves well....  finishing adding support for that in the IM Gateway plugin.  ;D  or rather, fixing it

I'm thinking about spending a little time and packaging up PyICQt and PyAIMt into 0.8 releases.  PyAIMt is more work because it means I'll be pulling over updates from PyICQt to PyAIMt, but hey, no worries.

I got to try out an eval version of Jabber XCP.  I will finally have something to test XCP support with with the Pys!  Yay!  Setting up XCP was not entirely trivial.  It took me about an hour and a half and about 80 pages of documentation.  I'm not sure how much of this was postgres sucking or not, but it definitely wasn't as easy a process as I expected.  That said, now that it's running, it's lightning fast.  (well it's also not handling the load of my real contact list, but still)  Anyway, thanks to the XCP folk for hooking me up with a copy to test against!  I've always hated supporting something I couldn't test myself. 

I've been finding that some of these java based tools for .. well all sorts of tasks .. are quite nice.  I like Jive's forum software quite a bit, I adore Fisheye... far more useful to me than websvn ever was.  In fact websvn was never really useful to me.  I also adore JIRA, though I must admit it's a little too "serious" for something I'd probably want to use with my Py transports.  I love it for the IM Gateway plugin though.  It's got a really nifty feel to it.  Something else I think is really neat is that it looks like a lot of the the folk that make such products "play nice together".  By that I mean, I see Jive Software (Jive Forums) using JIRA and Fisheye, Cenqua (Fisheye) using Jive Forums.. don't know if they use JIRA, the JIRA people using Jive Forums ... etc etc.  Same with IntelliJ... it's just really neat to see them all using each other products.  It's like one big product hug circle.  =)

I'm basically in charge of Java-JML nowadays.  The former developer doesn't have time for it anymore.  Sourceforge is still kinda similar to how I remember it, but they do appear to have a few neat new things.  =)  I like the project and am certainly cool with being in charge of it.  Will give me a better understanding of MSN anyway.  I've applied for an open source license of IntelliJ of my own so I can make use of it with non Jive projects.  At first I was tentative about this because I didn't want to start using a tool that like the original developer of java-jml couldn't use, but I based off the license, I could indeed share it with all other devs on the project.  That's very cool.  More importantly, I saw that the joscar folk and other such projects are using it, which gave me a lot of "ohhh  other similar projects are doing it, ok then" warm fuzzy.  =)

While this weekend has not really started off on a happy note, I think I might make it a release day and try to get a couple of things released today.