Blogs

Ok maybe not. I've been quiet for a bit as I've adjusted to my new job and various other projects. Plus... my father passed away a little over a week ago... While I miss him, I am glad he is no longer struggling with his failing body. Anyway, the point of me bringing this up is, I've been silent because I've been involved in a number of things.

I'm in charge of a number of projects nowadays. I figured I'd take a moment to go through some of those, throw out some kudos, and let folk know how things are coming along.

Kraken
Kraken, the new life of the Openfire IM Gateway plugin, has been coming along. I've added Facebook, MySpace, and SameTime support to it, though those are still works in progress. (with thanks to the SIP Communicator folk, Pidgin, and the GSoC folk who worked on Facebook support for SIP C.) One of my "primary-ish" goals coming up soon is that I want to make it work with more than just Openfire. I've been working up the details of that in my head. Might be a bigger undertaking ... not sure ... but I think it'll be worth the effort. Kraken's first actual release was a little while ago to fix some budding MSN issues.

jQQl
There's a new QQ java API, jQQl, that Kraken is using that is primarily a port of LumaQQ with english docs, stripping out the client part, and fixes to make it functional again. Kraken uses it for QQ support. Many thanks to the LumaQQ folk for the project, even if it does appear to have been abandoned! =)

jMySpaceIMl
There's also a new MySpaceIM library for Java. It's not particularly complete yet but it's coming along. Many thanks to Version 2 Software for getting this started, along with Srinivasan, for getting this started before I took it over.

OpenYMSG
OpenYMSG was moved to hosted on SourceForge and is thriving there as a couple of talented developers have joined up. It had it's first release the other day.

JML
JML is an MSN messenger java API, which has been around a while and also which I took over a long time ago. It's also been thriving under the addition of a new developer!

jOTR
I actually created a project that was intended to be a java OTR library API, jOTR. The problem is, I no longer work on an XMPP client and have no use for this, so absolutely no development has been done on it. If anyone is interested in picking up the reigns and running with it, please let me know and I'd be happy to hand it off to you or just add you as a developer if you'd prefer.

Anyway. As you can probably see I've been involved in a lot of stuff. I've been working a lot with Drupal lately and wrote a module for it as well. I actually had someone express interest in my oldest XMPP related child, JWGC. =D Course that hasn't worked with recent Jabber servers in ages but still cool to see. Someday I hope to get the C programming bug and play with that a bit more.

Amusingly enough, I was all prepared to put up some projects for the XMPP foundation GSoC group this year and it turns out they're not doing it. ;D No worries, just thought it was funny on my timing.

So anyway, that's been Daniel's life lately. I've also been playing a lot of Phantasy Star Online - Blue Burst on the schtserv.com servers. =D Just throwing that plug in there. lol

A Quick History Lesson

You may recall that not so long ago, I switched my blog site from Serendipity to Clearspace. At the time I was working for Jive and was interested in having a real reason to "know" Clearspace and write plugins for it and such. That said, Clearspace is a completely beast for something so simple. Plus it required being run under Tomcat which I had little or no reason to run beyond for Clearspace. After parting ways with Jive, and not being interested in throwing money at a hosted server anymore, I moved my blog back to Serendipity and migrated the few posts I had made since moving to Clearspace to Serendipity.

The Now

A little while ago I realized that there was one thing I kept doing over and over again with my sites that weren't really "for something". Like for example, my personal site, our family site, etc. I would always do one of two things:

• Come up with a "cool design" that I liked and never get around to writing content
• Write out some small content that had no real design to it

A number of my friends are big fans of Drupal, and I certainly am not living under a rock, I had seen and played with it before, but without a purpose Drupal just seemed like "meh, yet another cms, what's the point". So having some extra free time last month, I decided to do something real with it --- whip up a family site (http://vorpalcloud.org/) that actually had some form of look and feel to it, but also had some content, and was trivial to update/add to in the future.

So in the process, I happened to see how easy it was to manage Google AdSense units within Drupal, and I was attempting to see if I could get enough from ads to have the site pay for it's own hosting somewhere. (as it turns out, not so much so far, but hey, if I decide to punt the ads at some point, then that's trivial)

I had a number of projects out there (JML and such) that had crap sites and so I whipped up some Drupal sites for them and was pleased how those came out. At this point I'm running something like 13 sites under Drupal, including this one. And I've written my first Drupal module: Daily Twitter. Overall I'm highly impressed with the number of plugins, the community around it, and the product itself. I was able to whip up this theme for BlatherSource in very little time. The only thing I'm a little dismayed about is email notifications seem to be oddly hard to set up or something. Maybe I just haven't found the right modules. Like I'd love to be able to subscribe to all content changes on a site but I can't see any way to handle that. Likewise I was excited about making use of the project stuff but that's not ported to Drupal 6 yet. But overall, I'd say this thing is fun to play/work with and has made it so I could whip up a large number of functional sites in no time flat.

So kudos Drupal! You made me enjoy managing my websites again.

How did I migrate from Serendipity to Drupal?

Well first off, keep in mind that I was the only user on my Serendipity site. It was my blog and my blog alone. I also never used the "extended body" stuff. But what it boiled down to is I did the following steps:

1. Created database for new Drupal site (on the same MySQL server as my Serendipity blog)
2. Ran the Drupal installer against the new database, but don't go to "View Site" at the end of the install.
3. Logged into the database as root.
4. "use"d the serendipity database
5. Ran the following SQL, knowing that drupal_bs is my Drupal database:
   
   INSERT INTO
   drupal_bs.node(nid, vid, type, title, uid, status, created, changed, comment, promote, moderate, sticky, tnid, translate)
   SELECT id, id, 'blog', title, '1', '1', timestamp, last_modified, '2', '1', '0', '0', '0', '0' FROM serendipity_entries;

INSERT INTO
   drupal_bs.node_revisions(nid, vid, uid, title, body, teaser, timestamp, format)
   SELECT id, id, '1', title, body, body, timestamp, '2' FROM serendipity_entries;

INSERT INTO
   drupal_bs.comments(cid, pid, nid, uid, subject, comment, hostname, timestamp, format, name, mail, homepage, status)
   SELECT id, parent_id, entry_id, '0', title, body, ip, timestamp, '2', author, email, url, '0' FROM serendipity_comments;

UPDATE drupal_bs.comments SET uid = '1' WHERE name = 'Daniel Henninger';

INSERT INTO
   drupal_bs.node_comment_statistics
   SELECT nid,timestamp,name,0,count(cid) FROM (SELECT * FROM comments ORDER BY timestamp DESC) q GROUP BY nid;

UPDATE drupal_bs.node_comment_statistics SET last_comment_uid = '1' WHERE last_comment_name = 'Daniel Henninger';

INSERT INTO
   drupal_bs.vocabulary(vid,name)
   VALUES(1, 'Serendipity Category');

INSERT INTO
   drupal_bs.term_data(tid,vid,name,description)
   SELECT categoryid,'1',category_name,category_description FROM serendipity_category;

INSERT INTO
   drupal_bs.term_hierarchy(tid,parent)
   SELECT categoryid,parentid FROM serendipity_category;

INSERT INTO
   drupal_bs.term_node(nid,vid,tid)
   SELECT entryid,entryid,categoryid FROM serendipity_entrycat;
   
   Note that I also changed all comment references to myself to my actual uid (1) in Drupal.
6. Done, go to site and all data transferred over.

I wanted to share this in case it helps someone else though.

Edit: This is not working out like it's supposed to. Still getting a svn-lock error when trying to init the sync, so while this in concept was a good idea, it's not "taking".

Pre-warning: Do not do this unless you know what you are doing, it will wipe the entireness of your SVN history for your project. It will be as if you just created your project fresh.

I wanted to svnsync an old repository to a new one on SourceForge. In my haste, I started setting up trunk/branches/tags without thinking. Of course, if the repo is not at revision 0, then you don't get to svnsync. So I pinged support to get them to reset my sourceforge repository. After a little bit, it turns out I effectively had the tools in my own hands the whole time!

I'm sure you well know that there's no way to wipe an svn repository completely down to revision 0 remotely. (at least none that I've seen so far) Well SourceForge has a nifty little tool (in your project, go under Admin, and subversion) to migrate your repository from cvs to svn. Well it didn't seem very clear on that particular page, but you can -also- migrate from an SVN dump to your new SourceForge repository from the same tool.

So how does this fit together? Basically what you do is you create a local svn repository that's completely empty:

> svnadmin create /tmp/myrepo

Then you get a dump of it:

> svnadmin dump /tmp/myrepo > svndump
* Dumped revision 0.

Then you gzip it (not sure if this is absolutely necessary):

> gzip svndump

Then upload that dump to your SourceForge file repository uploads directory, I used sftp:

> sftp USERNAME@frs.sourceforge.net
sftp> cd uploads
Connecting to frs.sourceforge.net...
USERNAME@frs.sourceforge.net's password:
sftp> put svndump.gz
Uploading svndump.gz to /incoming/U/US/USERNAME/uploads/svndump.gz
svndump.gz 100% 185 0.2KB/s 00:00
sftp> quit

Then you go to the migration tool that's mentioned under Admin -> Subversion -- Look for "Migration Instructions" and click on the migrate link under it.

You'll see the svndump.gz you just uploaded. Make sure to click the Replace checkbox and then click Submit. Wait for it to complete and you are done! Back to revision 0 for you!

I hope this helps others who run into the same issue!

I tend to see the same set of things done over and over again in clients as they go from "I only care about XMPP chatting" to "users keep asking us for better transport support". Many clients look for bare JIDs in your roster and treat them in a special manner, which is fine except in some cases I have bare JIDs in my roster and the service it's attached to is gone, so the client gets a little confused. Clients like Psi will dutifully look for jabber:iq:register and offer that as an option on services. Many of them you have to go disco browsing to find the transports available in the first place. Nothing wrong with any of that. Some clients may choose to go ahead and look for things marked jabber:iq:gateway and add them to some special list. That's not really what this is about though.

A number of clients will add in basically "hacks" for individual chat services. Psi has "service icon sets", and if it's a new service that isn't in Psi's list, it comes up with default icons. Spark has a bizarre mechanism in which it explicitly adds support for each individual transport via what I call little java "nuggets". But a common theme I see here is individually handling each service. Some new service comes along, now all of the clients have to be updated if they want to have cool icons. Personally I think that's crap. ;) I'm a big fan of dynamic lists of things. In other words a client shouldn't have to -know- about any of the specific transports. It should be able to simply pull a list of services on the server in question, if it wants gateway ones look for jabber:iq:gateway, and --- and here's the missing part --- be able to retrieve whatever it needs to provide a nice experience to the end user.

So what all tends to be custom added? Generally it boils down to icons. Icons icons icons. Icons for presence, Icons for representing the transport, etc etc. I don't think I've ever seen anything beyond icons that couldn't be determined from disco queries. So why couldn't service icons be provided by the service itself? I could see something along the lines of using pubsub where the transport publishes it's icon sets, one for service icons (maybe a set of different standardized sizes) and one for presence representation icons. That way a client could just come in and go "gimme the icons I should use" and move on. I admit my knowledge of pubsub is -still- limited, but my understanding is that there are version numbers associated with the published data, and the client could cache what it got and simply check for "is there a newer set?".

The primary caveat here that I can think of is, that pulls the 'control' of the icons away from the client and into the transport, and there's always a chance that the client doesn't "like" the icons the transport is sharing. Of course, for the most part the clients are aiming to use icons similar to those of the service the transport is providing, so they kind of already have to deal with the possibility that for example AIM's icon might not look great in their client. So the question is ... would client authors use this or just end up scoffing at icons coming from the transports themselves?

I suppose if the clients really felt like they needed to have different icons for some services, they could always override what they're getting from the transport and do their own anyway. It just always feels like a hackjob to me to see things like "if this JID matches with aim.* then use the AIM set". Especially those instances make me cringe because JIDs can be anything for those transports, where the transports -will- be providing a gateway identity that tells you flat out what service it is, assuming the transport authors bothered to follow spec. ;)

It's been a little while since I thought much about my illustrious IM Gateway plugin for Openfire. My previous job kept me a tad too busy with other things for me to really have time for it, or be interested in spending my free time on it. Now, let me prefix this by saying I don't have a lot of free time right now, and I may not for a while. That said, I had some time to step back and look at the IM Gateway plugin and think about some possibilities.

So first off, I don't really know the state of the Py*t transports. Last I heard there was no one maintaining them really anymore. That is a shame as I was hoping to see them flourish outside my involvement. If that's not true, then I certainly apologize --- just know it's because I haven't kept up well. Regardless, I found that I prefer Java to Python for the most part anyway. So would I take those projects back if they were offered to me? Probably not to be frank.

But the IM Gateway plugin --- The fact that it only works with Openfire I feel is a disservice to other server implementations, but overall it's a very cool implementation that tackled some issues with external implementations. But if you need to restart the IM Gateway plugin, you have to take the server with it. Java seems to have a habit of being very plugin-able friendly but then easy to break things with it's container. I'm seeing that with an application server I'm working with -- it's not hard for an app to break the application server.

So I began thinking about what core things the IM Gateway plugin does that the Pys did not. Primarily it revolves around direct internal access to user's rosters so that things could trivially be kept in sync. Things being, nicknames, groups, actual rosters, etc. So pulling back I'm wondering how feasible it would be to do something like this:

1. Strip away the ties of the IM Gateway plugin to Openfire and make the implementation standalone
2. Take that stripped away part and turn it into a small "helper" plugin that works with the external gateway implementation to accomplish the same goals
3. Similarly, write a small "helper" plugin for ejabberd that does the same thing
4. What about other servers I don't have interest/time to write helper plugins for? Well then the transports would act like any other external transport.

How would these helper plugins do their jobs? Well somehow they'd have to have open communication back and forth with the actual transports to manage rosters, maybe even intercept packets and munge them a bit. A little ugly but hey.

As you can probably tell, over the years I have never come up with some solution that could be formed into an XEP for all of this. At some level I continue to help that I'll see the light through my experiments here, but I'm not holding my breath. In the meantime I'd like to see solid transports that work for everyone. Writing an ejabberd plugin would give me a real reason to learn erlang and really see what it can do.

Now, if you are someone who is excited about this concept, please understand that I don't have a lot of time right now. I need to spend my free time on things that make me money for the time being and unless someone is dieing to sponsor this concept, I couldn't put it ahead of other assorted things I am working on. That is not a beg for money, that is a statement of my situation and why I'm not diving into this "right now". ;)

I would love to hear feedback on this btw.

So you may recall I posted about some problems I had with IntellIJ IDEA previously ( http://blathersource.org/blog/archives/73-IntelliJ...-Can-we-talk-about-this.html ). Since that time I had managed to repair the bizarre problem with it by... well completely whacking my entire home directory configuration of it. I use a Mac so it was in Library/Application Support and Library/Caches and maybe even preferences. That said, after cleaning all of that out and restarting IDEA, the memory leak weirdness had stopped occurring and all was well. Of course I had to go reinstall all my plugins and such, but hey. So hopefully this info will help others if they ever end up running into this same problem. So uhm.. Google index this please. ;)

You also may recall that I posted about trying out Eclipse ( http://blathersource.org/blog/archives/74-Trying-Eclipse.html ). I got a lot of great comments about it, pointing me in the right direction. Plus my previous employers had a group of folk who were highly into Eclipse and had some good advice. Of course, once I got IDEA working again it was hard to argue with myself to spend time getting used to Eclipse whereas that time could be spent coding.

So flash forward to no longer working with Jive Software (as of around mid October for those who are not aware). I no longer have a license for IntelliJ IDEA, so in working with another set of projects, I found myself firing up Eclipse again and guess what ... I adore it! I feel like it really lets me really customize it to how I want it to feel, I love the level of plugin support for various things in it, and free is great. ;D I find more resources with it though. If I run into a confusion point I usually can quickly google an answer. Plus it's not hard to find a chatroom or something to pop into to talk to real people "live". If I "let go" of what I'm used to with IDEA, I actually find it more intuitive. I don't really feel like going into serious details of what I like about it compared to IDEA, but suffice to say I am highly pleased with it and likely will stick with it for future java programming. =) I actually even though of writing up a document on things I learned while converting. (like terms that IDEA uses and what they map to in Eclipse) Who knows if I'll ever write that.

I've had a couple of people ask me why I did not reapply for membership in the XSF, so I thought I might post a bit about it here. I joined the XSF about 2 years ago not really knowing what was involved in membership. No one seemed to be able to tell me anything that being a member did aside from voting on the council, or other members. In the time I spent on XSF, that is literally all I did. It seemed like all of the "important discussions" had better places to be discussed at, like the council list, the standards list, and even a few more. I think it boiled down to, what's left to talk about or even do? I don't have any interest in being a member of something just so I can say I'm a member of it, so I chose not to rejoin this year.

If there was something to being a member in the XSF that I was missing out on, please, speak up and help me understand. =) And I certainly don't have any ill feelings towards the membership, so please don't take this as a "bash". ;)

When I first started trying out Java coding, I found it rather annoying. I was using vi and command line builds. I find that java takes a long time to compile compared to C and such, and hence it was a tedious and time consuming endeavour. Later, Matt Tucker introduced me to IntelliJ's IDEA (lots of people refer to it as IDEA, I refer to it as IntelliJ). At first I found it a beast and was resistent to it (being that my only IDE experience had been Visual Studio), but soon I began to adore it. I love the autocompletion, I like the automatic catching of compile bugs and such before even going to compile, and overall it made Java fun for me. In fact it make me actually wish I had a good IDE for Python at the time (I was working on PyAIMt and PyICQt at the time).

I'd heard of Eclipse, but I had not really used it before. I tried it once and couldn't figure out how to open a project and in general nothing about it made sense to me. I decided "why waste my time on this". Then came Flex Builder. As you may or may not now, Eclipse based. It's kind of like Eclipse with a lot of the cooler options in Eclipse stripped out. Generally I feel like it keeps getting in my way. Lots of keystrokes don't do what I want them to, in fact I can't -find- keystrokes to do what I want in a lot of cases. It seemed like autocompletion was a little "off". It wasn't as awesome as IntelliJs. It didn't always seem to work at all. Often I'd hit ., wait for a completion to come up, nothing, delete, try again, repeat until it really worked. Now, with so many people enjoying Eclipse, I began to wonder why. First off, IntelliJ ain't free, but it is for open source projects. It's also waaaaay bigger in terms of resource usage. So that might be one reason, IntelliJ is not easily accessible to everyone. But I figured, maybe Flex Builder is giving me a warped perception of it. I'd gotten some friends from work to walk me through some simple things like "opening an existing project" and such, so I know what to do at this point. =)

So at this point, I figured I want to give Eclipse a real try. I know it's less resource intensive. I decided I'd like to write down what I "require" my IDE to do at this point for me to like it and after I play with it some, see if it measures up to my pickiness:

• autocompletion of methods on classes, with it displaying args for the methods in the drop down box
• hungry backspace (if I hit delete I like it to wipe all of the whitespace in my way and get me back to something that's real content)
• instant evaluation of errors without having to compile first, ideally including warnings for things that "could be better" and "you realize you didn't set this variable to anything, right?"
• subversion integration, display of updated files compared to latest SVN checkout, easy commit of SVN files from IDE including picking and choosing what gets committed

Is that asking too much? Maybe. LOL I'm spoiled by IntelliJ. But I'm optimistic to give Eclipse a try and see how it feels, and if there are plugins to get what I want out of it. Would I switch to it as my primary IDE? Well who knows actually. This isn't necessarily an experiment to switch, but an experiment to see if I can learn to appreciate. =)

IntelliJ, why do you insist on bleeding memory relentlessly as I use you? I should not see your usage increasing by 3 megs every second. Sure garbage cleanup is taking care of it, but geez...

Supposedly this was supposed to be fixed in 7.0.3. I'll get back up with JetBrains when I have the patience to do so. Generally it doesn't seem to cause any real problems, it's just damned bizarre! See attached movie clip for fun. ;)

Yes that's right, Clearspace. Those familiar with Clearspace might be thinking to themselves ... are you nuts? Clearspace is a -beast- for just a blog! Well, you're probably right. However, I like it, I work with it, and I've been interested in writing some plugins for it for a long time now. The thing is, without actually running it live anywhere for myself, I never give any priority to writing said plugins. Now I have a solid reason to. =)

So what exactly -is- BlatherSource nowadays since I moved all of my projects away from it? Well, for alll practical purposes it's my playground. I continue to use this blog for posts related to my projects, XMPP, that sort of thing. I'll be running a number of services at this site. for example, I'm now offering public XMPP services here. If you are so inclined, feel free to register with blathersource.org. As the author of the IM Gateway plugin, there's a good chance this will be the first place I upload new versions for testing, and if a number of folk start using the service here, it'll help me get a window into what errors might show up. If folk start having problems that I'm having trouble dulicating myself, hopefully I can get said folk to register at blathersource.org and "demo" the misbehaving accounts to me and such. And just in general, if you are looking for a place to house your XMPP account, you are quiet welcome to register it here.

I'll post more details on the XMPP services at a later date, but anyone is welcome to register now if they are so inclined. I'm not sure what else I'm going to run here just yet. I got a general purpose dedicated server from Hosting And Designs. I've been quite pleased with it so far! I hosted my previous web services at Modevia, who were wonderful! However, I decided I wanted to run more services beyond just web services, and also needed to feed my old sysadmin bug since I no longer do that for my job. =)

Anyway, I'm excited for my new site. Who knows, maybe it'll get me posting more. Probably not, but it's wishful thinking.