Right now, if I have X hours to work on a project, I seem to be averaging what feels like X/2 hours in paperwork and other "important steps" of the development process. None of which have helped my project to ship higher quality, or more feature complete code. Which means either my or my employer's time is being wasted. I don't have time to waste when I have 108 bugs open in Gerrit Code Review, and 64 bugs open in EGit and JGit.
Based on a private email chain I'm having with the Eclipse IP review team, it looks like the initial EGit code contribution was bungled not just by myself, but also by the foundation's IP review process. Which means I probably have to run EGit back through IP review, almost from scratch. But only after I write a script to datamine contributors out of the old EGit history and inject a complete, per-file
git short-log into each file header. Its a good thing I have an awesome version control system like Git to keep these records for me. Too bad nobody else on the planet can use it to obtain information they might want to know about our source code. I guess running software to read information about a file is too scary for some individuals. So I have to do it for them. Now, and for every change we make in the future. Yay. :-(The astute reader may notice in that above paragraph, "private email chain" doesn't jive with other publications from the Eclipse Foundation demanding that projects be run in an open and transparent manner (see how do I start a project on Eclipse Newcomers). I really do feel like JGit is a less open project now that it has moved to Eclipse.org. Conversations with the Eclipse IP team about the legal status of any contribution is always discussed by private email. These things never make it to the project mailing list. The IPzilla database is closed to everyone but committers. There are backroom deals going on about what our file headers should look like in order to sufficiently convey that the source code is under the new-style BSD. The discussion that led to the approval of the EGit IP log for 0.7.0, approved despite what appears to be an error in the initial review, also happened by private email.
It took a significant amount of effort on my part to even get JGit hosted at Eclipse.org. Originally, the new-style BSD license wasn't permissible for a hosted project, and I had to seek a special exemption from the Eclipse Board of Directors. A process that required significant backroom conversations, over at least 6 months. Again, not exactly open. The only reason I think I haven't pulled the project back is because of the huge initial investment I've already made in this.
Maybe JGit and EGit are just unique projects. But in my experience, I am not a unique snowflake, and neither is my work. I'm not as special as I might seem at first glance.
I wouldn't be surprised if I've lost at least 2 days every month to paperwork. That's about 30 days, or 1.5 person-months since the project really started this move in January 2009. 1/12 of my time over the past year has just gone to catering to the Eclipse development process. Food for thought. Join Eclipse... make sure you pick up at least 1/12 of another full-time developer just to deal with the red-tape.
The part that really troubles me with the red-tape isn't so much that it is there, but that committers bear the brunt of the effort, while large corporations that are strategic members reap the benefits of having a concise change history listed inside of each source code file, or knowing that every contributor who ever touched this source code has been grilled in detail on a bug tracker.
So back to my post title. The real tragedy is, these corporations who sell commercial products based on top of Eclipse.org distributions are pushing not just the open source development work, but also a whole ton of onerous legal and reporting constraints back onto their project committers. Its enough to make this committer start to reconsider things. I wish I had been using a time clock this past year, to accurately record how many days the Eclipse development process has robbed me of since the start of all of this. It feels significant enough that if I went to my manager with it, I think he'd go ballistic.
Posts
12 comments :
Wow, that sounds pretty horrible indeed. It's understandable that the eclipse foundation's priorities are with keeping their sponsors happy, but surely there's a way to do that without screwing over non-corporate contributions? Best of luck, hope things miraculously start picking up soon.
Also, the iplog is a... pdf file with xml?
Yup, the iplog is a PDF file holding XML. The foundation is trying to move from an HTML based iplog to an XML based format. So we generated our iplog in XML, using code I wrote in JGit to mine the Git revision history. Someone told the legal team at the Eclipse Foundation that you can't edit a PDF, so they "freeze" the iplog by putting its contents in PDF. But they don't have an XSL to format it in human readable text, so we get this instead.
Apparently someone needs to take some time to write an XSL:FO stylesheet for the iplog XSD so we can render it through Apache FOP to get the magically immutable PDF in human readable format.
If only all lawyers that work in tech were engilaywers with a decent grasp of how things work. Then it could be _their_ responsibility to provide everybody else with a decent system (such as that XSL formatting and the script to mine the data they need). Bureaucratics need to not get in the way of software development :(.
[...] Another day. Another compliant from me about running a project at Eclipse.org. This time it wound up in the jgit-dev mailing list archives, as replies to a thread that I think started from my blog post on the tragedy of Eclipse. [...]
It is useful to try everything in practice anyway and I like that here it's always possible to find something new. :)
Being completely ignorant of the "Eclipse Development Process", could you give some examples of what kind of "paperwork" is required?
The Eclipse Development Process is fine for normal committers/contributers
Dumping a large chunk of IP into an open-source project is a Big Deal and its right that the normal legal due diligence process is followed?
I understand its tedious (I've done it myself) and be strung out for weeks but it really is the right thing to do
I'm an eclipse and egit user. The extent of my interaction with the Eclipse foundation is zero and I know where to go download the software and how to make it work for my purposes. So, forgive me if I sound like I don't know what I'm talking about. I most likely don't... That being said, is hosting your project at the Eclipse.org side necessary? Couldn't you just cut the crap, fork your own stuff, and host it somewhere else? Host it at github, perhaps? Is the Eclipse seal of approval a prestige thing? I just want to understand why you have to put up with that crap when there may be alternatives.
Comment 5 from Patrick Mézard:
> Being completely ignorant of the “Eclipse Development Process”, could
> you give some examples of what kind of “paperwork” is required?
Read the "Eclipse Development Process" document I linked to above, but in a nutshell the foundation requires documentation proving each change was intended to be contributed to the project. This needs to be documented before the change gets committed to the source code repository *AND* again before the product can ship a release. It also needs to be documented in the source code history, in the header of each file the contributor modified, and in an XML file which is used at the release review.
So, make sure you fill out that contribution data in triplicate.
For most small bug fixes from non-core contributors to a project, the "paperwork" involved can be significantly more than the actual code change. And in most jurisdictions, copyright of the small bug fix might not even be valid.
Comment 6 from Anonymous:
> Dumping a large chunk of IP into an open-source project is a
> Big Deal and its right that the normal legal due diligence process
> is followed?
>
> I understand its tedious (I’ve done it myself) and be strung out
> for weeks but it really is the right thing to do.
I fail to see why its a Big Deal. Maybe its because I've worked for so long around open source. Companies that open source a lot of code don't make a Big Deal out of contributions. They just make the contribution and move on.
I think part of the problem right now is there is too much risk being assumed on the side of the project. If an employee of a company open sources a change that the company doesn't want to be open sourced, the company has no real recourse with the project... they have no business relationship. Their issue must be with that employee who open sourced something that they didn't have the rights to.
In communities like the Linux kernel or the Git C implementation changes get open sourced on a mailing list that goes to thousands of people, and to mailing list archives. Once its public it cannot be taken back. If that has caused damage to the company, it shouldn't be the responsibility of Linus Torvalds or Alan Cox to clean up all of those mailing list archives and user inboxes to "protect the IP". No, the best the company can do is sue the employee for damages, and the kernel team will revert any code from the kernel that may have been applied and rewrite it. But they won't go rewrite their entire history, nor do they need documentation in triplicate saying a change can be included.
So, its pointless. Its lawyers ensuring they have a job in the future to continue to review these documents. They aren't providing any real value into the process.
Comment 7 from Jonas Gorauskas:
> That being said, is hosting your project at the Eclipse.org side
> necessary? Couldn’t you just cut the crap, fork your own stuff,
> and host it somewhere else?
EGit and JGit used to be hosted elsewhere. We moved to Eclipse so we can have EGit bundled as part of the standard distributions that are based on the core Eclipse projects, that way users have GIt support "out of the box", no matter how they obtained Eclipse.
Unfortunately, the only way to get bundled is to host your project at Eclipse.org and run the project according to their rules. So we aren't doing it for us, we're doing it to make things easier for our users by making EGit more widely available.
Interesting post. I was just looking for the latest jgit jar file last night, and I could not figure out where to download it from. Is it possible to just download it or do I have to go through some weird gyrations with eclipse to get it?
Comment 9 by Bill Robertson:
> I was just looking for the latest jgit jar file last night, and I could
> not figure out where to download it from. Is it possible to just
> download it or do I have to go through some weird gyrations with
> eclipse to get it?
You can add an update site in Eclipse, http://download.eclipse.org/egit/updates to download the full EGit/JGit set and install it into the workbench. If you want just the JGit JAR, e.g. for embedding into other applications, the JAR itself is available from the update site too as http://download.eclipse.org/egit/updates/plugins/org.eclipse.jgit_0.7.1.jar.
Thanks for the direct link. I appreciate it. I don't use eclipse so that's easiest.
Post a Comment