Earlier this week I did a keynote at All Things Open. While the topic covered the opportunity of us building effective community collaboration and speeding up the development of Open Source and innovation, I also touched on some of the challenges.
One of these challenges is sustainability. There are too many great Open Source projects out there that are dead.
My view, although some may consider it rather romantic, is that there is a good maintainer out there for the vast majority of these projects, but the project and the new maintainer just haven’t met yet. So, this got me thinking…I wonder if this theory is actually true, and if it is, how do we connect these people and projects together?
While on the flight home I started thinking of what this could look like. I then had an idea of how this could work and I have written a little code to play with it. This is almost certainly the wrong solution to this problem, but I figured it could be an interesting start to a wider discussion for how we solve the issue of dead projects.
The Idea
The basic crux of my idea is that we provide a simple way for projects to indicate that a project needs a new maintainer. The easiest way to do this is to add a file into the source tree of the project.
This file is an .adopt file which basically includes some details about the project and indicates if it is currently maintained:
For example:
[Project]
maintained = no
name = Jokosher
description = An audio multitracker built for the GNOME desktop.
category = Audio
repo = http://www.github.com/the-project
discussion = http://mymailinglist.com/jokosher
languages = Python
[Contact]
name = Bob Smith
email = foo@bar.com
Now, this is a crude method of specifying the main bits of a project and much of this format will need tuning (e.g. we could pull out languages and frameworks out into a new block). You get the drift though: this is metadata about a project that also indicates (a) whether it is maintained, (b) what the key resources are for someone to get a good feel for the project, and (c) who the contact would be to help a new potential maintainer come in.
With this file available in the source tree, it should be publically available (e.g. the raw file on GitHub). A link to this file would then be pasted into a web service that adds it to a queue.
This queue is essentially a big list of .adopt files from around the web. A script then inspects each of these .adopt files and parses the data out into a database.
This database is then used to make this list of unmaintained projects searchable in some way. For example, you could search by category or programming languages. While maintained continues to be set to no the project will remain on the list.
When a suitable maintainer steps up and the project is alive again, all the maintainer needs to do is set this maintained line to yes. On the next scan of the queue, that particular .adopt file will be identified as now maintained and it will be removed, thus not appearing in the database.
A First Step
To provide a sense of how this could work I threw some Python together at https://github.com/jonobacon/adopt-a-project.
It is built using CherryPy to keep it simple. I wanted to avoid a full-fledged Django-type framework until the core premise of how this works is fleshed out. A caveat here: this is a really quick, thrown-together prototype designed to encourage some discussion and ideation.
It works like this:
- Run
website.pyto spin up a local webserver on127.0.0.1:8080that will display the empty queue. You can then add some remotely or locally hosted.adoptfiles by clicking the button at the top of the page. I have included three examples on GitHub 1 2 3. These are added toqueue.list. - You then run the
adopt-queue.pythat will scan the queue and create a sqllite3 database with the data. - The website then includes a really simple and crude list of projects and the links to the relevant resources (e.g. code, discussion).
Now, as you can tell, I have only spent a few hours knocking this together and there are many things missing. For example:
- It doesn’t include the ability to search for projects or search by language.
- The schema is a first cut and needs a lot of care and attention.
- The UI is very simplistic.
- There is barely any error-checking.
Topics For Discussion
So, this is a start. I think there are a lot of interesting topics for discussion here though:
- Is this core concept a good idea? There is a reasonable likelihood it isn’t, but that is the goal of all of this…let’s discuss it. 🙂
- If it is the core of a good idea, how can the overall approach be improved and refined?
- What kind of fields should be in an
.adoptfile? How do we include the most important pieces of information but also keep this a low barrier for entry for projects. - What should be the hand-off to encourage someone to explore and ultimately maintain a project? A list of dead projects is one thing but there could be instructions, guides, and other material to help people get a sense of how they maintain a project.
- Maintaining a project is a great way for students to build strong skills and develop a resume – could this be a carrot and stick for encouraging people to revive dead projects?
- What kind of metrics would need to be tracked in this work?
To keep things simple and consistent I would like to encourage this discussion over on the project’s issue tracker. Share your comments, thoughts, and methods of improvement there.
Thanks!
Perhaps an entry for why the project was abandoned. examples might include “lack of time”, “another project superseded this one”, “ran into a fundamental issue and didn’t want to redo everything to fix it”, “not enough testers/maintainers”, etc.
One issue I can think of with this idea is that, much like a gym membership, the project dies long before the maintainer/creator is willing to admit that it died. It kind of just stagnates for a long time. Perhaps the maintainer intends to continue, but it is only after much time has passed that he admits to himself that it is dead.
Great points.
One interesting solution to the gym membership side of things could potentially be tools for automating the detection of inactivity. For example, if a project has not been touched in six months, it is probably inactive. Just an idea.
Where would you automate this? At the VCS level? Project level? Externally?
Potentially a script at the VCS level (would need to be externally run, I suspect).
i feel this is a excellent concept with great potential.
i am thinking, however, that the process of stamping a project as “abandoned”, or in need of adoption, needs to be somehow automated. or at least, semi-automated. the .adopt file could be just one part of a larger system of identifying dead projects.
it’s generally not too difficult to determine the last date a project has been updated. using this date as a base guide a list of potentially abandoned projects could be automatically generated; say, 3 years without an update. this list could then be published on a matchmaker site, along with other basic information that could also (hopefully) be automatically extracted from the source files, such as languages used, program description, url of the code repository, last known maintainer contact, etc.
“lonely hearts” (people looking for open source projects to help with) could then search through this list and attempt to contact the last known maintainer to ask about the status of the project. no response within a set time (say 4 weeks) would signal a high probability of abandonment.
just some thoughts ….
I like the idea, but the problem I see is that most unmaintained projects get that way because the maintainer wanders off or gets busy… so there’s no one to add/edit the. adopt file!
your new project lacks a .adopt file 🙂
Good point. 🙂
This concept sounds a lot like Debian’s WPNPP/Orphaned Packages Process: https://www.debian.org/devel/wnpp/work_needing
I’ll add that if a project is orphaned its pretty much dead and hard to reactivate in these cases many of these projects just get forked. There are plenty of dead projects out their that are decades old and still have active repos but have not seen new releases and commits in years.
I think you raise a great point, Ben.
One of the things I have been thinking a lot about is the genealogy of a project, particularly when forking is a simple and encouraged premise to evolve it (such as in GitHub).
As I said, I am not sure my idea here even works, but the discussion is bringing out some interesting considerations. 🙂
The idea of the adopt file is good, but it requires the current maintainer to explicitly say he or she is looking for a new one. Sometimes this doesn’t happen, and also the current maintainer should already be aware of this project. So it could be extended in two ways: send an email to each project which last commit is old (define old) to propose the adopt file or add all the projects with old last commit as probably unmaintained. Also a name like “maintainer wanted” or “the old last commit” could be cool for such a project 😉
Have you looked at DOAP (https://github.com/edumbill/doap/wiki )? It includes similar information and is already used in gnome projects (among others, example: https://git.gnome.org/browse/polari/tree/polari.doap )
Cool, and still relevant. It would especially be interesting to look at some stats on github of project age vs. activity, especially for small projects (<5 contributors, maybe?) and any larger projects.
One advantage of github is the possibility – if you had a killer idea with this – that they’d stick it into some of their default helper features like the license suggester.
One non-starter: using your own data format. Never a good idea for a project you want wide adoption. DOAP is full-featured, but honkin’ huge XML. YAML or JSON would be the obvious ideas. Whatever file format has to be instantly consumable by any major programming language without thought by a new developer.
Similarly, echoing or getting a listing in any sort of package manager or foundation listing would make it much more likely to ever take off.
But this is a good concept – there have been a few stellar talks in the past couple years about the oral history of software development as a human endeavor which are fascinating.
Monktoberfest 2016: Bryan Cantrill – Oral Tradition in Software Engineering